Hi - I'm similar to terrier above; 40+, mortgage, two kids, and just starting trials having never ridden off road before.
After quite a lot of searching around I use a Fox Titan Sport body armour jacket - lightweight mesh jacket with back, chest, shoulder and elbow guards. When its warm I just wear a T-Shirt over the top, and when colder a jacket. I don't find it restrictive and it gives me a good bit of confidence.
I also have knee pads that go under my trousers.
When I get better and more able I may reduce the armour but frankly I'd rather wear it than not! It's helped me bounce off a few trees and rocks so far so I'm happy!