My experience on the subject.
So I’m near 50, and got into trials early this year after 30 odd years of road and road racing. Bought a 300 Ossa (yeah, I know) because I like the techno, I like having something different, and it and it was cheap.
First up, I really wondered what everyone was on about, 300s being so hard to ride. I found it a pussycat, bumbling around mountain trails.
I then bought a 125 gas gas something for the GF’s lad to ride with me. And about then I started heading more towards proper organized trials.
Well, straight away I found that I could ride the sections so much better on the 125. Throttle sensitivity, power delivery, all that was much easier on the 125. In short, I was finding that the little bike just forgave my errors much better.
Anyway, now about a year in (and riding 2 -3 times a week), I’m starting to go back to the 300. Ideally, I think a 200 would be the best all-rounder, but I’m not ever finding the big bike intimidating, just that it can be a little more demanding. I tend to train more on the 125 if it's free, and compete on the 300 (in S3 level)
So, I don’t think that it makes a huge difference for a mediocre competition rider, and someone just playing around on green lanes even less. If you want to progress quickly, it’s a minor handicap, but not something that really poses a problem.