Absolutely brilliant. I love your analogy. As a 60 year old who has tried to keep up with 20 year olds; it is definitely mostly physical. Also, if you aren't mentally into it, you are going to do well.
I think your problem is not so much that you're mentally holding back, it's that you percieve other players as unknown threats, which they are, for the most part. I would diversify your game. Hop in a rec league or play at some pick-up games to get used to playing different people. If you can, get in a premier league in the off-season. It sounds like you know your team and you know what to expect from them, so get more experience outside your team.
Obviously the game is more mental, that's why some teams start out slow- because they aren't focused, and others get a goal in the first minute- those who are mentally prepared. You can see this a lot with professionals. Young players like Wayne Rooney often seem unprepared and play poor and lose their temper. Others like Maldini and Ballack seem to show up to play every day. Also, at the high-school level, this is an obvious factor. The kid who scores four goals per game in practice often is unproductive in games, while the kid who was messing around in practice goes off four two or three in a game. For me however, the effect seems to be the opposite. How well I play in soccer often seems to effect how well I do in school, etc.