This is so interesting. You have a very similar view as my best friend and my brother. Both were bullied by their first crushes too, my brother was a big kid and we were really poor so bullied for both those things, and my best friend had terrible acne and the girls called him pizza face.
Even though they are both handsome men now, fit, charming, kind, funny - they still have extreme lack of self confidence and need for external validation.
Everything you said was all about the need for external validation as well, and what I've come to learn is that true confidence is not from external validation at all, it comes from achieving difficult things that YOU are proud of. Sure, external validation helps, but when that goes away, you're still left questioning yourself.
My boyfriend is a perfect example of this, as he's getting older (42 this year) and losing the looks/body that he attached his worth to, and he lost his job - he has zero confidence, even though he still checks all the boxes in my mind.
My mother was a model and always complimented my looks, I hated it. I shaved my head and became a tomboy because I just wanted to be called smart. What the hell do looks get you??? But, you're right, later in life, I was able to get away with things because of my looks and be respected in a different way but never, ever would people assume I was a kind, smart person. They assume I was a dumb bitch because I was hot.
There are multiple sides to all stories - and I think the key is to figure out what makes YOU feel confident and that will exude through you. If you love to cook, learn more shit and feel good about it - so that even if others don't like it you can be like - "well, I'm proud of it!". You're very clearly an intelligent man, reflective, aware - you sound a lot like my brother who is 35 and struggles with women, but then gets angry at them for not seeing past whatever shit they have as their ideal criteria. Thats a them problem, not a you problem. You just haven't met the person that sees your brilliance yet.
Try to find ways to feel good about yourself without needing it externally. My confidence went to shit over the pandemic and I have been building it back up. Learning a new language, eating healthy, engaging with people, getting back to writing. All those things start to compile, I hope you can find that too! You're already a rarity as a 36 year old male being able to write like you do - you're worthy, trust me - maybe you just need to change who you surround yourself with! BUT I'll bet it comes down to confidence <3