Hermes, you seem to be making a big and unsubstantiated assumption about other people’s ego.
That said, your position seems to be essentially Aristotle’s. The problem is that externals are not up to us, they depend on chance. And so does your happiness.
Focusing on internals, like both Stoicism and Buddhism counsel, by contrast, puts eudaimonia (what you really seem to be referring to, not happiness as normally construed) in your own hands.
The Stoics make an argument for why virtue is the crucial thing to focus on. They define virtue as prosocial behavior guided by reason. And given that the distinguishing characteristic of human beings is precisely that we are highly social and capable of reason, it follows that a good human life consists in the realization of our most crucial qualities.
Contra what you seem to be implying, however, a focus on virtue, or on reduction of suffering, does not mean that one is also not free to experience pleasure, pursue one’s goals, etc. It means that the latter shouldn’t be the all-consuming goal in life.