What's called egoism is from start to end a social phenomenon in which the emphasis is on one kind of self object to the detriment of other kinds of self object. It's best seen in others, and if in yourself then it is when you imagine your appearance from the outside. If there is a problem with it it falls under morality or etiquette, but it has no metaphysical significance. Self-objects of any kind are assumed and unexamined - since no such object can be its own ground, they don't qualify as self in the only sense that would count. It is quaint that a certain kind of behaviour is taken to be self-directed. First find out what this self is that you would favour and when you do then all question of egoism is irrelevant. How can you favour that in which everything appears?