1 You are not familiar with it
According to psychologists familiarity makes us find faces more attractive. Many people think their side view is not attractive because they don't see it often.
2 You are used to a certain angle
Most people are used to seeing themselves from a certain angle. When those people accidentally see themselves from another angle they believe they look bad, See why you look ugly in some mirrors.
3 The lighting conditions
In order for a person to see himself from the side he usually needs two mirrors. If the lighting conditions were bad then this might result in seeing an ugly reflection, See Why we look bad in sunlight.
4 You are looking at the wrong side
Human faces aren't 100% symmetrical. A typical person might have one side looking better than the other, See why we find more symmetrical faces attractive.
5 Your side view is comparatively less attractive
If your side view is comparatively less attractive than your front view then you might think your side view is ugly if you saw it right after seeing your frontal view, See why you look bad in photos.
6 Your beliefs about attractiveness doesn't match your features
According to psychologists a person will feel he looks attractive if his beliefs about attractiveness matched his facial features. If you unconsciously believe you belong to an inferior race then you will think you look ugly when you see yourself in the mirror, See why sunglasses improves our looks.