Half of the men voted for Ser Alliser
During the elections Jon snow won by only one vote. Ser Alliser had many supporters who didn’t want Jon Snow to be in charge.
Ser Alliser Thorne hated Jon Snow
The Hatred Ser Alliser Thorne had for Jon Snow might have motivated him to plot to kill him.
He rescued the Wildings
Jon Snow ordered a mission to rescue the remaining Wildlings. This mission made many of his men question his loyalty.
Many men died during the rescue mission
During the mission to rescue the Wildlings many of the Night’s watch men died. This fact made Jon Snow lose some of his popularity.
Jon allowed the Wildlings to pass the wall
Many of the men weren’t happy with the decision to allow the Wildlings to pass the wall. (See Game of Thrones Season 3 Summary)
They had doubts about his loyality
Many of the men questioned Jon’s loyalty to the crows especially that he has a certain level of affection to the wildlings.
Olly did it for his parents
Olly ,who saw his parents getting killed by the Wildlings, couldn’t accept the fact that Jon Snow allowed them to pass through the wall. (See Why Game of Thrones Season 5 is bad?)
Jon had no supporting friends
The death of Maester Aemon and the fact that Samwell left resulted in leaving Jon snow with almost no friends at Castle black.