When we first meet Olly, he is a likable character. He's just a little boy that had his parents murdered by wildlings on the orders of the King-Beyond-The-Wall Mance Rayder. He tells the Night's Watch about what happened to his village after a wildling let him go to do just that. Jon Snow steps up to mentor him early on. However it would be what he'd do later that made him one of the most hated characters on Game of Thrones ever.
Despite being the one who shot Ygritte, Jon Snow still favors Olly and tries to groom him for command in a similar fashion that Jeor Mormont did Jon Snow himself. As Lord Commander, Jon makes Olly his steward and has him listen in on important stuff such as a private talk between Jon Snow and King Stannis Baratheon.
However due to Jon Snow protecting the Free Folk, Olly betrays Jon along with Ser Alliser Thorne and other mutineers and stabs him in the heart.
Which backfires when Jon is brought back to life by The Red Woman, and has them all hanged for treason against The Night's Watch. One of the most satisfying demises on the show I think. It's a shame Olly stayed quiet when Jon Snow stared at him with anger and sadness.