The home side dominated much of the proceedings in the first half and were denied by Newcastle Karl Darlow but finally broke the deadlock on the stroke of half-time through Timm Klose’s header.
Aleksandar Mitrovic equalised after 71 minutes but Dieumerci Mbokani restored the Canaries’ lead moments later with a stunning strike.
Mitrovic looked to have secured Newcastle a point with a late penalty.
But Olsson struck deep into injury time to condemn Rafael Benitez’s side to defeat.
The win moves Norwich four points clear of the relegation zone in 17th while Newcastle remain 19th, six points from safety.