Discipline. The one thing the Spurs backline surprisingly lacked at White Hart Lane.
Mauricio Pochettino’s side are the stingiest in the Premier League – having only conceded 20 times – but the defending was shambolic at times today.
There was Kyle Walker’s headed backpass that was clawed off the line by Michel Vorm, then there was the failure to track Joe Ledley, who was unlucky not to score.
The Spurs defence also stood still as Yohan Cabaye’s delivery flashed across the six-yard area. So perhaps Kelly thought this would be as good a day as any to end a 1,545-day scoring drought.
Like Ledley, he was left free to run into the area and convert a carefully-weighted ball from Zaha.
Key Spurs defender Toby Alderweireld was left on the bench for Sunday’s tie. Pochettino might think twice before doing that again, as his team focus success in the Premier League and Europa League.
Welcome boost for Palace
DDLLLLLDL – Palace’s last nine league results. The FA Cup has been a competition that has provided solace for Alan Pardew’s side.
The Eagles rediscovered that dogged spirit that saw them reach fifth in late December.
Unlike Spurs, Pardew elected to play his general in defence, Scott Dann. He held the defensive wall together as it faced a barrage of attacks that increased in frequency as the match wore on.
Hennessey was exceptional in keeping out two Kane efforts and Cabaye blocked an Alli header on the line. They also rode their luck in the first half when Alli’s sidefoot shot came off both posts before it was cleared by Joel Ward.
It is unlikely the Eagles will be dragged into the relegation fight, so perhaps they can increase their focus on winning this competition, which would be their first major trophy in the club’s history.
Crystal Palace manager Alan Pardew: “We’ve had a tough run, with a lot of injuries, but we have them back now and look like we’re back to ourselves.
“Wilfried set the goal up and he was terrific from that moment forward. We had some great performances out there. We learnt our lessons from two previous meetings with Spurs. We worked on shape and they delivered it.
“It was a victory that was well earned on the training ground.
“No-one’s ever won it without a little bit of luck. We’ve beaten three Premier League teams to get here. We hope the draw is kind to us.”
Tottenham manager Mauricio Pochettino: “It was tough. It’s like a Premier League game but you need to be clinical in front of goal. We managed the game, created chances but it was difficult to win.
“There was no tiredness. If we analysed the game we played better but you need to score. It’s hard to explain.
“It’s a tough period for us. We have a crazy schedule but now we need to look forward to be ready for next Thursday [v Fiorentina].
“It’s impossible to know if losing here will help us in the long ruin. Now we feel disappointed because we’re out.”
The stats you need to know
Kelly scored his first goal since scoring for Liverpool at Chelsea in November 2011.
Kelly is the fourth Palace defender to score in all competitions this season, along with Dann, Damien Delaney and Ward.
Zaha registered his third assist of the season, setting up Kelly’s opening goal.
Palace have won their last four away FA Cup games after winning just four of their previous 16 before that.
Spurs failed to score in a home FA Cup tie for the first time since March 2002, a 4-0 quarter-final defeat to Chelsea.
Tottenham are at home to Fiorentina in the second leg of their last-32 Europa League tie on Thursday with the score poised at 1-1, and Palace are away at West Brom next Saturday in the Premier League.
Have you added News Alerts in the BBC Sport app? Simply head to the My Alerts menu in the app – and don’t forget you can also add score alerts for your favourite football team, the Six Nations, and more.