Middlesbrough lost ground on leaders Burnley as they were beaten by a late Lee Frecklington strike at Rotherham.
The visitors dominated the first half but David Nugent wasted a one-on-one and Gaston Ramirez struck the post from a free-kick.
Neil Warnock’s side improved after the break and defender Kirk Broadfoot headed just wide.
Frecklington stole in to grab a winner before the hosts had Greg Halford sent off in the 89th minute.
Boro remain second, now four points off the pace after Burnley won at Fulham, while Rotherham stay 22nd but are just a point from safety.
In a one-sided first 45 minutes, both Nugent and Ramirez gave the hosts plenty of problems – the England international heading wide from six yards while the on-loan Southampton playmaker was just off target with a dipping half-volley.
The second half was a largely dull affair until the closing minutes, when Millers skipper Frecklington arrived at the back post to shoot high into the net, before Halford was sent off for a second yellow card for a challenge on Stewart Downing.
Albert Adomah had Middlesbrough’s best chance to level, but his shot was charged down by a determined home defence.
Rotherham manager Neil Warnock: “I am enjoying it, the Championship is my thing really. I love the blood and thunder.
“That’s not my fault they missed chances, they were trying to help me out. (David) Nugent had a couple of good ones, you have to take your chances don’t you?
“It’s a great win for us, we have to scrap for everything, the extra two points are a bonus but we kept fighting until the end.”
Middlesbrough head coach Aitor Karanka: “We should have won by 4-0 or 5-0 and we have lost three points.
“We are second in the table, we have a really good squad so I am not concerned.
“They decided to win, they were fighting for every ball as if it would be the last ball for them. They are fighting to stay in the Championship.”