The Hammers were back at the London Stadium for the first time this campaign after an athletics meeting last month forced them to play three successive away matches.
And, having lost them all, there was speculation that birthday boy Bilic could be facing the axe if they failed to register any points from this game.
Promoted Huddersfield arrived unbeaten with two wins and a draw from their opening three games, while they had not conceded a goal.
And it looked like David Wagner’s men would take a share of the spoils away from the game as it drifted towards a stalemate with 20 minutes remaining.
But a heavily deflected effort from Pedro Obiang and a strike from substitute Andre Ayew with 13 minutes remaining sealed a much-needed win for the Londoners to move them off the bottom of the table and up to 18th.
Despite suffering their first defeat of the season, Huddersfield remain sixth on seven points.
The fit-again Andy Carroll almost created an opening goal for the hosts in the second minute when he brought down the ball on the left of the area and fired across the face of goal with the stretching Cheikhou Kayoute just inches away from converting into the net at the back post.
The Hammers continued to press and went even closer in the 13th minute when Michail Antonio burst away down the right before teeing up Javier Hernandez whose effort smacked against the crossbar.
On 26 minutes James Collins got up highest in the area to head down Aaron Cresswell’s left-wing free-kick which deflected off the back of a defender and was then saved on the line by Jonas Lossl.
Phillip Billing had the first Huddersfield attempt on 54 minutes but his low effort from outside the area was comfortably saved by Joe Hart.
With the second half lacking in clear-cut chances and looking set to end goalless, the Hammers took the lead in fortuitous circumstances in the 72nd minute.
Obiang picked up the ball just outside of the area and fired a shot at goal which deflected off the back of Mathias Jorgensen and looped into the net with Lossl completely wrong-footed.
The goal gave the Hammers a real lift and they went two-up five minutes later.
Jose Fonte saw his effort from Cresswell’s right-wing corner blocked on the line, but the ball rolled out to Ayew who made no mistake from close range.
Huddersfield almost pulled one back on 81 minutes but Tom Ince saw his effort from just inside the area take a slight nick off Collins’ and crash against the crossbar.