Pelly Ruddock's cross was turned in by Paddy McCourt to give Luton, who had won one of their previous nine league games, their first goal in 366 minutes.
Reggie Lambe and Chris Clements fired straight at keeper Elliot Justham as the Stags sought an equaliser.
McCourt then turned provider for Ruddock to smash in and seal victory.
Luton manager Nathan Jones told BBC Three Counties Radio:
Media playback is not supported on this device
"I'm a very proud manager today. I thought we were excellent today from start to finish. I thought we defended well, at times we were a little bit deep but we tried to play.
"We have to have a squad here that wants to work hard, that wants to graft, that has a structure, that wants to do things right. If we get that then we can climb the league and start to do things.
"We have to get a bit more know-how, a bit more fluency in our play and once we do that we will be heading in the right direction."
Match ends, Mansfield Town 0, Luton Town 2.
Second Half ends, Mansfield Town 0, Luton Town 2.
Substitution, Luton Town. Frankie Musonda replaces Pelly Ruddock.
Foul by Jack Thomas (Mansfield Town).
Cameron McGeehan (Luton Town) wins a free kick in the defensive half.
Attempt saved. Chris Clements (Mansfield Town) right footed shot from outside the box is saved in the centre of the goal.
Chris Clements (Mansfield Town) wins a free kick in the attacking half.
Foul by Alex Lawless (Luton Town).
Attempt missed. James Baxendale (Mansfield Town) left footed shot from outside the box is high and wide to the left.
Foul by Adi Yussuf (Mansfield Town).
Alan Sheehan (Luton Town) wins a free kick in the attacking half.
Substitution, Luton Town. Jack Marriott replaces Craig Mackail-Smith.
Attempt missed. Krystian Pearce (Mansfield Town) right footed shot from the right side of the box is close, but misses to the right.
Substitution, Mansfield Town. Jack Thomas replaces Adam Chapman.
Stephen O'Donnell (Luton Town) is shown the yellow card for a bad foul.
Matty Blair (Mansfield Town) wins a free kick in the defensive half.
Foul by Stephen O'Donnell (Luton Town).
Attempt missed. Craig Mackail-Smith (Luton Town) left footed shot from the right side of the box is close, but misses to the left.
Attempt missed. Adi Yussuf (Mansfield Town) header from the centre of the box is too high.
Substitution, Luton Town. Olly Lee replaces Paddy McCourt.
Attempt missed. Matt Green (Mansfield Town) right footed shot from the left side of the six yard box misses to the left.
Attempt blocked. Craig Mackail-Smith (Luton Town) right footed shot from the centre of the box is blocked.
Craig Mackail-Smith (Luton Town) wins a free kick in the attacking half.
Foul by Ryan Tafazolli (Mansfield Town).
Alan Sheehan (Luton Town) is shown the yellow card.
Substitution, Mansfield Town. Adi Yussuf replaces Reggie Lambe.
Substitution, Mansfield Town. Matty Blair replaces Mitch Rose.
Goal!  Mansfield Town 0, Luton Town 2. Pelly Ruddock (Luton Town) right footed shot from the right side of the six yard box to the top right corner. Assisted by Paddy McCourt.
Hand ball by Cameron McGeehan (Luton Town).
Corner,  Luton Town. Conceded by Sean Kavanagh.
Foul by Krystian Pearce (Mansfield Town).
Craig Mackail-Smith (Luton Town) wins a free kick in the defensive half.
Attempt missed. James Baxendale (Mansfield Town) right footed shot from outside the box is close, but misses to the left.
Corner,  Luton Town. Conceded by Krystian Pearce.
Second Half begins Mansfield Town 0, Luton Town 1.
First Half ends, Mansfield Town 0, Luton Town 1.
Attempt missed. Krystian Pearce (Mansfield Town) header from very close range is too high following a corner.
Corner,  Mansfield Town. Conceded by Jake Howells.
Foul by Chris Clements (Mansfield Town).
Cameron McGeehan (Luton Town) wins a free kick in the attacking half.
Nathan Jones enjoyed his first win as Luton boss as they picked up the points at play-off chasing Mansfield Town for the first time in a league match.