Jones, watching the action from the stands ahead of taking charge on Monday, saw the hosts produce an accomplished display at the Northern Gas and Power Stadium.
Pools dominated possession from the off but spurned a great chance to take the lead when striker Rhys Oates was denied from the penalty spot following Ronnie Henry's 40th-minute handball.
It mattered little, though, as the hosts netted twice in the next six minutes.
Featherstone grabbed the opener, firing into the roof of the net after seeing an initial effort superbly saved by Jamie Jones, before Walker, restored to the starting line-up, turned home a second following good work from Oates.
Stevenage responded well in the second half, with Matt Godden twice rattling the crossbar either side of a Tom Pett header which went narrowly wide.
But Hartlepool, who also went close through substitute Padraig Amond, ultimately secured a deserved victory.
Report supplied by the Press Association
Match ends, Hartlepool United 2, Stevenage 0.
Second Half ends, Hartlepool United 2, Stevenage 0.
Foul by Ryan Loft (Stevenage).
Scott Harrison (Hartlepool United) wins a free kick in the defensive half.
Luke Wilkinson (Stevenage) wins a free kick in the attacking half.
Foul by Padraig Amond (Hartlepool United).
Attempt missed. Tom Pett (Stevenage) left footed shot from the right side of the box is close, but misses to the left.
Attempt missed. Steven Schumacher (Stevenage) right footed shot from outside the box is high and wide to the right.
Charlie Lee (Stevenage) is shown the yellow card for a bad foul.
Foul by Charlie Lee (Stevenage).
Nathan Thomas (Hartlepool United) wins a free kick on the right wing.
Michael Tonge (Stevenage) is shown the yellow card for a bad foul.
Foul by Michael Tonge (Stevenage).
Michael Woods (Hartlepool United) wins a free kick in the attacking half.
Attempt saved. Matt Godden (Stevenage) header from the centre of the box is saved in the top centre of the goal.
Substitution, Hartlepool United. Nathan Thomas replaces Lewis Alessandra.
Luke Wilkinson (Stevenage) wins a free kick in the attacking half.
Foul by Scott Harrison (Hartlepool United).
Corner,  Hartlepool United. Conceded by Michael Tonge.
Corner,  Hartlepool United. Conceded by Jamie Jones.
Attempt saved. Padraig Amond (Hartlepool United) left footed shot from the centre of the box is saved in the bottom left corner.
Matt Godden (Stevenage) hits the bar with a header from the centre of the box.
Substitution, Stevenage. Ryan Loft replaces Ben Kennedy.
Substitution, Hartlepool United. Josh Laurent replaces Brad Walker.
Attempt blocked. Lewis Alessandra (Hartlepool United) right footed shot from outside the box is blocked.
Lewis Hawkins (Hartlepool United) wins a free kick in the attacking half.
Foul by Michael Tonge (Stevenage).
Substitution, Hartlepool United. Padraig Amond replaces Rhys Oates.
Substitution, Stevenage. Michael Tonge replaces Dale Gorman.
Foul by Ronnie Henry (Stevenage).
Lewis Alessandra (Hartlepool United) wins a free kick in the defensive half.
Attempt missed. Liam Donnelly (Hartlepool United) right footed shot from outside the box is close, but misses to the left.
Corner,  Hartlepool United. Conceded by Fraser Franks.
Corner,  Hartlepool United. Conceded by Luke Wilkinson.
Substitution, Stevenage. Josh McQuoid replaces Henry Cowans.
Ben Kennedy (Stevenage) wins a free kick in the defensive half.
Foul by Liam Donnelly (Hartlepool United).
Rhys Oates (Hartlepool United) is shown the yellow card for a bad foul.
(Stevenage) wins a free kick in the attacking half.
Foul by Rhys Oates (Hartlepool United).
New Hartlepool manager Dave Jones looked on as goals from Nicky Featherstone and Brad Walker earned the hosts a much-needed win over Stevenage.