The Loons are, however, eight points clear of second-top Arbroath, who drew 1-1 at home to Stirling Albion.
Clyde were hammered 4-1 away to Elgin City and Annan Athletic won 1-0 away to bottom-of-the-table Cowdenbeath.
Edinburgh City and Montrose fought out a 1-1 draw in the capital, a result that gives the former a four-point lead over Cowdenbeath.
Darren Lavery opened the scoring for Berwick with a low dig from outside the box, but goals from David Cox and Josh Peters made it 2-1 to Forfar.
The Loons' Danny Denholm was sent off for a second yellow card before Jamie Bain's own goal made it 2-2.
Gary Phillips grabbed the winner at the death before Berwick had a trialist sent off.
Stirling hit the front against Arbroath thanks to Darren Smith's close-range strike, but Steven Doris squared it.
Clyde had a day to forget in Elgin, with Mark Nicolson notching the opener and Shane Sutherland scoring from the penalty spot before Ross Perry's own goal made it 3-0.
Sean Higgins pulled one back with a header, but Craig Gunn scored again for City and Clyde had Jordan McMillan sent off late on for a second yellow card.
Daren Ramsay scored the only goal as Annan won at Cowdenbeath.
Edinburgh took the lead through Josh Walker at home to Montrose, but Cammy Ballantyne drew the visitors level late on.
Table-topping Forfar came out on the wrong side of a five-goal thriller away to Berwick Rangers.