One measure of that borrowing is used to calculate the annual budget deficit. Public borrowing was around £154bn in 2009-10 and is expected to fall to just over £90bn in 2014-15 - so we have seen the deficit fall by around a third.
The coalition government oversaw a generally smooth decline in the deficit. The Labour administration, which ran from 1997 to 2010, witnessed a very different path in government borrowing. Between 1997-98 and 2000-01, Labour reduced government borrowing and actually ran a surplus for three years (where the government received more money than it spent). However, between 2001-02 and 2009-10, the Labour government saw the deficit rise from £1bn to £154bn.
Looking ahead, the Conservatives want to eradicate the whole deficit, and even run a surplus, by the end of the next parliament.
Meanwhile Labour is chasing a different target. Rather than eradicate the whole deficit - covering all public sector borrowing - Labour hopes to eradicate only one part of the deficit: the current budget deficit.
This measure of the deficit only looks at borrowing to cover day-to-day spending. Under Labour's plan, the government would still be allowed to borrow money to invest in long-term projects, such as infrastructure. So the Conservative plan is stricter.
What's the truth behind the politicians' claims on the campaign trail? Our experts investigate the facts, and wider stories, behind the soundbites.
Read latest updates or follow us on Twitter @BBCRealityCheck
Governments that spend more than they receive have to borrow money to meet their outgoings.