The UMP, led by ex-President Nicolas Sarkozy, boosted the number of councils it controls from 40 to 67 in the vote.
President Francois Hollande's Socialists and other leftists won 34 councils - down from 61 previously.
Marine Le Pen's far-right National Front (FN) won at least 60 seats but failed to get control of any council.
The elections are seen as a key test of public opinion ahead of the 2017 presidential election.
Paris and Lyon, France's two biggest cities, did not take part in Sunday's election.
Mr Sarkozy, who secured the UMP leadership last November, said "never... has our political family won so many councils".
He told his supporters that voters had "massively rejected the policies of Francois Hollande and his government".
Bastions of the Socialists like the Nord department around Lille swung to the right, as did President Hollande's own fiefdom of the Correze in central France.
More than 90% of the results have been counted for the 4,108 council seats. Local councillors have responsibility for schools, roads and social services in their areas.
As ever in France's two-round elections, voters from left and right united in round two to keep the FN from power, the BBC's Hugh Schofield in Paris reports.
Ms Le Pen hailed a "historic" day for the FN, saying: "I thank all our voters for this magnificent success."
"The goal is near, reaching power and applying our ideas to redress France."
The FN made especially strong gains in Pas-de-Calais, in the far north, and Gard, in the south.
But the two-round system meant that the FN's strong showing in the first round - a quarter of the total vote - did not translate into seats.
The FN won the European elections in France last year, and surveys suggest that Ms Le Pen could reach the second round run-off in the 2017 presidential election.
Our correspondent says the FN's exclusion from mainstream politics is part of its attraction to disillusioned voters.
It had only one departmental council seat before these elections - but now it has 62. That is still only 1.55% of the total counted so far.
French Prime Minister Manuel Valls said the French people "have declared... their anger at a daily life that is too difficult". He vowed to redouble efforts to boost the economy, and said his focus was "jobs, jobs, jobs".
He added that the rise in the FN's popularity was "a sign of a lasting upheaval of our political landscape and we will all need to draw lessons from it".
Half of the 40-million-strong French electorate abstained from voting - a figure that was, however, better than the 55.3% abstention rate in the last local elections in 2011.
(Based on 3,990 seats, out of 4,108 in total)
Source: French Interior Ministry
France's opposition conservative UMP Party has made big gains in local elections, at the expense of the ruling Socialists.