Workers building a sports stadium had earlier unearthed the bomb in the town of Goettingen.
It was not immediately clear why the bomb, reportedly weighing 500kg (1,100lb), had detonated.
Unexploded WWII bombs dropped by Allied planes are frequently found in Germany, though it is unusual for them to explode unexpectedly.
A special commission is investigating the causes of the explosion, while prosecutors are considering whether the team leader should face charges of manslaughter through culpable negligence, the BBC's Oana Lungescu reports from Berlin.
The blast happened an hour before the defusing operation was due to start.
Officials said the three men who died were experienced sappers, or combat engineers, who over 20 years had defused up to 700 bombs.
More than 7,000 people were immediately evacuated when the 500kg bomb was found. Several schools, a kindergarten and local companies remain closed.
Last week, another device was successfully defused close to the site.
In Berlin, four bombs have been found in the past two months, including one near the capital's main airport.
Three people have been killed and six injured trying to defuse a World War II bomb in central Germany.