Britain's Chris Froome, 12 seconds behind the German in second, will move into the overall lead.
Etixx-Quick Step's Martin crashed in the final kilometre with defending champion Vincenzo Nibali also involved.
Martin's team-mate Zdenek Stybar came through to win the 191.5km stage from Abbeville to Le Havre.
"Tony Martin will fly to BG Hospital in Hamburg immediately for surgery, and must withdraw from Le Tour de France," a team spokesman said.
Team doctor Helge Riepenhof added: "The collarbone is in lots of pieces, so it was a major impact. One of the pieces came through the skin, which means it's an open fracture. Therefore, even if it was Tony's wish to start tomorrow, I have to say he is not allowed to. Riders always want to race. Tony especially."
Martin is the second race leader to pull out after Swiss rider Fabian Cancellara fractured two vertebrae in his lower back on Monday.
Team Sky's Froome tweeted shortly after Thursday's stage that he had managed to get away with a "minor graze to his knee" after finding himself in the middle of the chaos, but Martin was not so fortunate.
"I was unlucky. I don't even remember how I went down. I touched the rear wheel of the rider in front of me - Bryan Coquard - but it's the Tour, luck and bad luck are always close," the German said.
Martin, who was in yellow for the second day, went down heavily within the last kilometre and, after eventually picking himself up, had to be nursed to the finish line by three of his team-mates.
Martin - and indeed the rest of those involved, including GC contender Nairo Quintana, who also went down - were awarded the same time because the crash took place within the last three kilometres of the stage.
His Etixx-Quick Step team-mate Mark Cavendish - beaten in a sprint finish on Wednesday - crossed the line unscathed and will hope to contest what is expected to be a bunch sprint at the end of stage seven on Friday.
That would be the 26th stage win of Cavendish's career as he goes in search of his first victory since stage 13 of the 2013 Tour - but his chances will not be helped with Martin, a key member of his lead-out train, absent.
Media playback is not supported on this device
The dramatic late drama led to an initial period of confusion as riders tried to scrape themselves off the road - and it came at the end of what had been a relatively sedate day's racing, with the peloton rolling into the sunshine of Le Havre later than the organisers expected.
The most notable achievement had been by MTN-Quebeka's Eritrean rider Daniel Teklehaimanot, who was part of a lengthy three-man breakaway that enabled him to collect enough points to claim the King of the Mountains jersey.
He did so by finishing first in the day's three category four climbs, collecting one point for each to overhaul Joaquim Rodriquez and in doing so become the first African to wear the polka dot jersey.
Teklehaimanot, Europcar's Perrig Quémeneur and, eventually, Cofidis rider Kenneth Van Bilsen were all swept up as the peloton picked up the pace towards the finish.
The sharp climb at the end was supposed to rule out the pure sprinters and favour the likes of Slovak Peter Sagan and German John Degenkolb but it was Czech Stybar who burst clear to win a stage on his debut Tour de France.
Sagan was left a frustrated second - the 14th time he has finished second at a Tour stage.
Media playback is not supported on this device
Full list of standings available
Stage 6 result
1. Zdenek Stybar (Cz) Etixx - Quick-Step 4hrs 53 mins
2. Peter Sagan (Slo) Tinkoff - Saxo +2 seconds
3. Bryan Coquard (Fra) Europcar same time
4. John Degenkolb (Ger) Giant
5. Greg Van Avermaet (Bel) BMC Racing
6. Tony Gallopin (Fra)  Lotto
7. Edvald Boasson Hagen (Nor) Team MTN
8. Davide Cimolai (Ita) Lampre
9.  Julien Simon (Fra) Cofidis
10. Gorka Izagirre (Esp) Movistar
General classification
1. Tony Martin (Ger) Etixx - Quick-Step  22hrs 13mins 12secs
2. Chris Froome (GB) Team Sky +12secs
3. Tejay van Garderen (US) BMC Racing +25secs
4. Peter Sagan (Svk) Tinkoff - Saxo +27secs
5. Tony Gallopin (Fra) Lotto +38secs
6. Greg Van Avermaet (Bel) BMC Racing +40secs
7. Rigoberto Uran (Col) Etixx - Quick-Step +46secs
8. Alberto Contador (Spa) Tinkoff - Saxo +48secs
9. Zdenek Stybar (Cze) Etixx - Quick-Step +1min 06secs
10. Geraint Thomas (GB) Team Sky +1min 15secs
Selected others:
13. Vincenzo Nibali (Ita) Astana +1min 50secs
16. Alejandro Valverde (Spa) Movistar +2mins 03secs
17. Nairo Quintana (Col) Movistar +2mins 08secs
47. Mark Cavandish (GB) Etixx-Quick Step +11mins 59secs
Race leader Tony Martin is out of the Tour de France after breaking his collarbone in a dramatic crash at the end of stage six.