Australian Prime Minister Malcolm Turnbull tweeted: "In a world of superficiality, he was always informed and honest. We've lost a good man."
Mr Colvin had struggled with a rare auto immune illness he contracted while covering the Rwandan genocide in 1994.
He received a kidney transplant in 2013, and his death has led to renewed calls for people to sign up as donors.
The Australian Broadcasting Corp, where he had worked for decades, said it was grieving "a giant of Australian journalism".
Bill Shorten, leader of the main opposition Labor party, described him as "a gentleman of journalism" while Greens leader Richard Di Natale wrote on Twitter that "a fascinating life ended far too soon. You will be missed".
As well as the many tributes on Twitter from politicians, colleagues, friends and the public, there was even a tweet from his own account, posted after his death, saying: "It's all been bloody marvellous."
"Today we lost our beloved Mark," Mr Colvin's family said in a statement asking mourners to donate to the Prince of Wales Hospital trust in lieu of flowers.
"The family would like to thank the doctors and nurses at the Prince of Wales hospital, as well as the community, the ABC, his friends and colleagues, who have stood by him and supported his career and life."
Mr Colvin's career as a journalist started as a youth radio reporter with the ABC in 1974, spending decades at the national broadcaster, where he most recently hosted the flagship PM programme.
At the age of 28, he became the ABC's London correspondent, and later moved on to Brussels.
He also worked as a correspondent in Africa and a feature of his on the famine in Ethiopia was runner-up for an International Emmy award.
ABC's obituary described the award-winning journalist as being "among Australian journalism's most authoritative voices, and a master interviewer with a depth of knowledge in world affairs".
ABC News Director Gaven Morris said he had been "one of Australia's finest journalists, admired and respected by his workmates and audiences alike for his intellect, wit and absolute integrity".
Last year, Mr Colvin published a book detailing how he learned his father was a British spy.
Mr Colvin contracted Wegener's granulomatosis, a rare auto immune illness damaging the lungs and kidneys, while reporting from Rwanda in 1994.
He recovered after a long time in hospital but the illness left his kidneys permanently damaged.
In 2013 he received a new kidney, donated by a woman he once interviewed when reporting on the News of the World phone-hacking scandal.
The unique story of Mary-Ellen Field donating her kidney to the journalist eventually even became the subject of a play.
His death on Thursday has led to many voices renewing their call on people to sign up as organ donors.
Tributes are pouring in for Australian veteran journalist Mark Colvin who has died aged 65.