Diesel cars could have to pay an additional charge to come into inner London by 2019 and buses should be retrofitted to meet European standards.
The mayor said they must act because nearly "9,500 Londoners" a year were dying from respiratory problems.
More than 440 schools also break safe legal limits on pollution.
The Supreme Court ruled in April that London's air quality breaches European clean air rules.
9,400
People die from air pollution in London each year
500,000 aged under 19 who live in areas that breach EU limits
443 schools that have unsafe pollution levels
86 of these are secondary
2025 year London is expected to meet EU limits
"The air in London is a killer, it makes people sick and it's illegal so it's time for action," said Mr Khan, who has adult onset asthma.
Speaking to BBC Radio London, he added: "We can't carry on with business as usual," and urged the government to pass national legislation on air pollution in 2017.
Announcing his plans on the 60th anniversary of the Clean Air Act, the mayor also wants to:
He said these proposals would be cost-neutral and would not impact a promised fares price freeze.
Doctors, environmentalists and think tanks seem pretty supportive of the mayor's direction of travel.
But there are details to fill in, and this is the first stage of a two-part consultation.
The Institute for Public Policy Research says the change is not so burdensome for domestic car owners as there is a big choice in the car showroom, and competitive prices.
But innovation has been slower in the commercial vehicle sector, which means there are not many cheap options on the market for van owners at the moment.
The mayor would not say whether he would offer exemptions or discounts over a transitional period.
His answer is to push the responsibility to government which - he says - should introduce a diesel scrappage scheme as soon as possible.
The Federation of Small Businesses said although it supported the plans in principle, it was concerned about the cost of implementing them.
Sue Terpilowski OBE, London Policy chair at the organisation, said: "Micro and small businesses face disproportionately higher costs than medium and large-sized ones in carrying out business activities.
"We do not want to see tradesmen, coach companies, construction business owners or market traders refusing to serve London, which is why transport policy in London needs to recognise the difference between essential and non-essential journeys."
Motoring group the RAC said it feared bringing in the Ultra Low Emission Zone a year earlier could unfairly penalise businesses who had recently bought diesel vehicles in good faith thinking they were doing the right thing for the environment thanks to their lower CO2 emissions.
It added that more was needed to be done to improve congestion as it said data had indicated that some average traffic speeds in London were "slower than those in the horse and cart era".
Caroline Pidgeon, Liberal Democrat London Assembly Member, said the measures did not go far enough.
She called for more charging points for electric vehicles and for Transport for London to bulk purchase electric taxis to help drive down their price and to then lease or sell them on to taxi drivers or garages.
The announcement was made at Great Ormond Street Hospital (GOSH).
Its chief executive, Dr Peter Steer, said: "Children living in highly polluted areas are four times more likely to have reduced lung function in adulthood, yet improving air quality has been shown to halt and reverse this effect.
"When the UK's most seriously ill children come to GOSH for world class care, we want to ensure that they are not exposed to high levels of harmful pollution and so we are pleased improving London's air quality is a priority for the Mayor."
The consultation closes on 29 July.
Cars made before 2005 may have to pay an additional charge for entering the congestion zone in London, under proposals put forward by the mayor.