An oil pipeline, railway and motorway will also be built linking Lamu to South Sudan and Ethiopia.
Newly independent South Sudan plans to use Lamu as its main oil export outlet.
A BBC reporter says security concerns for the project may explain the presence of Ethiopian and Kenyan troops in Somalia aiming to pacify the region.
Kenya's leader Mwai Kibaki launched the project along with his South Sudanese and Ethiopian counterparts, Salva Kiir and Meles Zenawi respectively.
"I have no doubt that this day will go down in history as one of the defining moments - when we made a major stride to connect our people to the many socio-economic opportunities that lie ahead," AFP news agency quotes Mr Kibaki as saying at the inauguration ceremony.
Known as Lamu Port South Sudan Ethiopia Transport Corridor (Lapsset), it is expected to be completed within four years with initial costs coming from the three governments and plans to attract international investment.
Steven Ikuwa, the administrator in charge of Lapsset, told the BBC the scale of the plans was huge.
"I am proud to say this is one of the biggest projects that we are carrying out in Africa."
The BBC's Noel Mwakugu in Lamu says there are worries about the impact of the project on Lamu district, which is one of East Africa's most beautiful and relatively unspoiled environments along the Indian Ocean and includes a cultural heritage site on Lamu Island.
"Lamu is a living heritage. Already Unesco has declared Lamu a World Heritage Site - as an endangered site," Mualimu Badi from the Save Lamu group told the BBC.
"If 500,000 people come to work as workers, we stand to lose that status."
Mr Badi also said local residents fear they would be made homeless by the project as most people in the area are unable to prove their right to live in their homes.
In response to these complaints, Mr Kibaki has announced that residents will be issued with land title deeds and his administration will provide training for 1,000 young people to prepare for future opportunities presented by the port.
Our correspondent says Lamu's 32-berth port will be five times larger than Kenya's only other Indian Ocean port, Mombasa - which has been struggling to serve the needs of landlocked countries to the south and west of Kenya.
Lamu will allow Kenya to earn more revenue from its northern landlocked neighbours, he says.
Ethiopia will have an alternative sea port to Djibouti and another export route if oil is discovered in its Ogaden region, which borders Somalia and where oil exploration is currently under way.
For South Sudan, it offers a solution to its dependence on Sudan from which it split last July - six years after the end of a bitter civil war.
Since the secession, the two countries have been embroiled in a dispute over oil, with South Sudan turning off the tap on its oil exports though Sudan.
Our reporter says the project will also open up development to northern parts of Kenya, which has been neglected since independence from the UK in 1963.
The north-east of Kenya also hosts the world's largest refugee camp, Dadaab, home to more than 450,000 Somalis who have fled conflict and famine during the past 21 years.
Kenya sent in troops over its long border with Somalia in pursuit of al-Shabab militants in October, after several people, including foreign tourists, were kidnapped, including some from near Lamu.
The authorities in Kenya blame the Islamist al-Shabab group, which controls much of southern and central Somalia, for much of the instability affecting the sub-region.
Analysts say Kenya and Ethiopia, whose troops now occupy substantial parts of the country, hope that with progress on the military front, Somalia's bickering politicians can come up with a new constitution that will allow elections to be held and relative peace to return to the lawless country.
Construction has begun on a $23bn (£14.5bn) port project and oil refinery in south-eastern Kenya's coastal Lamu region near war-torn Somalia's border.