The rally was mostly on behalf of Tamil people.
Many protesters were prevented by the authorities from attending, but a big pro-government demonstration was allowed to go ahead.
Tamil Tiger rebels fought a 26-year war for a separate state in the north and east before they were defeated in 2009.
Rights activists say that some of those who disappeared were fighting for the Tamil Tigers, some fought for the government and some were civilians.
They say that many Tamils remain in the hands of the security forces.
They want a UN-led international probe into alleged human rights abuses during the war, but the government has rejected the demand and denies being responsible for most of the disappearances.
Opposition politician and anti-disappearances campaigner Mano Ganeshan told the AP news agency that buses packed with about 1,000 Tamils were prevented from leaving the northern town of Vavuniya on Tuesday to make the 210km (130-mile) journey to Colombo for Wednesday's protest.
He said that government forces and police had intimidated the bus drivers, warning them not to proceed with the journey.
However, military spokesman Brig Ruwan Wanigasooriya said that police had stopped the buses to prevent possible clashes in Vavuniya.
He said complaints were made to police earlier on Tuesday that people had thrown stones at the buses.
The US embassy in Colombo has expressed concern about the Vavuniya reports. It has called on the Sri Lankan government to allow free movement of its citizens "calling for information about their missing loved ones".
One of those attending the Colombo protest, Perinparani, told the BBC's Charles Haviland that her son Pradeepan, 20,  was taken from her in the northern town of Jaffna in 2008.
"The army officers came in four field motorbikes," she said.
"There were about eight army officers, with beards. These people came and took him. They had no letter, they didn't tell me why they were taking my son away."
Her son was a labourer with no connection to the rebels, she said.
Her repeated visits to prisons have yielded little information. At the rally in Colombo, she and other distressed parents held up photos of their missing children.
Meanwhile the government organised a big rally, also in the middle of the capital, at which demonstrators accused Tamil MPs of never having criticised the Tigers for perpetrating atrocities.
The families of people who disappeared in Sri Lanka during and after the country's long civil war have staged a demonstration in the capital Colombo.