Unicef, the UN children's fund, said that Islamist militants in the north of the country had forcibly recruited at least 175 boys aged 12-18.
Unicef said it had collected evidence that also indicated the groups had raped and sexually abused girls.
A military coup in Mali in March 2012 allowed Islamist militants to take control of the north of the country.
The UN Security Council said on Thursday it was not ready to back a West African intervention force in northern Mali, but instead passed a resolution calling for sanctions against the groups.
Unicef said as well as abuse and the use of children as fighters, children have also been killed and maimed by landmines and other ordnance.
It went on to say that about 300,000 children have been affected by the closure of schools in unstable areas.
Since rebels took control of the north of Mali, many children have been living in camps where they face food shortages, Unicef said.
The UN said the information they had presented was only a partial picture, however, as it was too dangerous to undertake proper research in northern Mali.
Unicef said it was working with partners in certain areas of the country in an attempt to help communities to protect children.
"Children in the north are witnessing or becoming victims of violence and they must be protected," said Theophane Nikyema, Unicef's Mali representative.
The ongoing conflict in Mali has seen mosques and shrines in the historic city of Timbuktu attacked in recent weeks by militants.
Ansar Dine, a group said to have links to al-Qaeda, said it had an objective of destroying all mausoleums that were not in line with Islamic law.
The group seized control of Timbuktu in April.
Children are being sexually abused and are being recruited to fight for armed groups in Mali, according to the UN.