The Syrian Observatory for Human Rights said that the Ahrar al-Sham, Jund al-Aqsa and Nusra Front groups had taken the city on Saturday.
Idlib is only the second provincial centre to fall into rebel hands, after Raqqa was seized by Islamic State (IS).
Syria's civil war, which began four years ago, has killed more than 200,000 Syrians and displaced 11 million.
The UK-based observatory said the militant groups seized the city after four days of intense fighting.
Syria has not confirmed the loss.
State television said: "The army is fighting fierce battles to restore the situation back to what it was."
The capture of the city, with a population of 100,000, would be a serious blow to the government of President Bashar al-Assad, correspondents say.
Idlib has been the subject of severe fighting between the government and opposition forces for months
Idlib is close to the strategically important main highway linking the capital Damascus to the commercial capital Aleppo. It is also close to the coastal province of Latakia, a stronghold of President Assad.
Supporters of Ahrar al-Sham and the Nusra Front have posted pictures of fighters celebrating in the city.
It is the second defeat of government forces in the space of a week after rebels in southern Syria seized the historic city of Busra in Deraa province.
On Saturday, UN Secretary General Ban Ki-moon said he was angry and shamed by the failure to halt the Syrian civil war.
Speaking at an Arab League summit in Egypt, he promised to step up diplomatic efforts to stop the fighting.
Islamist rebels have captured the north-western Syrian city of Idlib from government forces, monitors say.