Abu Khaled al-Suri was among several people who died when a base of Ahrar al-Sham, part of the Islamic Front, was targeted on Sunday, activists said.
The Islamic State in Iraq and the Levant (ISIS), a jihadist rebel group, is thought to have been responsible.
Suri was reportedly sent to Syria by al-Qaeda to end the infighting between ISIS and other rebel groups.
It is said to have left more than 2,000 people dead since early January.
Abu Khalid al-Suri was killed along with six comrades from Ahrar al-Sham in Sunday's attack, according to the UK-based Syrian Observatory for Human Rights.
Two rebels told the Reuters news agency that five men had entered the rival group's headquarters in Aleppo and opened fire before one blew himself up.
There was no immediate claim of responsibility for the attack, but a spokesman for the Islamic Front, Akram al-Halabi, said he believed ISIS was behind it.
"The first fingers of blame point to the State," he told the Associated Press. "Unfortunately this is going to make the infighting worse."
Suri, a Syrian-born militant whose real name was Muhammad Bahaiah, is believed to have been close to the late al-Qaeda leader Osama Bin Laden and his successor Ayman al-Zawahiri.
He fought against US forces in Afghanistan and Iraq.
Rebel sources said he was sent to Syria a few months ago on a mission to mediate in the conflict between ISIS and other rebel groups, including the al-Qaeda-affiliated al-Nusra Front.
An audio recording attributed to Suri was released last month in which he said ISIS had "sought to corrupt the jihad in Syria, as it did in Iraq and Afghanistan".
He called on the group to stop attacking other jihadists, adding: "Direct your car bombs at the infidels and do not busy yourself with fighting the mujahedeen and killing them."
Earlier this month, al-Qaeda's general command insisted that it had "no connection" with ISIS and was not responsible for its actions.
ISIS grew out of the former Islamic State of Iraq (ISI), a jihadist militant umbrella group that included al-Qaeda in Iraq. It is believed to have helped create the al-Nusra Front in mid-2011.
In April 2013, ISI leader Abu Bakr al-Baghdadi announced the merger of his group and al-Nusra - effectively a takeover - and the creation of the Islamic State in Iraq and the Levant (ISIS).
But the move was rejected by al-Nusra leader Abu Mohammed al-Julani and Zawahiri, al-Qaeda's overall leader, who recognised al-Nusra as its sole Syrian offshoot.
Since then, ISIS and al-Nusra have operated as separate entities, with the latter focusing on toppling President Bashar al-Assad and maintaining better relations with other rebels. ISIS has seemed to be more concerned by territorial gains and implementing its extreme interpretation of Islamic law.
Also on Sunday, several people were reported killed by a car bomb near a field hospital in northern Syria, close to the border with Turkey.
Activists said the blast happened in the rebel-held town of Atmeh, which hosts a camp for thousands of people displaced by Syria's civil war.
It was not immediately clear who had carried out that attack.
According to UN figures, 6.5 million Syrians have been displaced by the country's civil war, and 2.5 million are registered as refugees. Lebanon has taken the highest number of refugees, followed by Jordan and Turkey.
A rebel leader linked to al-Qaeda has been killed in a suicide bomb attack in the northern Syrian city of Aleppo.