Gajendra Singh, from the western state of Rajasthan, was taken to hospital, where doctors pronounced him dead.
Mr Kejriwal's Aam Aadmi Party (AAP) had staged the rally in Delhi to protest against Prime Minister Narendra Modi's controversial land acquisition bill.
Opponents of the bill say it will hurt the interests of farmers.
However, the government says it will boost the economy.
The bill was passed in the lower house of parliament last month but has not yet been approved by the upper house, where Mr Modi's Bharatiya Janata Party is in the minority.
Since 1995, more than 300,000 Indian farmers have killed themselves because of debts or poor harvests. However, correspondents say this is the most high-profile incident of its kind, taking place before thousands of people at a rally in the capital.
Witnesses say the farmer only attracted the crowd's attention when he climbed to the top of the tree and tied a scarf around his neck.
AAP volunteers reportedly tried to stop Mr Singh from hanging himself but they could not reach him in time. Spectators at political rallies in India often climb trees and other tall structures to get the best view.
A suicide note, written in Hindi, was recovered, in which Mr Singh said he had three children.
"I was driven out of my home after my crops were destroyed [by bad weather]. Please tell me how I go home," he wrote.
A local journalist who visited the farmer's home in his village in Rajasthan's Dausa district told BBC Hindi that his family appeared unaware of the news.
However, others in the village had learnt of the suicide from the television news.
Mr Kejriwal addressed the rally shortly after the man was taken to hospital. He said he had been telling the police to save the farmer but they were "not in our control".
The chief minister visited the hospital after his speech. He also said the country's farmers had lost faith in Mr Modi's government.
Mr Modi tweeted that he was shocked by the death.
"At no point must the hardworking farmer think he is alone. We are all together in creating a better tomorrow for the farmers of India," he said.
India's Home Minister Rajnath Singh tweeted that he was "deeply pained" by the news and ordered the Delhi police commissioner to hold an inquiry.
Senior Congress party leader Rahul Gandhi also visited the hospital after news of the suicide and urged the country's farmers not to panic.
Unseasonal rainfall and hailstorms in many parts of India in recent weeks have destroyed standing crops, putting further strain on impoverished farmers.
An Indian farmer has died after hanging himself from a tree before a huge crowd at a rally addressed by the chief minister of Delhi, Arvind Kejriwal.