The button appeared beside Gmail's normal send button and allowed users to shut down an email thread by sending a gif of a Minion dropping a microphone.
However, a flurry of complaints about the button appeared on Google's forums.
The firm has since withdrawn the feature and apologised.
"It looks like we pranked ourselves this year," Google said in a statement.
"Due to a bug, the MicDrop feature inadvertently caused more headaches than laughs. We're truly sorry."
A mic drop is a popular meme in which someone makes a conclusive statement or rebuttal before dropping a microphone and walking off.
Google was not immediately able to clarify details of the bug.
Google added that users who were still able to see the feature could switch it off by reloading or restarting Gmail.
The button was not enabled on the Gmail accounts of enterprise business customers.
"Thanks to MicDrop I just lost my job," claimed one user on Google's product forums.
"I am a writer and had a deadline to meet. I sent my articles to my boss and never heard back from her. I inadvertently sent the email using the MicDrop send button."
Another complained that they had been having interviews for a job with a company for three months and then accidentally sent a mic drop email to the HR department.
And a third said that they had sent an "important email to 30 recipients".
Users had been informed of the button and what it would do via a message in Gmail when it first appeared.
Furthermore, users with the "undo send" capability enabled would have been able to retract a message within a certain time period after sending it.
Google also unveiled a couple of April Fool's joke videos, including one about a new parachute delivery service and an "actual reality" headset.
Google originally announced the creation of Gmail on 1 April 2004 and, at the time, there was speculation as to whether the email service was an April Fool's stunt.
It's hard to know what to trust on April Fool's Day as there are are many stories that seem rather strange but are in fact genuine.
The BBC has compiled a round-up of some of this year's odder stories that are apparently true.
Google has removed an April Fool's Gmail button, which sent a comical animation to recipients, after reports of people getting into trouble at work.