Firefighters were called around 23:30 (12:30 GMT) on Wednesday after reports of an explosion.
It took them 40 minutes to extinguish the blaze before finding three bodies inside. Local media say the victims are believed to be two men and a woman.
The factory was often used as a shelter by homeless people.
Arson experts will investigate the cause of the fire, police said.
One witness, taxi driver Mohamed Ali, said he had heard screams coming from the factory in Footscray, in the city's west.
"A few minutes later I saw the fire engines come," he told the Australian Broadcasting Corp.
"For me, it's a horrible situation because we couldn't help. I haven't seen anything like this in my whole life. I'm not feeling good."
Prominent Melbourne youth worker Les Twentyman said homeless people often took refuge at the site.
"There is a little alcove near the car park - people sleep in that little alcove,'' he told the Herald Sun.
"It just shows how desperate they are."
The site was originally Victoria's largest ropeworks and operated from 1903 to 2002. After the factory shuttered, the state government backed plans to build a large apartment complex on the site.
Three people have died in a suspicious fire that tore through an abandoned factory in Melbourne, Australia.