PNG Prime Minister Peter O'Neill said in a statement he had met Australia's immigration minister, Peter Dutton, in Port Moresby on Wednesday.
"Both PNG and Australia are in agreement that [Manus Island] centre is to be closed," the statement said.
PNG's Supreme Court found in April that detaining asylum seekers and refugees at the centre was unconstitutional.
Mr O'Neill said the Manus Regional Processing Centre, which currently houses 854 men, would not close immediately.
What next for Manus Island asylum seekers?
"A series of options are being advanced and implemented," Mr O'Neill said.
"It is important that this process is not rushed but carried out in a careful manner.
"This must take into account the interests of the people of Papua New Guinea and the wellbeing of asylum seekers and refugees."
Australia sends asylum seekers who arrive in the country by boat to offshore detention centres in Manus Island and the Pacific nation of Nauru.
Even those who are found to be genuine refugees are not allowed to settle in Australia.
Mr Dutton said in a statement that the Australian government "had a longstanding position" to work with PNG's government to close Manus Island and resettle refugees.
"Our position, confirmed again today with PNG, is that no one from Manus Island Regional Processing Centre will ever be settled in Australia," the statement said.
Manus Island: Australia's Guantanamo?
Australia's controversial asylum policy
Five judges on Papua New Guinea's Supreme Court's bench ruled in April that the Manus Island camp breached section 42 of the constitution, which guarantees personal liberty.
The court said "all steps" should be taken to end the "illegal" detention.
Papua New Guinea's constitution guarantees personal liberty for all people, except in defined circumstances relating to crime, illegal immigration and quarantine.
In 2014, Papua New Guinea's government amended section 42 of the constitution to add a paragraph that allowed for "holding a foreign national under arrangements made by Papua New Guinea with another country".
But the Supreme Court ruled this amendment was unconstitutional, as it did not meet a requirement to respect "the rights and dignity of mankind".
Mr Dutton said after the decision in April that there was excess capacity at Nauru to house the detainees from Manus Island.
Australia could also reopen its centre on Christmas Island to house the men.
The leak of 2,000 incident reports from the detention centre on Nauru detailing sexual assault, violence and self harm have thrust Australia's treatment of asylum seekers back into the spotlight in recent weeks.
Australia has agreed to close a controversial asylum seeker detention centre in Papua New Guinea (PNG).