The refugees spent Monday ahead of the closure filling up empty bottles and bins with rainwater. Food hasn't been distributed since Sunday, and many say they're hungry and scared. Electricity to the center was switched off on Wednesday, the refugees said.

"They are terrified, it's not about the facilities, it's about their safety. These men are terrified of violence if they stay in the detention center and they are terrified of violence if they leave," said Daniel Webb, from the Human Rights Law Center in Australia.

The center, which re-opened in 2012 as an offshore processing site for asylum seekers found in Australian waters, is being returned to the PNG Defence Force, which owns the land.

The notice distributed ahead of the closure said: "All power and water will cease. There will be no food supplied -- and no dinner service this evening."

Australian Foreign Minister Julie Bishop said Tuesday that PNG had provided alternative accommodation for the men, with "food, water, electricity and medical services."

She said the Australian government was in "constant communication" with its PNG counterparts and expressed support for the PNG plan. "We are there to support PNG," she said.

Police numbers strengthened

Extra PNG police officers had been deployed to Manus Island, according to Chief Superintendent Dominic Kakas. However, he told CNN that force wouldn't be used to move the refugees.

"There will be no use of force or harm done to the refugees. We will not be forcing them to go. They will do so freely and willingly," he said.

An injunction has been filed in PNG Supreme Court to stop the closure, according to Melbourne-based barrister Greg Barns. An emergency hearing is expected Wednesday morning.

More than 300 Manus Island residents have signed a petition calling for the refugees and asylum seekers to be relocated in Australia, according to the Refugee Action Coalition.

They say they don't want the men, from places like Iran, Sri Lanka, Afghanistan and Myanmar, to be living so close to the town. Lorengau has a population of just 6,000 people, and the arrival of hundreds of outsiders has caused suspicion and fear.

The Australian government, however, remains adamant that the men will never be settled in Australia.

Refugees remaining in the center have been asked to move to two other locations -- the East Lorengau Refugee Transit Center or West Lorengau House -- both provided by the Australian government at a cost of up to $190 million (A$250 million dollars) a year. Asylum seekers who haven't been found to be refugees are being asked to move to a third location, Hillside House.

The men say their safety is at risk at all three centers, which they claim don't offer the same level of security as the fenced detention center.

'Nothing more than subterfuge'

In a strongly worded statement issued Tuesday, Immigration Minister Peter Dutton said "the constant claims of the IMAs and advocates about their situation in Manus are nothing more than subterfuge."

"They have long claimed the Manus RPC was a 'hellhole' -- but the moment it was to be closed they demanded it to be kept open," Dutton said.

IMAs refers to "Illegal Maritime Arrivals," the name Australia gives to asylum seekers who arrive in the country's territorial waters by boat. Lawyers say the use of the term "illegal" in relation to asylum seekers who arrive by boat is an invention of the government, as it's not an offense to seek asylum.

Dutton also directed his anger towards Australian Greens Senator Nick McKim, who on Tuesday visited the detention center to reassure the refugees that their message was being heard.

"Senator McKim's duplicity is breathtaking," Dutton said in the statement. "Senator McKim claims to care for these men, but continues to provide them with false information and uses them for cheap political stunts to build his public profile."

McKim rejected suggestions he was using the situation for his own political purposes.

The center was originally opened in 2001 as part of then Prime Minister John Howard's "Pacific Solution," to process asylum seekers in offshore facilities.

The government has suggested that relaxing the policy would encourage other asylum seekers to attempt the treacherous journey, leading to more deaths at sea.