Police moved into the Manus Island detention center, which houses and processes men who sought asylum in Australia, early Friday morning for a second day in a row to remove those still inside, said Dominic Kakas, the superintendent of Papua New Guinea Police. The men were then transported to two other facilities in Papua New Guinea.
At least one man alleges they were removed violently.
"They brought us by force, they beat us by sticks and they abused us," Ezutallah Kakar, a Pakistani man who has been on the island since August 2013, told CNN in a voice message. "They just beat us like animals, like we are not human beings."
Kakas, the Papua New Guinea Police superintendent, told CNN he had no information about any use of force, and added "I'm sure it was peaceful."
CNN was sent video showing police inside the facility holding long sticks in their hands as they pull up a seated asylum seeker.
When asked about the video, Kakas told CNN just because the police were holding sticks doesn't mean they were used on the asylum seekers. He said he had no information about force being used against the men.
The refugees said they didn't want to leave because the new facilities don't give them enough protection from locals who don't want them living on the island.
Once at the new camp, many were upset. Some were crying, Kakar said. Kakar was an award-winning kickboxer and mixed martial arts fighter in Pakistan, which he said he fled because it was unsafe for him.
"We don't need any new prison or camp," he said.
Australian Immigration Minister Peter Dutton said in a statement Canberra was aware that all the men had departed.
"Since May, information was provided on a weekly basis that the RPC (Regional Processing Center) was closing and services and support would transition to the alternative accommodation sites," the statement said.
Kakas denied those allegations. He described the asylum seekers as being "rowdy" but said officials were not using force against them.
Dutton, in his statement, also denied the allegations.
"Advocates in Australia are again today making inaccurate and exaggerated claims of violence and injuries on Manus, but fail to produce any evidence to prove these allegations," the statement from Minister Dutton said.
Australia has a very strict immigration policy for asylum seekers who try to reach the country by boat. They're told they will never be able to settle on Australian soil and are sent to island camps for processing.
The Manus Island Regional Processing Center was originally built by the Australian government in 2001, together with another center on the small Pacific country of Nauru, to process asylum seekers who arrived by boat.
It was reopened in 2012 after a rise in the number of boat arrivals off Australia's shores. The refugees currently housed on Manus come from a range of countries, including Iran, Sri Lanka, Afghanistan and Myanmar.
International human rights advocates have long condemned Australia's use of the facility on Manus and the treatment of the refugees there.