In the heart of the Northern Territory's Kakadu National Park, Mirarr traditional owners have big federal government-backed plans to transform the former uranium town of Jabiru into a tourist hub.

But it has left Indigenous residents from another language group living in a town camp on Jabiru's edge desperately worried about the future of their community.

Alarm bells started to ring for Manaburdurma Jabiru Town Camp leader Kevin Buliwana when he woke up one morning in July to contractors demolishing the house next door to his.

"I was shocked because I didn't know what to do," he said.

He said the house had been empty for a few months because the roof leaked.

But in the middle of a national and Northern Territory housing shortage, the retired health centre worker thought it should be fixed rather than removed.

His concerns were increased when two more town camp houses, and a shed used for pandanus basket weaving and bark painting, were also knocked down by Jabiru's Mirarr traditional owner body, the Gundjeihmi Aboriginal Corporation Jabiru Town.

"I think they just slowly, slowly, getting rid of us, and that is why they want to demolish the houses," he said.

Gundjeihmi Aboriginal Corporation Jabiru Town chief executive Emma Young told the ABC the demolitions were carried out because the houses and shed were dilapidated and unsafe.

"There were some structures that hadn't been used in years that were well beyond economic repair and actually dangerous," she said.

The town campers from the Kunwinjku language group came to Jabiru from Arnhem Land for work in the 1980s, after the town was built by the Ranger Uranium Mine to house its staff.

After the mine closed three years ago the federal government returned control over Jabiru to its Mirarr traditional owners.

Mr Buliwana said about 40 town camp residents, who live in 11 remaining houses, now feared they could also be removed.

The houses appear slightly run-down, but are otherwise similar to the typical NT tropical elevated and ground level houses found in Darwin suburbs.

"I just don't know what's going on, we'd like the traditional owners to come to tell us," Mr Buliwana said.

His nephew, Jeshua Djandjomerr, who has brain and physical disabilities following a car crash, said he was very worried about whether he would be able to stay in his community.

He described the town camp as feeling "quiet and safe", away from the alcohol and violence of bigger centres.

"I feel lots of worry and hurt because I have a big family here. If our house gets demolished where can we go live?" he said.

"We got big family that's buried in this ground, we've been living here for so long."

Asked if the Mirarr wanted to move the town campers out of Jabiru, Dr Young said: "That's not something I've heard, the Mirarr obviously want to look after everyone who is on their land."

But she said the Mirarr did plan to demolish the town camp so the Darwin to Kakadu National Park access road could be diverted from its current route through what is now Manaburdurma.

The Darwin to Kakadu road currently bypasses Jabiru, and the Mirarr plan to change it so that it funnels visitors into the town, as a key part of their aim to develop it into a tourism hub.

"The main aim was to make sure people had to go through the centre of town to access the rest of Kakadu," Dr Young said.

"So the longer-term plan is actually not to have residents there."

The federal government has promised to help fund the building of the Mirarr's planned new World Heritage Visitor Centre in Jabiru.

It has also committed $11 million over its next four budget years to upgrade Jabiru's roads.

But in a statement, the federal environment department, which manages Kakadu National Park jointly with traditional owners, told the ABC that the realignment of the Jabiru access road was "not included in the current planned works".

Dr Young said the Mirarr would not immediately remove more town camp houses.

"There can't be anything knocked down unless there was an alternative provided for the people who live there," she said.

Asked where the alternative housing would be found, she said it could be in Jabiru, or elsewhere.

"We've asked for a study to be done externally around affordable housing and what options might be available in Jabiru," she said.

"Once we get the options, our board will consider what we could do in Jabiru.

"There maybe multiple options not just based in Jabiru, but maybe on outstations as well."

The former uranium mine company is renovating more than 100 worker houses to return to the Mirarr, but Dr Young said she did not think any of those could go to town camp residents.

"All the houses are fully allocated, there's a waitlist for housing in Jabiru; people are saying there are a lot of empty houses, but there aren't," she said.

Mr Buliwana said the town camp residents do not want to leave.

"They are saying you can move and go, but they are not saying that you can move but stay around here somewhere," he said.