Charles Jenkins died from heart failure and was discovered unconscious by his daughter at his home in Sado Island, northern Japan, on Monday night, Yoshiyuki Tomi from the Japanese Cabinet Office told CNN.
Police and the hospital said there were no irregular circumstances.
His wife, Hitomi Soga, released a statement saying she was "very shocked by this sudden incident."
Jenkins crossed into the North in 1965 while stationed at a US military unit near the demilitarized zone (DMZ), the heavily guarded border that separates North and South Korea.
Jenkins later claimed to have regretted his defection and blamed the decision on alcohol.
It was during this time he met Hitomi Soga, a Japanese national who was kidnapped from her home in Sado Island by North Korean spies in 1978.
The pair were married in 1980 and had two daughters, Mika and Brinda.
Soga returned to her homeland in 2002 following an agreement struck between North Korean and Japanese leaders.
Jenkins had been initially reluctant to join Soga for fear of being extradited to the United States to face a military court-martial.
However, on his return to the US in the early 2000s, he was dishonorably discharged and spent less than 30 days in jail.