In the letter, Trump wrote that, based on the North's recent statement, he felt it was "inappropriate, at this time, to have this long-planned meeting".
Trump called it "a missed opportunity" and said someday he still hoped to meet Kim. He said it was "to the detriment of the world" that the summit would not take place.
"You talk about your nuclear capabilities, but ours are so massive and powerful that I pray to God they will never have to be used," he wrote.
The North Korean statement that he referred to called Vice President Mike Pence a "political dummy" for comparing North Korea - a "nuclear weapons state" - to Libya, where Muammar Gaddafi gave up his unfinished nuclear development program, only to be later killed by NATO-backed fighters.
Senior North Korean official Choe Son Hui responded, issuing a bellicose statement threatening: "Whether the US will meet us at a meeting room or encounter us at nuclear to nuclear showdown is entirely dependent on the decision and behaviour of the United States."
Even as North Korea carried out the destruction of the nuclear test site, North Korea and Washington remained poles apart over the pace of any road to denuclearisation, with North Korea wanting a staged approach and the US pushing for complete denuclearisation before any concessions were made.
North Korea had been increasingly angered at public statements by Trump’s inner circle, starting with national security advisor John Bolton and later repeated by Pence and Secretary of State Mike Pompeo comparing North Korea to Libya.
Pompeo read out Trump's letters to legislators in Washington shortly after it was released publicly.
Asked how the cancellation of the summit left the US in the eyes of the world, Pompeo said Trump’s team didn’t think there could be a successful outcome.
Loading
In recent days the US side had not received any response from North Korea to efforts to prepare for the meeting, he said.
He insisted Trump had been prepared for the summit. But analysts had questioned whether the
Trump team was prepared to deal with Kim, and Pusan National University’s Robert Kelly had commented earlier that Pompeo’s call for a rapid denuclearisation by North Korea was "such an obvious non-starter with North Korea - they’ll never go for upfront... that’s how Gaddafi met his end".
In the letter, Trump politely thanked Kim "for the release of the hostages who are now home with their families" and said some day he looked forward to meeting him.
After the nuclear test ground was dismantle, North Korean state media had issued a statement saying it had been conducted with "high level transparency".
Director of the Korea Centre at the Wilson Centre, Jean Lee, said North Korea would have hoped dramatic video of the destruction of the test site would "prove" its commitment to denuclearisation, and Pyongyang could then ask Trump to do something to prove his commitment.
Chinese foreign minister Wang Yi, in Washington on Wednesday, had earlier urged the US to attend the summit, saying "if you want peace, now is the time" at a joint press conference with Pompeo.
But Trump had recently expressed his suspicion that China had been behind a "change in attitude from Kim Jong-un" and said Chinese President Xi Jinping was a "world class poker player".
Kim met with Xi for a second time earlier this month.
North Korea’s increasingly hostile tone also coincided with TV appearances by John Bolton who compared the path for North Korea’s denuclearisation to the Libyan model.
South Korea’s presidential office said on Thursday night it was trying to understand Trump's intentions, and South Korean President Moon Jae-in convened a midnight meeting with his security and foreign policy aides.
Moon had brokered the opening for summit with his invitation for North Korea to attend the Winter Olympics in January. Moon held an historic meeting with Kim last month, where the two Koreas signed a declaration that they would seek a peace treaty this year and work towards denuclearisation.
CNN said White House sources said the door was still open. Trump writes in the letter if Kim changes his mind "do not hesitate to call me or write".
Kirsty Needham is China Correspondent for The Sydney Morning Herald and The Age
Morning & Afternoon Newsletter
Delivered Mon–Fri.