Organizers did however warn that it could restrict the number of fans in the event of a state of emergency or amid any other restrictions to curb the rise of Covid-19 infections.
Those attending the postponed Olympics will have to abide by a number of protocols aimed at stopping the spread of cases.
"Masks should be worn in venues at all times; speaking in a loud voice or shouting will be prohibited; congestion should be avoided by means of appropriate announcements; and visitors should leave venues in a staggered manner," read a statement outlining the guidelines.
"Spectators will be requested to travel directly to venues and return home directly, and to take all necessary precautions when moving between prefectures."
Risk posed
The organizers' decision came amid concern about the spread of Covid-19. Tokyo has now shifted to a quasi-state of emergency until July 11.
Last Friday, a group of infectious disease experts in Japan submitted a report on Covid-19 countermeasures at the Games to the government and organizing committee.
They said holding events without spectators posed the least risk -- organizers already decided in March to ban overseas spectators.
They also recommended stricter restrictions than those currently in place for large-scale events if spectators are allowed at the Games' venues. They added that if Covid-19 inflections look like they're rising, organizers should ban spectators at venues.
Tokyo 2020 President Hashimoto Seiko responded that the Games may be held without spectators depending on the infection situation.
On Monday, Japan's prime minister Yoshihide Suga said he will not rule out an Olympics without spectators if Tokyo is under a state of emergency.
"I would like to take on board chairman Omi and everyone else's proposals," Suga told reporters while visiting a vaccination site. "In the event a state of emergency being declared, then we can't rule out not having spectators -- the public's safety and security is a top priority."
The decision for the number of spectators at this year's Paralympic Games will be made on July 16, a week before the Olympics is set to begin.
CNN's Aleks Klosok contributed reporting.