In a joint press conference Wednesday alongside Myanmar's de facto leader Aung San Suu Kyi, Tillerson called for a full and independent investigation into the situation in Rakhine state.
"We are very concerned by reports of widespread atrocities committed by Myanmar security forces," Tillerson said.
"What we know occurred in Rakhine state ... has a number of characteristics of crimes against humanity," he added.
"Whether it meets all the criteria of ethnic cleansing we continue to determine ourselves."
More than 615,000 Rohingya refugees have fled the country since August 25, according to aid agencies, pouring into neighboring Bangladesh, which has struggled to deal with the influx.
However, Tillerson was adamant that broad-based sanctions -- as some have called for in Washington -- were not an answer to the crisis.
During his visit to the Myanmar capital Naypyidaw, his first to the country, Tillerson pledged an additional $47 million in aid for refugees, bringing the total the US has earmarked for the crisis to $87 million since August.
Sanctions skepticism
Continued reports of horrific suffering and atrocities since the crisis escalated have intensified criticism of Myanmar's government and Suu Kyi in particular.
However, Tillerson said more information was needed for Washington to use that language, and called for an independent inquiry into the causes of the crisis and alleged atrocities.
"The United States will continue to work with our partners to ensure there are consequences for individuals confirmed to have been responsible for atrocities," he said.
Those consequences could include targeted sanctions, but Tillerson was clear he felt broad-based sanctions targeting the Myanmar government or other institutions were not appropriate.
"We want Myanmar to succeed, we want its democracy to succeed," he said. "I have a hard time seeing how (broad-based sanctions) help this crisis."
Acknowledging proponents of sanctions are "well intentioned," he called for patience in dealing with the "very complex situation" in Rakhine.
"It's difficult when you're sitting half way around the world and you see the images from the refugee camps in Bangladesh to not want to just rush to do something," he said.
'I have not been silent'
Speaking alongside Tillerson, Suu Kyi said what was "most important, is that we bring peace and stability to this country, which can only be done on the basis of the rule of law."
Since the crisis began, the Nobel Peace Prize laureate has come in for intense criticism from the international community.
Myanmar considers the Rohingya refugees illegal immigrants from neighboring Bangladesh, despite the fact that many Rohingya families have lived in Rakhine for decades. Bangladesh considers them Burmese.
Responding to a question about her alleged silence on the issue Wednesday, Suu Kyi said "I haven't been silent."
"What people mean is what I say is not interesting enough," she said. "My statements are not meant to be exciting, they are meant to be accurate."
"I've not been silent, (but) I have not been making very incendiary statements," Suu Kyi added, before thanking Tillerson for the "open mindedness with which you came here."