South Korea conceded the winner to Mana Iwabuchi late in the second half in their opening match of the East Asian Football Federation (EAFF) E-1 Football Championship at Soga Sports Park in Chiba, Japan.
The four-nation tournament also features North Korea and China.
Earlier in the day, North Korea beat China 2-0, thanks to Kim Yun-mi's double.
South Korea, led by head coach Yoon Duk-yeo, will next face North Korea on Monday. The Taeguk Ladies are looking for their first East Asian football title since 2005.
Playing on a rain-soaked pitch, Yoon's side gave up the lead to Japan just seven minutes after the kickoff when Mina Tanaka headed down Miho Manya's cross from the left flank.
South Korea leveled the score in the 14th after they were awarded a penalty kick. Han Chae-rin's cross was blocked by Japanese captain Rumi Utsugi's hand in the box, and South Korean captain Cho So-hyun later converted the spot kick to make the score 1-1.
The visitors then created a couple of chances to take the lead against the hosts. In the 17th, Yoo Young-a's header off a corner kick went over the net, while Lee Min-a's powerful volley off Kang Yu-mi's cross also flew over the crossbar one minute later.
Japan, the 2011 FIFA World Cup champions, tested the South Korean defense with Mizuho Sakaguchi's strike off Iwabuchi's corner kick in the 32nd that went straight to goalkeeper Kim Jung-mi. In the 45th, Yui Hasegawa's cross-cum-shot from the left wing went just over the net.
The sides produced three goals between them in the second half and it was Japan who opened the scoring.
After Shiori Miyake headed down Ayumi Oya's corner kick in the 70th, Emi Nakajima got the ball and fired one in from close range to make it 2-1 Japan.
South Koreans earned another equalizer in the 80th when Han's left-footed volley off Lee Min-a's cross beat Japanese goalkeeper Sakiko Ikeda.
But the 2-2 score didn't last long. In the 83rd, Nakajima's long-range effort hit the crossbar and bounced to Iwabuchi, who slotted home the rebound easily for the winner.
South Korea had three minutes of extra time to try to level the score, but failed to find the back of the net. (Yonhap)