The midfielder was involved in a clash of heads early in the first half, then scored the only goal in Austria's 1-0 victory over Ukraine at the European Championship on Monday.
Baumgartner jumped to clear a corner in the 17th minute but was knocked to the ground after colliding head-to-head with Illia Zabarnyi. He was given medical attention and was cleared to continue playing.
Four minutes later, he knocked in a corner from Austria captain David Alaba on the other end to give his team a lead it was to carry to the final whistle.
Baumgartner didn't make it that far, however. He was replaced in the 32nd minute, walking off the field while holding his head.
“He's OK,” Austria coach Franco Foda said.
“He had some problems before he scored the goal but luckily, he stayed on the pitch, scored the goal. After that, he felt dizzy and we just decided to take him off the pitch. He scored the goal so I'm very happy with him.”
Baumgartner was in pain but delighted.
"I can't quite comprehend it all, my head is really hurting. This is really special for all of us. Dear Austria, enjoy it,” he told the Austrian ORF broadcaster.
“We've already written the history, but it's not over yet.”
Austria came into Euro 2020 with zero wins, two draws, and only one goal scored from regular play in six games played in the 2008 and 2016 editions of the European Championship.
A 3-1 victory over North Macedonia in the opening group match was the team's first victory at a major tournament in 31 years. Despite a 2-0 loss to the Netherlands, the Austrians finished in second place in Group C behind the Dutch and will next play against Italy in London on Saturday.
“They are on an unbeaten run but it will end one day,” Foda said about Italy.
“If we can show the same performance we showed tonight and even improve, then I think we're able to compete with Italy.”
Ukraine is in third place in the group with three points and could still qualify for the round of 16 as one of the four best third-place teams. It will have to wait for confirmation when the other groups are finished.
“It was a bad game for us,” Ukraine coach Andriy Shevchenko said.
“We had tries in the second half, we were fighting till the end but this is not the level of football we want to show.”
Alaba played in the middle of a back three in the previous games, but he was at left back in a four-man defensive line on Monday, supporting the attack with well executed crosses and corner kicks from both sides.
Austria had more chances to score, particular from striker Marko Arnautovic, but he missed the target when he had only Ukraine goalkeeper Georgiy Bushchan to beat late in the first half.
Ukraine tried to attack in the second half and Austria goalkeeper Daniel Bachmann saved a goal-bound header from teammate Stefan Lainer, who was under pressure following substitute Viktor Tsygankov's cross in the second half.
Ukraine striker Roman Yaremchuk, who scored a goal in each of his team's first two matches at the tournament, fired narrowly wide with an angled shot toward the end. (AP)