Talismanic batsman Smith was struck on the neck by a Jofra Archer bouncer on Day 4 on Saturday and retired hurt before later returning to resume his innings.
The in-form Smith, who made centuries in each innings of the first Test at Edgbaston, did not play at all on Day 5, though, after suffering with delayed symptoms.
Marnus Labuschagne was called in as a concussion substitute and made a brilliant 59 as the tourists battled to a draw to maintain their 1-0 series lead.
Smith said he was "hopeful" he would recover to feature at Headingley, but any chances of participation in Leeds were ended by Langer on Tuesday.
The 30-year-old missed training with the rest of his Australia teammates before the news was confirmed.
Coach Langer was due to discuss Smith and the upcoming Test with the media later on Tuesday.
Smith is returning to Test cricket in this series after serving a 12-month ban for his role in the team's ball-tampering scandal in South Africa last year, which also saw him lose his role as captain.
He has been in sensational form in England, making 144, 142 and 92 from his three Ashes knocks to put Australia ahead in the series prior to his injury.