Champions League holders Liverpool suffered just a fourth loss of the campaign in Tuesday's last-16 first leg after Saul Niguez's fourth-minute strike.
Klopp made the surprise decision to substitute Mane, the match-winner against Norwich City at the weekend, for Divock Origi at half-time.
Mane was cautioned for clashing with Sime Vrsaljko and Atleti's players pleaded with referee Szymon Marciniak to dismiss the Senegalese when he caught the same player with a flailing arm.
Klopp confirmed after the game the decision to replace Mane was not down to an injury concern, the forward having only recently returned from a hamstring injury.
"It's part of football, I don't like it, but the plan was to get Sadio out the game with a second yellow card," he said at his post-match news conference.
"I was a little afraid the opponent would go down even if Sadio had a deep breath. I didn't want that situation, that's why I took him off."
Klopp was also forced to replace skipper Jordan Henderson late on after he tweaked his hamstring, but the Liverpool boss downplayed the severity of the injury.
"Hendo felt something, unfortunately," he said. "We have to wait – he was not too concerned, but it was enough for him to go off."
Saul's strike came as a result of some poor defending from Liverpool, but the visitors felt they should have been awarded a throw-in in the lead up to the corner that Atletico scored from.
Klopp's men had 73 per cent of possession in the Spanish capital but failed to register a single attempt on target, with Mohamed Salah and Henderson coming closest to equalising.
"Would I have liked to have got the throw-in? Yes. That would have made our life easier," Klopp, himself shown a yellow card for dissent in the second half, told reporters.
"Coming here and having 70 per cent possession is exceptional, our build-up was outstanding against a high-press team – it was next level, the best we have done for a long time.
"The intensity was exactly how it should be, putting the opponent under pressure in the final third we were not clinical enough. We didn't have enough clear-cut chances.
"We had chances but not enough. Staying in the game is development. We had games in atmospheres like this where we lost focus and they scored a second. That did not look like happening tonight.
"It is 1-0 at half-time. We don't give up when it's 15 minutes half-time so why should we when we have three weeks? The second half will be played in a different stadium, our stadium, and you will feel that. We are looking forward to it.
"All Atletico fans who have a ticket for the game, welcome to Anfield."
For in-depth, objective and more importantly balanced journalism, Click here to subscribe to Outlook Magazine