Striker Rossi won the Golden Boot as the tournament's six-goal top scorer and also took the Golden Ball award as its best player.
Zoff, the goalkeeper and captain of that Azzurri team, said he had no inkling that Rossi was seriously ill.
Rossi's death, reportedly following a battle with illness, was confirmed on Thursday by his wife and Rai Sport, where he worked.
He was 64 and is best remembered for his 1982 feats at the World Cup hosted by Spain.
Rossi scored a famous hat-trick against Brazil at that tournament, before adding two goals in the semi-final with Poland and going on to net the opener in the 3-1 win over West Germany in the final.
Zoff, quoted in La Gazzetta dello Sport, said: "I'm so sorry. I don't know what to say, it was a bolt from the blue. We've always had a great relationship with Paolo, a nice guy, intelligent, we haven't heard from each other for a while, they told us something but I didn't think it was so serious.
"The relationship with him was wonderful, he was very nice. It's something that is difficult to understand."
Ciao Pablito, Eroe Azzurro pic.twitter.com/sUyhd3q0LV— Nazionale Italiana (@Vivo_Azzurro) December 10, 2020
Rossi was also a Ballon d'Or winner in 1982 and his Italy career saw him score 20 goals in 48 games.
At club level, he won two Serie A titles, a Coppa Italia and a European Cup among a string of trophies with Juventus.
Fulvio Collovati, a centre-back in the 1982 World Cup team, said there would be a feeling of collective loss within that band of champions.
"A part of us has gone," Collovati said. "A part of my life goes away."
Italy's former prime minister Matteo Renzi added on Twitter: "In our hearts, forever. Farewell Pablito."
For in-depth, objective and more importantly balanced journalism, Click here to subscribe to Outlook Magazine