Taylor, who is of Samoan heritage, describes racist locker room "banter" and casually racist comments from some New Zealand team officials.
“Cricket in New Zealand is a pretty white sport,” Taylor writes in his book, entitled “Black & White."
“For much of my career I've been an anomaly, a brown face in a vanilla line-up. That has its challenges, many of which aren't readily apparent to your teammates or the cricketing public.”
The 38-year-old Taylor said many people assumed he was of Maori or Indian heritage because Pacific Island representation in New Zealand cricket is so rare. He said locker room banter sometimes was racist and hurtful but he was concerned that raising the issue might make the situation worse.
“In many ways dressing-room banter is the barometer,” Taylor wrote.
“A teammate used to tell me you're half a good guy Ross but which half is good? You don't know what I'm referring to.' I was pretty sure I did."
He said other players also had to put up with comments that dwelt on their ethnicity.
“In all probability a (white New Zealander) listening to those sorts of comments would think Oh, that's okay, it's just a bit of banter,'” Taylor said.
"But he's hearing it as white person and it's not directed at people like him. So there's no pushback; no one corrects them.
“Then the onus falls on the targets. You wonder if you should pull them up but worry that you'll create a bigger problem or be accused of playing the race card by inflating harmless banter into racism. It's easier to develop a thick skin and let it slide, but is that the right thing to do?”
Taylor said a former manager and coach of the New Zealand team made comments that were unintentionally racist.
The manager told Taylor's wife, Victoria, that in his experience players of Maori and Pacific Island heritage have problems managing money and offered his assistance.
Taylor said former coach Mike Hesson, who guided the New Zealand men's team for six years from 2012, once told him my cleaner's Samoan. She's a lovely lady, hard-working, very trustworthy.'
“All I could say was oh, cool,' Taylor wrote.
“I have no doubt that (the officials) and the guys who engaged in the banter' would be dismayed to learn that their remarks landed with a thud.
“Let me be clear: I don't think for one minute that they were coming from a racist perspective. I think they were insensitive and lacked the imagination and empathy to put themselves in the other person's shoes.
“What to them is a bit of harmless banter is actually confronting for the targets because it tells them they're seen as being different. Instead of the message being You're one of us, mate,' it is, in effect, You're one of them.'”
A New Zealand Cricket spokesperson told the New Zealand Herald newspaper that the national body “deplores racism, is a staunch supporter of the New Zealand Human Rights Commission's Give Nothing to Racism campaign and is deeply disappointed Ross has been exposed to this type of behavior.”