Bauer, 27, took his place for Garmin-Sharp in the team time trial on the opening day of competition earlier this week however, the experience was enough to tell him that he was not quite in the form he needed to be.
"After the TTT I simply decided to withdraw myself because of fatigue," he told roadcycling.co.nz. "I have been struggling to find form since the Olympics and have decided I'm not where I need to be to help the team.
"I don't want to be there just making up numbers on Sunday."
Meantime James Oram will compete in the New Zealand under 23 team following Tom David's withdrawal due to a medical issue.