Agreed. This whole thing is a fool's errand.
Burgundy will definitely look terrible. As will navy, or brown, or green, or anything else you care to dream up.
I like contrast collar and have a few shirts myself. It's funny how in America contrast collar became 'for the rich' and pinstripe and 3 piece suits are 'Wall Street.'
It's been a long winter and my waistcoat has really given me that additional layer to keep me warm since I don't wear sweaters between my shirt and blazer. Historically the waistcoat served as that extra layer before the mass introduction of heating.
As for contrast collar, I find the most versatile colours are white collar on light blue or light red. The only rule is don't match the tie to pocket square , otherwise try any combination until it suits you.
I suppose there is no real issue in matching the shirt, or a part of the shirt. White pocket squares with white shirts are pretty standard when putting together a business-formal outfit. There is a texture contrast between the cotton shirt and the linen or silk (cream) square, which helps. Also, the shirt is generally regarded as a background aspect of the outfit, while the tie and pocket square are both foreground pieces.