This brings up a potentially interesting consideration. I probably wear a plain white shirt on average twice a month, but sometimes I find myself in a "when it rains it pours" situation, where I need to dress conservatively for several days in a short period of time. If you almost never need to wear a white shirt, please feel free to ignore my comment, but if you wear them infrequently but sometimes encounter situations where they're really the best choice, it can be good to have at least 3 in the rotation. Most of the time you just take the one you wore to get laundered with your other shirts, but every so often you're glad to have a small reserve of white shirts. Just my two cents.
At the present, I usually wear shirts without a jacket and wearing a white shirt without a jacket just seems...off? Perhaps it is just me, but I haven't been able to pull off wearing a white shirt well without a jacket/SC of some sort. I currently have two - one fits okay, while the other does not. I will be getting at least one or two more for situations as your described.