I'm not very, very pale, but I'm definitely a bit pale in the winter months. Light grey and white wash me out as well. I agree with acecow in that "brown, beige, green, blue and purple" work well. I think burgundy can work well, though brighter reds don't work for me at all.
Another thing to consider is on the opposite end of the darkness spectrum. I think dark navy, dark charcoal, dark chocolate brown, and black can also be tough as they create such a stark contrast. I have better luck with mid-blues, mid-greys, mid-browns. One way to work the darker colors into your wardrobe is to wear them as layering pieces with complementary colors between the "bad" color and your face (e.g., you could wear a light blue or purple collared shirt under a chocolate brown v-neck sweater). Also, you can possibly wear a cream shirt in place of a white shirt, which can minimize the washed out effect without losing the nice contrast a white shirt can provide against darker shades; this suggestion can be dicey, though, depending on your exact complexion.