Part of becoming successful in an office situation is understanding the culture. What works for one situation does not work for another. The advice to stay within the general level of formality is good advice - especially when you are a junior employee. It is also important to recognize that the culture in a given area may be very different than another. The style of shirt/socks you might wear in business-casual in LA may be very different than what people consider acceptable in Arkansas. And the culture in that local can often dictate how receptive people are to those differences.
So, even within the same spectrum of formality, you have to understand how well the culture in your office/geography is receptive to different styles.
Now, having said all of the above, not everyone's opinion matters the same. It's the opinions of the movers and shakers in your office that matter most when you're going to push the envelope on style.
For example - my office is business casual. When you wear a suit you stick out like a sore thumb if you're not an executive. If you're below senior level management you will be mocked. It's just the way it is. 15 years ago when it was business attire, things were different. Now, you can choose to wear nice fitted shirts of high quality and quality dress shoes - but if you sport some wild colors (one employee used to) you push a bit too far outside the comfort zone of many people. For example there was a post in a thread here where someone was sporting some pretty wild socks with their AE shoes - that type of a style statement hear will probably get quite a few chuckles. But nice chinos, fitted shirt, quality shoes and matching belt probably gets you compliments.
At some point you have to determine what is more important to you - career success or individuality. You can be stylish while still conforming to the norms within your office. But when your personal style goes further and further outside the norms you risk hurting your career success.