Mmm.......I don't know... a good pair of sneakers or boots can really do a lot for an outfit, even with a simple tee and jeans. People focus too much on the top while ignoring the bottom. A bad habit!
I would say they're all equally important, though there's more room for variation at the top.
I personally think that fit is one of RFX's best. Not to pick on him, but if you remember in the past many of his fit pics were paired with levis, and since he's experimented with various designer jeans/pants his fits look 100x better. I think that just goes to show how important each individual piece can be.