There are a couple of hubs for second-hand, or vintage, clothing in Tokyo (apart from flea markets) - one is Harajuku and the other is Shimokitazawa, or "Shimokita", as it's often called.
Shimokita has quite a village feel, with a more laid-back atmosphere than much of Tokyo. It has quite a few vintage clothing stores, plus some vintage record/CD stores, too. A few that I remember are "Ragla Magla", "Don Don Down" (which is part of a chain of vintage clothing stores), "Haight Ashbury" and "Flamingo".
There's quite a few others, too, including a couple of stores that have more of a slant towards military/naval-style clothing.
In Harajuku, most of the vintage clothing stores are clustered around Takeshita-dori and in the area between Takeshita-dori and Meiji-dori.
It helps if you're quite small, as most of the stuff in the shops is, in my experience, between a US34-38 in size (ie 28-32 waist). You'll still find some stuff if you're a size 40 but any larger than that and you'll most likely be out of luck. Of course, that also applies to shopping in Japan in general - most places have a greatly, greatly reduced selection of anything above a size 40.