Things may have changed since I had much time to shop at outlets, but I think a lot of it has to do with which outlet stores you hit. It seems to me that the outlets at many of the destination outlet centers (such as Cabazon here in Southern California) are like cheaper (in many cases quality as well as price) versions of outlet malls, where I see more of the made-for-outlet style clothing.
There were specific outlets, though, often more standalone, that had all first-quality merchandise, just overstock, past seasons, etc. A couple of those stores in Secaucus (not sure if I spelled that right) were that way and an Armani store I came across in Vermont or upstate New York (great black label and collezioni stuff at 20% of retail) and the Orvis outlet in Roanoke, Virginia.
Personally, I've always had my best luck at the discount stores (not outlets). When I was in college, the bargain bins at Filene's Basement in Boston turned up many Oxxford and Louis odd jackets for $50-100. But, again, that was some time ago, and things may have changed.