I think slim/fitted is just one of a number of menswear rabbit holes. For most guys in decent shape, even mall-level clothes of said persuasion are a step in the right direction. Relaxed clothing below the SF-approved level (e.g. Yohji, Harnden, Cabourn, EG, etc.) is generally brutal. Think casual fridays, gangsta warmup suits and logo tees, early-to-mid 2000s skate / snowboard wear, electronics salesmen. Fitted clothing is pretty accessible and not overly terrifying to someone trying to upgrade their look. Dark tapered jeans, proper t-shirt, fitted button down, non-obnoxious footwear... done. Add a sweater and a pea coat for F/W. Charcoal blazersuit and tie for weddings, decent restaurants, and third dates.
In contrast, pulling off a loose and flowy wardrobe with substance requires a much thicker wallet and/or balls of steel. And quite possibly a forgiving partner and an open-minded social circle. I would surmise that one would need to either be already living or assiduously craft a lifestyle befitting the wardrobe.