This is a good question.
I really like the look of a nice white dress shirt with clean cut dark pants but like you say, it can look a bit formal in some casual settings (class for instance). I think the key to not come acros as brat or too formally dressed business student is to play with proportions, cuts and going for nice details.
I usually wear a white small collared COS shirt with a pair of self cropped black Uniqlo pants. Looks pretty cool given the price. Other options are hidden placket, shirt being very long in the back, elongated rise on the trousers, stacks, cropped, etc...
The shoes play a big role of course. You probably shouldn't go for "menswear" derbies if you worry about looking too formal. A wedge sole derby or side zips is probably a wiser choice. Common Projects and Red Wing postman look good and are affordable.
Thanks, I definitely got know what you meant. I'd wear white placket shirts, elasticated waist cotton or wool trousers and black derbys or boots all days, but sometimes I feel w/o knit or bomber it would be just too formal looking for class or office job for example. Even when I do know that my clothes aren't regular menswear. They're more like jil, dries, cos kind of clothes and I'd never wear ties and such. Maybe I'd need start wearing more playful sneakers? to me derbys and boots are too cool to be forgotten lol