This. In the beginning you will mess up how you want things to fit, what you like best, etc. Buy few, but decent quality items and build from there. College attire is inherently more casual in nature, so you are probably on track there.
Suggestions to get you started:
1. If this is mainly motivated by looking good for the womens, then go to the gym.
2. Invest in 1-2 decent pairs of shoes. One clean sneaker (such as converse) and one lace-up (I would recommend a plain dark brown).
3. Dark medium weight jeans that fit, and for summer I highly recommend a pair of Essex pants from J. Crew (I prefer the classic fit, it is a little more tailored but no obnoxiously slim).
4. 3-4 collared shirts/OCBDs (I recommend a white and light blue, then pick colors/patterns you like). I recommend a lighter cotton, but you can try linen, which is admittedly cooler, but wrinkles pretty heavily. Stay away from material that is easy to see through.
5. V-neck and crew neck t-shirts. Pick whichever you like (grey is a good color if you want a recommendation). American Apparel and J. Crew have more fitted ones and tall sizes. My personal favorite for summer is the grey Jaspe V-neck T-shirts from J. Crew. The particular heather pattern they use hides sweat well in the summer and they are reasonably cool. Urban Outfitters also has nice T-shirts.
6. V-neck sweaters.
If you have a fraternity or college event where people get "dressed up", I would highly recommend a navy blazer, a white collared shirt, grey wool slacks, and a black shoes with a leather sole. You will look nice without looking so far out of place. It will seem like a trivial outfit on these boards, but if you make sure the clothing fits, you will look much better than everyone else there. I have been to many of these functions and 90% of the guys will be in: a blue shirt of some sort that is too big for them, black pants, black clunky square-toe shoes, and a tie they took from their dads closet. Styleforum also as a general rule hates black shoes, but they are pretty normal in the real world. If you want to spice it up a bit, get a wing tip or something other than plain toe. Make sure the clothing is fitted and don't button the top button on your shirt. Match your socks to your pants, unless you have black shoes, then black socks.
Don't wear a fedora, you will be that guy. I would stay away from ties unless it is a real formal event.