A few years ago Filene's Basement (the original FB) in Downtown Crossing closed up shop to make way for a mixed use space in its location. This was in my opinion the best place to shop as they had the inside track on most of the post sale overflow from stores like Barneys, Wilkes, Mario's, Mr Sid, and Louis. Since this location shuttered and subsequent other FB stores have closed the discount opportunities here in Boston have died.
There are still the usual suspects at full retail like Neiman's, Saks, Barney's, and Louis in Boston. The 1st three are all in Copley/Prudential mall right in Back Bay with Louis being on the waterfront by the ICA. Newbury St is a good walk with Ralph Lauren, Scoop (never been) and Stel's. Take the redline to Cambridge adn you have Gary who owns Drinkwater's (close to Porter Red Line stop) and Andover Shop (Harvard stop). If you are coming from NY I wouldn't get too excited but if you are from somewhere that doesn't have some of the higher end stores listed above you can hit them all relatively quickly.
There are tons of great food options in Boston, Cambridge and Somerville too. I can offer some suggestions if you tell me what you want and where you are staying. Notables include Craigie on Main, The Hungry Mother, Butcher Shop, Hamersley's, Island Creek Oyster Bar, Neptune Oyster, etc. Many good places for beer bars as well: Deep Ellum, Redbone's, Lower Depths, Mead Hall, Bukowski's.
Forgot to mention Bobby From Boston - you need to go here. Vintage men's clothing without absurd prices. He has been written up in Esquire and has been doing this for years.
http://www.selectism.com/news/2011/03/28/bobby-from-boston-shop/