this is what i would do... over 2 days
Rent a car and head out on the LIE, take the Veterans Memorial Highway to Sunrise Highway...
Get off in Southampton and enjoy lunch at The Southampton Brewery the food and beer are delicious. from southampton get on montauk highway and continue East...
Now you can either take montauk highway all the way to the end and check out Montauk Point (which is beautiful) then return to east hampton in the evening or head straight to east hampton go to the beach for an hour or so then get dinner and stay the night then head to montauk the next day. Even if its cool out a stroll on the beach in east hamton is going to be soothing and delightful.
On your way home the next day you could jump off in Hampton Bays and take Dune road down to Westhampton Beach (stopping off at one of the beaches for a stroll.) Then stop in Westhampton and eat at Star Boggs restaurant. Then head home....
Here is a dining thread at Chowhound.com related to the Hamptons
http://www.chowhound.com/topics/340454
And here is the route i just suggested on google maps.
http://maps.google.com/maps?f=d&hl=en&geocode=&saddr=NY,NY&daddr=Southampton,+NY+to:East+Hampton,+NY+to:Montauk,+NY+to:Ponquogue,+Southampton,+Suffolk,+New+York,+United+States+to
une+Rd,+Suffolk,+New+York,+United+States+to:westhampton+beach,+NY+to:NY,+NY&sll=40.819006,-72.820129&sspn=0.690079,1.2854&ie=UTF8&ll=40.85589,-72.502213&spn=0.344848,0.6427&z=11&om=1
There is lots to see in the hamptons and some of the best afternoons can be spent just driving along the country and beach side roads looking at all the beautiful sights.
hope this helps.
Brendan
just a quick note. I would not do this trip leaving on a friday or returning on sunday... the best days to me would be leave thursday return friday or leave wendsday return thursday.