How about the Soggy Dollar on Jost van Dyke?
If you are offered the 'pistol' say yes
This was us pulling in. We loved this place. I've been all over the UK but the absolute best fish & chips I have ever had was at Soggy Dollar, and that's including some pretty nice places in London even. It was so fresh it was incredible. The bloody marys were beyond belief as well, and inspired my wife to go slightly crazy doing internet research and buying ingredients to try to replicate them. Would go back any time.
We swam from our boat to the beach, but I may have paid with a "soggy credit card" so I don't know how authentic I can claim to be. Still, it would make for one hell of a styleforum meetup if this was part of the process of getting there.