totally depends on where you get your motivation and exhilaration from ... if you like riding for the exhaustion, the miles on the road, and as a fitness/health regime, then do the road thing, plus it is more social because you can ride with a pack of like-minded buddies.
but there is also great thrill and fun coming from the technical stuff off-road ... bendy single-trails, muddy conditions, small drops, rocky downhills ... it is less a serious exercise but has a bit more video-game character to it. if that is what keeps you going (it is for me) then get a proper mtb ... and not a single-speeder or fixie either.
cheers.