It's up to you, but I don't ever vibram the heels.
Occasionally I'll do the sole, but trust me, it's fine to leave as is.
The first 20 wears or so it'll seem like it's wearing down really fast and you'll get a bit worried about it. After that, it kinda doesn't change for a good 2 years of wear, and at that stage you can just get them re-soled. Re-soling costs about $100 and there are SF-recommended cobblers who'll do an excellent job, just as good if not better than the originals.
Of course, if you vibram you'll likely not have to get them re-soled, but you will eventually have to get them re-vibrammed. So it kinda defeats the purpose IMO.