Living in Santa Barbara, and having been up to the Los Olivos area where Neverland Ranch is many a time, I can tell you that this will be a traffic nightmare. The 154 and 101 can support the traffic, but once onto the side roads I suspect it will become more and more of a parking lot the closer one gets to Neverland Ranch.
There's some nice wine tasting to be had in the area though. Maybe pick a winery that's relatively close, get there early on Friday for parking, then hike into Neverland. Kohler might work, but then you're taking some side roads that may be private....
Jason <-- No, not going