I routinely walk between downtown and the wharf. Often between the wharf and the marina. Sometimes between the Richmond and the wharf. A good weekend walk from the wharf through Fort Mason, Crissy Field and the Presidio and ending with a movie at the Four Star or the Balboa can be nice. Walking from the wharf to the Zoo would be a bit much.
And Filbert Street makes a nice shortcut to Polk, so long as I don't need to smell good.
I live in a van down by the wharf.