If you must go, I'd recommend staying at The Cavalier - which marks the northern end of the strip of commercial development along the boardwalk (42nd street). Lots of amenities and your choice of the historic property on the hill across Atlantic Ave or the modern property on the ocean front. Close enough to the kid-friendly attractions you mention - but not too close.
The ocean front becomes residential north of the Cavalier, and the beach becomes wider, nicer and less crowded the farther north you go - up through the 80s until you get to Fort Story.
There's a Ramada that is sort of an outlier up in the 50s which would be a good call if avoiding the crowds is your priority.
I used to spend a lot of time in Va B in the 1990s at a friend's house in the 80s. Beach is very nice, wide and uncrowded up there. All of the commercial, touristy stuff south of the Cavalier might as well be on the moon - total non-issue.
Doc Watson's for brunch. Taste Unlimited for lunch/sandwiches/take-out. One Fish, Two Fish for dinner.