Because it's sizing down from your typical pants size, not natural waist size. I hear it's better now, but when I bought my NSes years ago I had to get 29s while I was wearing 32 in Levi's. You typically don't expect vanity sizing to that extent so newbies are usually wary when starting with APCs.
It's also to account for stretching. I've been burned before by that. It's denim, not wool. Just because the waist starts at 32 doesn't mean it's going to stay there.