I guess I can throw my .02 in to this.
I worked at a mainline Nordstrom for about 2 years and we'd constantly send perfect quality firsts that were out of season to the Rack. I don't think it's unreasonable as an uneducated consumer to expect that Nordy's would mark seconds. All items that were altered or worn are marked as such (in my local Rack at least). All jeans suits etc are shown as altered or washed on the tag. Shoes are marked as preworn on the tag. If I am not buying something that is marked in this way I'd expect it to be first quality of the label. Maybe to OP didn't notice it on the price tag, then it's on him, but maybe it wasn't disclaimed.
I think it's more on Nordstrom than AE if they truly were seconds and not disclaimed and I would bring the issue to them. They have a good replacement policy (hint: I had to exchange 10 Nordstrom brand dress shirts that were more than 5 years old for a lady because the elbows ripped out after that much wear) but, to have a clear conscience, you'd have to have proof that it wasn't on the original tag at the Rack.