I wear no-show socks and loafers frequently during the summer. It's a staple North East preppy look, so I don't find GMMs bare ankles jarring at all. I do think that the grey pants are what makes it look more formal, though, and might make the socklessness appear incongruous. With linen, chinos, Nantucket reds, etc I don't think it would have been noticed especially.
Just my 2c.