Twills have more texture than royal oxford, do they not?
I don't really like gucci loafers by association, but I don't think they are objectively unpleasing. I think what you're suggesting is fine, but I feel like something like oxford or linen if you're going to wear a white shirt is better. I like white shirts with denim but you have to really make it not dressy either via texture or other pieces, and if you're wearing a jacket I think texture would be the way to go, and not cufflinks especially. I think generally speaking the kind of shirt you want in this ensemble are the non french cuff ones anyway.