What I'd do with the extra fabric is use some of it to have what might be termed a "belt-buckle stabilizer" made and sewn on to the waistband. I'm not sure of the accepted term for what I'm suggesting, but it is a small, half-size belt loop placed right at the longitude of the fly on the waistband, and is used to run the buckle prong through when cinching up the belt. It functions to keep the buckle--and hence the belt--precisely centered on the waistband, preventing it from "migrating" up or down (or, for that matter, from side to side), something that can look lousy in a dressy outfit. I've had my tailor make these for me with the cuttings at the cuff when having new trousers hemmed, and they really help in keeping the buckle precisely centered. Of course, it will only work with standard dress-type belt buckles with central prong.
Great idea, Bengal-stripe about keeping the fabric for invisible mending later if needed.