That kinda depends on your overall build I think. I'm 6'3/175 w/ slightly more broad than not shoulders, ~40 chest. I wear an L in the Fisherman sweaters I've owned. Seems like an XL would be ideal, but an L could work depending.
Is it that you can't find any XL's in stock, or that you think they won't fit? It's the wrong season for seeking right now. In a short bit, you'll have plenty of options.
I'm shorter (5'9") and considerably stockier at 180, so I feel like that might be a stretch. I can't find XL's in stock for the particular sweater I'm in the market for, but I also don't want to pay full price for it. I saw someone on here selling an L in the Fisherman sweater in gray that I liked but I've never worn SNS before so sizing is speculative for me.