And Florida and the Caribbean, in the 21st century, are sufficiently different from the rule's 19th century New England origins, that you could probably get away with a great deal of flexibility.
Take all the relevant factors into consideration, then decide for yourself. And if you decide to deviate from the Labor Day rule, do so confidently. Few things look worse than a man who is uncomfortable with what he's wearing. Who is unsure as to whether he's dressed appropriately. Who is silently second guessing his sartorial choices. Conversely, few things will make you look better in your clothes than will confidence and self-assurance that you look right.
If you simply do not have it in you to decide for yourself, then identify a few men whose taste and judgement you respect, and see what they wear when they're in situations similar to those in which you find yourself. And taking into account certain personal differences between you and them (differences in age, build, etc.), copy them. Well, not slavishly, but at least let what they wear serve as your guide with regard to what's appropriate for you to wear, and when, and how.