As long as you are wearing it in the right season you should do it. Like others have said, if you have the confidence, people will say that and think you are stylish, if you have no confidence, they will think you are oblivious to style.
First off, the shirt has to be perfectly tailored as Hawaiian shirts are notoriously cut way too large. Second, the shorts are going to make it really hard to pull off correctly. I think some slim chino's or linen pants would be much more believable. Good luck and I would love to see pics of the final product.