I think the trouble here lies in different conceptions of the word "dress". Dress to the layman (almost everyone) nowadays refers to anything more than he would wear casually, or maybe anything he would wear for business, if that's more than casual. However, "dress" used to mean that which was worn after six, to formal events, etc. It used to be that men wore only white shirts after six, during dress hours. Even blue shirts were relegated to business only. Similarly, what we consider a dress shirt today often has a pocket, which would be out of place on a "true" dress shirt, as a dress shirt would not be worn for business, so it would not need a pocket. Now that "rules" are very loosely interpreted, wear your green shirt proudly for whatever purposes you like (including with navy or blue pants, probably). Just understand that you may confuse some people by referring to it as a 'dress shirt'.