There are a lot of options to go with long pants. Generally a casual derivative of a more formal shoe, like the the AE Elgin or McTavish works well. These shoes will have thicker soles and "rougher" leather than their more formal counterparts.
To go with shorts, you are probably going to want something sleeker than the above mentioned shoes. An Italian-style loafer works well, as does the boat shoe. You can also go with penny loafers, canvas shoes, etc. Take a look at the offerings by Sperry to get an idea of shoes in this vein. I generally try to avoid any shoes that have a clunky rubber sole that wraps up around the sides or back, but that is just personal preference.