The answer from a legal perspective is "no" BUT neither should you disparage him in any way if he puts you as a reference and you are contacted by a potential new employer. All you do is say he worked for you--confirm it--and decline to elaborate. It's sad but true that crummy employees have managed to flip things this way but this is standard protocal in any personnel office these days. I am sure you will get many similar replies. Good luck.
6/29/11 at 3:13pm