I personally like it. I do it every once in a while. It may end up looking better with a linen or cotton pocket square as opposed to silk. I have used all though. (Linen,Cotton,Silk)
I've been wearing the look for about ten years. I travel a lot with a single jacket, often a blue blazer so it is a simple way to add a bit of color to an outfit.
I wear glasses, and I clean my glasses with my pocket square, so I have one on me most days. Informal is a better look, I agree, but the way you wear it is as important as the kind.
I like the look. I may not wear a tie if I'm having a casual day, but I still like to have a pocket square. Plus, it allows me to get a little crazy with my selection since I don't have a tie to "match" it too.