I'm no Zizek, but when I was just starting out (as a lecturer), I taught in white t-shirts.
Now, I wear a coat/tie...but still remain in the minority on my campus (and even more of a minority for wearing a PS).
There's another discussion for "formality/uniform" and context where it is required. I think the classroom is one of those more formal contexts, but people these days like it casual on campus. Certainly, a public scholarly lecture demands a coat and tie. Sweats, no.