It's a very country specific thing as well. In the US and UK it matters very much, same for France, but for other European countries it matters much less where you went to school.
In Holland, for example, no one cares where you went to school, and in general people don't talk about it or make it an important factor in professional or social life.
On the other hand, in the UK, you'll have people from Oxbridge who only talk to other people from Oxbridge, and still talk about those 3 years they spent there 30 years after the fact, and probably work in firms that only hire Oxbridge people. Groupthink much?
I think a large part of what went wrong with banks before the financial crisis is actually due to this groupthink, caused by only hiring people from similar backgrounds. Luckily things are a bit more diverse now.