No. Unless is 40 degrees and the sun is out, which is very possible. If its 10 and windy you won't be outside for 10 minutes.
Your jacket is just a shell(or softshell), correct? You have no insulating layers. You want at least 3 layers. A base (the one you have is a solid choice), an insulating layer and a shell. The base wicks moisture and keep you dry (and therefore warm), the outer layer stops the wind. Right now you have nothing in between. If you wanted to drop the mid layer, you would want to get a parka, such as this.
If you keep the base and shell, this vest would work for you as long as it does not get too cold, in which case you will want the jacket version of this.