If I were you I would walk in rocking out (Means wearing it proudly) a sweatervest,If someone thinks that's "over the top" Don't worry about it, I would wear a sweatervest to school,Someone would ask me, Are you going to church tonight? I would reply no I just like wearing it.
I work in high tech and it is the same here - at least at the first 2-3 levels in the organization - but when we have to go to customers more often the dress goes up too.
I said screw it and wear a sport jacket everyday. A suit is too much I agree and I am considering seeing what the reaction to knit ties are when it starts getting warmer and I shed the jacket.
I am not a sweater fan as I need to adjust to all of the temperature ranges here so a jacket work better. Everyone is used to it now and I get very few comments any more. It also feels great to be the rebel by not wearing the uniform.
Thanks OP will appreciate your advise, considering that he is still among the living.