According to
Secunia IE 6 still has 19 (of 106) unpatched but known problems.
Firefox
has 3 (of 36) (not counting this one) unpatched but known problems.
The noscript extension can be used to let you block javascript on any sites you don't trust.
On the whole I consider Firefox safer and less vulnerable. Of course you must judge for yourself.
The main thing though is like personal security you can be much safer just by watching where you travel and take precautions (such as the noscript extension) when you must travel the danger zones.