Easily that much and more but compare that to a CPU with say a 3GHz clock ie. 3 billion ticks per second, that's not much.

Plus the other overheads of switching are already there - it's not cheap. I don't expect the overhead of reloading from L2 cache to add much (to repeat, I'm not an expert though).