I have nothing to back this up with* but my understanding is that simple carbohydrates like sugar are more readily converted to bodily fat. Dietary fat is digested and processed by the body into component parts that are less readily converted into bodily fat. It's not like your body just takes fat and stores it directly in your love handles unmodified.
