It's the opposite, when you are wearing one of those masks you touch your face more often. It also is a warm damp environment for bacteria.
Now if you are sick and you are travelling to the doctors or something, then yes, a mask is a good idea. If you are healthy and trying not to be sick a mask is a bad idea, gloves and frequent handwashing with soap and water is a much better plan.
Why would you touch your face more often? At least you won't have a direct contact with your hands. Besides, why do you think putting on a mask and washing hands are somehow mutually exclusive?
To adjust the mask. You won't have direct contact with your hands, but the mask is a better environment for disease than your skin is, so touching it instead of your skin isn't any better.
People adjust the masks often. They aren't very comfortable and it's unpleasant to wear them for long, they can feel itchy which promotes face touching. The seal around your mouth probably won't be great unless you've been fit-tested as well so you'll be tempted to touch there.
All of those sound like issues that would be fixed by using better masks, that fit snugly and feel comfortable, and can be washed or even medicated, instead of the very cheap disposables.
