That is my assertion as well. The simple physics of it require that filtering air effectively will reduce the air-for-effort. If you're not experiencing this, you're pulling air from around the sides of your mask.

(Note that not noticing it is not the same as not experiencing it.)