Exactly this. There is a crossover point where their pain of listening to your horrible French exceeds their enjoyment of watching you suffer, and that's when they let you off the hook.

In Paris in the summer this crossover point is higher since I think they're pissed they are stuck in the city for the summer dealing with tourists rather than frolicking in the countryside.