The same way you avoid exposure to anything. Stay home, don’t go near sick people, etc. The point is that in the event that you are somehow exposed, it’s better to be vaccinated than not.

I wouldn’t tell you to not trust the advice of your doctor. But I would tell you that there is an always argument both ways for something like this. A different doctor may feel differently, and sometimes doctors change their minds when given a bit of pushback. It might be worth asking about the possibility of oral immunotherapy in addition to avoidance, rather than as a substitute for it.