Yes, this is the trade off. If you are working in an industry where you need to be highly sensitive for data access even for short periods of times then oauth/oidc/jwts are probably not for you. If you really need an emergency escape hatch you can always rotate your singing keys and jwks and invalidate all of your tokens and force everyone to sign back in.