The models are still ingesting text, are they not? Those framework are providing textual guidance to what the task at hand should be aim for. Those are formalising part of the context passed to the LLM, regardless of the model itself.