Getting in, say, video frames instead of words is more difficult, but possible imo. After all, they demonstrated it’s possible to do with raw audio: https://openai.com/blog/jukebox