Which conditional probability sequences can be exploited for engineering utility cannot be known ahead of time; nor is it explained by the NN. It's explained by investigating how the data was created by people.
Train a NN to generate pictures of the nightsky: which can be used for navigation? Who knows, ahead of time. The only way of knowing is to have an explanation of how the solar system works and then check the pictures are accurate enough.
The NN which generates photos of the nightsky has nothing in it that explains the solar system, nor does any aspect of an NN model the solar system. The photos it was trained on happened to have their pixels arranged in that order.
Why those arrangements occur is explained by astrophysics.
If you want to understand what ChatGPT can do, you need to ask OpenAI for their training data and then perform scientific investigations of its structure and how that structure came to be.
Talking in terms of the NN model is propaganda and pseudoscience: the NN didnt arrange the pixels, gravity did. Likewise, the NN isnt arranging rap lyrics in that order because it's rapping: singers are.
There is no actual mystery here. It's just we are prevented form access to the data by OpenAI, and struggle to explain reality which generated that data -- which requires years of actual science.
It has a lot of things already encoded regarding the solar system, but it cannot really access it, it cannot - as far as I know - run functions on its own internal encoded data, right? If it does something like that, it's because it learned that higher-level pattern based on training data.
The problem with NN arrangements in general is that we don't know if it's actually pulling out some exact training data (or a useful so-far-unseen pattern from the data!) or it's some distorted confabulation. (Clever Hans all again. If I ask ChatGPT to code me a nodeJS IMAP backup program it does, but the package it gleeful imports/require()s is made up.
And while the typical artsy arts have loose rules, where making up new shit based on what people wish for is basically the only one, in other contexts that's a hard no-no.
Train a NN to generate pictures of the nightsky: which can be used for navigation? Who knows, ahead of time. The only way of knowing is to have an explanation of how the solar system works and then check the pictures are accurate enough.
The NN which generates photos of the nightsky has nothing in it that explains the solar system, nor does any aspect of an NN model the solar system. The photos it was trained on happened to have their pixels arranged in that order.
Why those arrangements occur is explained by astrophysics.
If you want to understand what ChatGPT can do, you need to ask OpenAI for their training data and then perform scientific investigations of its structure and how that structure came to be.
Talking in terms of the NN model is propaganda and pseudoscience: the NN didnt arrange the pixels, gravity did. Likewise, the NN isnt arranging rap lyrics in that order because it's rapping: singers are.
There is no actual mystery here. It's just we are prevented form access to the data by OpenAI, and struggle to explain reality which generated that data -- which requires years of actual science.