Learning and thinking are separate things. Today's models think without learning -- they are frozen in time -- but this is a temporary state borne of the cost of training. I actually like it like this because we don't yet have impenetrable guardrails on these things.
> If you look at the training data (the internet), the hardest and most ambiguous problems don't have a simple question input and answer response, they instead have a lot of back-and-forth before arriving at the answer, so you need to simulate that same back-and-forth to arrive at the desired answer. Unfortunately model architecture is still too simple to implicitly do this within the model itself, at least reliably.
Today's thinking models iterate (with function calls and Internet queries) and even backtrack. They are not as reliable as humans but are demonstrating the hallmarks of thinking, I'd say.
> If you look at the training data (the internet), the hardest and most ambiguous problems don't have a simple question input and answer response, they instead have a lot of back-and-forth before arriving at the answer, so you need to simulate that same back-and-forth to arrive at the desired answer. Unfortunately model architecture is still too simple to implicitly do this within the model itself, at least reliably.
Today's thinking models iterate (with function calls and Internet queries) and even backtrack. They are not as reliable as humans but are demonstrating the hallmarks of thinking, I'd say.