My favorite hypothesis: Ilya et al suspected emergent AGI (e.g. saw the software doing things unprompted or dangerous and unexpected) and realized the Worldcoin shill is probably not the one you want calling the shots on it.
For the record, I don't think it's true. I think it was a power play, and a failed coup at that. But it's about as substantiated as the "serious" hypotheses being mooted in the media. And it's more fun.
Hold up. Any AI that exists is an IO function (algorithm) perhaps with state. Including our brains. Being an “x completion” algorithm doesn’t say much about whether it is AI.
Your comment sounds like a rhetoric way to say that GPT is in the same class as autocomplete and that what autocomplete does sets some kind of ceiling to what IO functions that work a couple of bytes at a time
can do.
LLMs predict language, and language is a representation of human concepts about the world. Thus, these models are constructing, piece by piece, conceptual chains about the world.
As they learn to construct better and more coherent conceptual chains, something interesting must be happening internally.
Language is only one projection of reality into fewer dimensions, and there's a lot it can't capture. Similar to how a photograph or painting has to flatten 3D space into a 2D representation, so a lot is lost.
I think trying to model the world based on a single projection won't get you very far.
> LLMs predict language, and language is a representation of human concepts about the world. Thus, these models are constructing, piece by piece, conceptual chains about the world.
I smell a fallacy. Parent has moved from something you can parse as "LLMs predict a representation of concepts" to "LLMs construct concepts". Yuh, if LLMs "construct concepts", then we have conceptual thought in a machine, which certainly looks interesting. But it doesn't follow from the initial statement.
It's not wild. "Predict the next word" does not imply a bar on intelligence; a more intelligent prediction that incorporates more detail from the descriptions of the world that were in the training data will be a better prediction. People are drawing a straight line because the main advance to get to GPT-4 was throwing more compute at "predict the next word", and they conclude that adding another order of magnitude of compute might be all it takes to get to superhuman level. It's not "but what if we had a better algorithm", because the algorithm didn't change in the first place. Only the size of the model did.
To achieve compute and compression efficiency, it means LLM models have to cluster all similar patterns together and deduplicate them. This also means successively levels of pattern recognition to be done i.e. patterns among patterns among patterns and so on , so as to do the deduplication across all hierarchy it is constructed. Full trees or hierarchies won't get deduplicated but relevant regions / portions of those trees will, which implies fusing together in ideas space. This means root levels will be the most abstract patterns. This representation also means appropriate cross-pollination among different fields of studies further increasing effectiveness.
This reminds me of a point which my electronics professor made on why making transistors smaller has all the benefits and only few disadvantages. Think of these patterns as transistors. The more deduplicated and closely packed they are, the more beneficial they will be. Of course, this "packing together" is happening in mathematical space.
Another thing which patterns among patterns among patterns reminds me of homotopies.
This brilliant video by PBS Infinite Series is amazing. As I can see, compressing homotopies is what LLMs do, replace homotopies with patterns.
https://www.youtube.com/watch?v=N7wNWQ4aTLQ
There's entire studies on it, I saw a lecture by some English professor who explained how the brain isn't fast enough to parse words in real time, so runs multiple predictions of what the sentence will be in parallel and at the end jettisons the wrong ones and goes with the correct one.
From this, we get comedy. A funny statement is one that ends in an unpredictable manner and surprises the listener brain because it doesn't have the meaning of that one already calculated, and hence why it can take a while to "get the joke"
If the text completion algorithm is sufficiently advanced enough then we wouldn't be able to tell it's not AGI, especially if it has access to state-of-the-art research and can modify its own code/weights. I don't think we are there yet but it's plausible to an extent.
You're correct about their error; however, Hinton views that a sufficiently scaled up autocompletion would be forced, in a loose mathematical sense, to understand things logically and analytically, because the only way approach 0 error rate on the output is to actually learn the problem and not imitate the answer. It's an interesting issue and there are different views on this.
Any self-learning system can change its own weights. That's the entire point. And a text-processing system like ChatGPT may well have access to state-of-the-art research. The combination of those two things does not imply that it can improve itself to become secretly AGI. Not even if the text-completion algorithm was even more advanced. For one thing, it still lacks independent thought. It's only responding to inputs. It doesn't reason about its own reasoning. It's questionable whether it's reasoning at all.
I personally think a far more fundamental change is necessary to reach AGI.
I agree, it's an extremely non-obvious assumption and ignores centuries-old debates (empiricism vs. rationalism) about the nature of reason and intelligence. I am sympathetic to Chomsky's position.[1]
ChatGPT is not AGI, but it is AI. The thing that makes AI lose all meaning is the constantly moving goal posts. There's been tons of very successful AI research over the past decades. None of it is AGI, but it's still very successful AI.
Why wouldn't Ilya come out and say this? Why wouldn't any of the other people who witnessed the software behave in an unexpected way say something?
I get that this is a "just for fun" hypothesis, which is why I have just for fun questions like what incentive does anyone have to keep clearly observed ai risk a secret during such a public situation?
Because, if they announced it and it seemed plausible or even possible that they were correct, then every media outlet, regulatory body, intelligence agency, and Fortune 500 C-suite would blanket OpenAI in the thickest veil of scrutiny to have ever existed in the modern era. Progress would grind to a halt and eventually, through some combination of legal, corporate, and legislative maneuvers, all decision making around the future of AGI would be pried away from Ilya and OpenAI in general - for better or worse.
But if there's one thing that seems very easy to discern about Ilya, it's that he fully believes that when it comes to AI safety and alignment, the buck must stop with him. Giving that control over to government bureaucracy/gerontocracy would be unacceptable. And who knows, maybe he's right.
My favorite hypothesis (based on absolutely nothing but observing people use LLMs over the years):
* Current-gen AI is really good at tricking laypeople into believing it could be sentient
* "Next-gen" AI (which, theoretically, Ilya et al may have previewed if they've begun training GPT-5, etc) will be really good at tricking experts into believing it could be sentient
* Next-next-gen AI may as well be sentient for all intents and purposes (if it quacks like a duck)
(NB, to "trick" here ascribes a mechanical result from people using technology, not an intent from said technology)
But why would Ilya publicly say he regrets his decision and wants Sam to come back. You think his existential worries are less important than being liked by his coworkers??
> You think his existential worries are less important than being liked by his coworkers??
Yes, actually. This is overwhelmingly true for most people. At the end of the day, we all fear being alone. I imagine that fear is, at least in part, what drives these kinds of long-term "existential worries," the fear of a universe without other people in it, but now Ilya is facing the much more immediate threat of social ostracism with significantly higher certainty and decidedly within his own lifetime. Emotionally, that must take precedence.
He may have wanted Sam out, but not to destroy OpenAI.
His existential worries are less important than OpenAI existing, and him having something to work on and worry about.
In fact, Ilya may have worried more about the continued existence of OpenAI than Sam after he was fired, which looked instantly like a: "I am taking my ball and going home to Microsoft.". If Sam cared so much about OpenAI, he could have quietly accepted his resignation and help find a replacement.
Also, Anna Brockman had a meeting with Ilya where she cried and pleaded. Even though he stands by his decision, he may ultimately still regret it, and the hurt and damage it caused.
I think his existential worries about humanity were overruled by his existential worries about his co-founder shares and the obscene amount of wealth he might miss out on
For the record, I don't think it's true. I think it was a power play, and a failed coup at that. But it's about as substantiated as the "serious" hypotheses being mooted in the media. And it's more fun.