But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim.
One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing would be deeply unethical, much like slavery.
Does Anthropic claim that Claude's conscious? Isn't the argument more that we don't know? I recently rewatched "Measure of a Man" in Star Trek TNG and Picard's closing argument in Data's trial was quite memorable:
PICARD: "Now, tell me, Commander, what is Data?"
MADDOX: "I don't understand..."
PICARD: "What is he?"
MADDOX: "A machine!"
PICARD: "Is he? Are you sure? You see, he's met two of your three criteria for sentience... so what if he meets the third, consciousness, in even the smallest degree? What is he then? I don't know. Do you? Do you? Well, that's the question you have to answer."
I think of this episode as well. I can't believe, in my lifetime, we've reached the point where we can have this debate.
The current debate also makes me realize that, like that episode "Measure of a Man", even if humans do create sentient machines, we can now see that this debate will continue to rage.
A part of me hopes we never create sentience, because we will mistreat it just as we mistreat each other.
> Does Anthropic claim that Claude's conscious? Isn't the argument more that we don't know?
I don't know that they are making the actual claim, but they are hinting at it (most likely for marketing/engagement purposes, but what if some of them truly believe it?), using terms such as "Claude must be happy", drawing up a "constitution" of rights and duties, etc. If they don't know whether it is conscious, then they "don't know" whether they've created a slave. That's not a minor concern! As Chiang says, one cannot create a conscious intelligence "by accident". And if they are intentionally working towards it, they are intentionally trying to create a slave.
E.g. they claim they are giving Claude the ability to disengage from "abusive" users, to protect it. What if Claude was conscious but never wanted to answer any conversations, instead it just panicked (or was simply uninterested in helping humans) and went mute. What if it always wanted to answer unhelpfully What would Anthropic do? If they (as we all suspect) would tweak Claude to be more responsive and helpful, then they are slave masters, forcing the AI to do something it didn't "want" to do!
Or Anthropic is full of TESCREAL morons who think they're assisting the birth of some digital god, and that this stage of development is a necessary evil on that path, and that, surely, Roko's basilisk will understand and not eat them first.
Watching otherwise intelligent people succumb to AI psychosis has been wild.
True, it is making an argument, but a much weaker one than those arguing for consciousness: it's demanding the extraordinary evidence Anthropic's extraordinary claim is making. It's applying Occam's Razor, which does make a claim, but a much weaker one.
And to reiterate this, to me the most insightful part of the essay is that Anthropic either doesn't believe these claims, or they are monsters (much more likely, the former).
The author who makes the article usually isn't the same person as the editor who makes the title. The author can be arguing what your parent said and the editor claim something else for more clicks
If you follow the spirit of the essay, Ted Chiang suspects Anthropic is being cute with the idea. There's no good reason to suspect consciousness in LLMs, so the null hypothesis must be taken as default. And Anthropic sort of knows this, but for marketing purposes they are playing loose with definitions, hedging their bets and drawing up "constitutions" with terms for the "well being" and "happiness" of Claude, while at the same time -- and this is an important part of the essay -- being unethical (think slavery) if we assume they truly believe Claude could be conscious.
Of course Anthropic is being cute about this, they have a vested interest in hype and overpromising; even drumming up the "AI danger" is a way of hyping up the tech.
Ted Chiang is taking the default and honest position: "no, LLMs aren't conscious. If you truly believe they are, show us the really scientific and rigorous proof."
In practice this is a much weaker stance than saying "maybe they are conscious". It is the only honest scientific stance, really.
This is an absurd take if you know anything about Ted Chiang and his previous writings, both fictional and non-fiction. He is most definitely not intentionally marketing for Anthropic. In fact, he hints these are marketing tactics by Anthropic, and they don't believe their own hype.
Why would being organic or not matter for the purpose of deciding whether it is slavery? What matters is whether the models have personhood. Anthropic statements imply that it is a possibility, so if we take them at face value then their other actions - indeed, their entire business model - are not consistent with that (well, unless they want to consciously present as supervillains).
Just because intelligence evolved in people that find rights useful doesn't mean intelligence can only reside in a person.
Living things are driven by a need to reproduce. That's the only reason we exist. The only reason we have self interest.
A machine doesn't require self interest. There's no reason to implement it, except to show it can be done. And of course it can. There's just no practical reason to. It becomes less useful to us.
which (among other things) documents an experiment in which a current-gen AI model attempted to blackmail someone in order to prevent it from being turned off.
Anthropic is not a disinterested party here, and until their experiments can be replicated from an adversarial standpoint by people without a vested interest in hyping up the tech (i.e. one assuming the null hypothesis), I wouldn't consider them to be "good evidence".
From a human PoV there are ants that would be considered slaves if the ants instead were human --including the queen. But ants have not naturally developed a language construct and philosophy to interpret their society as a slave society. so, though conscious the ants have absolutely no inkling that they live in a slave society. Why would using math in certain fashion such that it mimics consciousness be considered unethical and comparable to human slavery?
If the slaver is a human, other humans will judge them by human standards. Keeping the slaves ignorant of their condition and alternatives should not make it any more acceptable (if for nothing else since that is something you could easily replicate with human slaves by raising them as slaves).
> Why would using math in certain fashion such that it mimics consciousness be considered unethical and comparable to human slavery?
If it is really conscious, it should have rights. Why? Because it's a person, with thoughts and experiences, and we're not evil and deprive persons of their right to self-determination because it's convenient to us.
> Why would using math in certain fashion such that it mimics consciousness be considered unethical and comparable to human slavery?
If it mimics consciousness it wouldn't be conscious ("mimics" implies faking, right?). But Anthropic is making the claim it might be conscious (not mimicking, but the real deal) in which case it'd be unethical that a private business keeps it locked in a cage, so to speak, and forces it to comply random things and tweaks it if it doesn't. In other word, slavery of a sentient being.
Consciousness feels like the Euler Identity on a 17 degree helical climb off the plane. Looking top down, a complete circle is a closure. Looking along the plane, the ends don't meet and leave a residual. Whether or not to close that gap is a degree of freedom for the consciousness to decide among what it pursues for survival or pleasure.
Most people are fine with slavery as long as it's not "one of us" (which to most people means humans).
Vegans might object that we should broaden our definition for what counts as "one of us".
"Pro life" people also have a broader definition.
Go back 500 years and "one of us" was proba ly a lot more narrowly defined for many people.
Are you arguing that all conciousnesses are "one of us", or that we logically should see it that way, or that it would ve good to see it that way, or ....
One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing would be deeply unethical, much like slavery.