Why are the two necessarily unrelated? Can human being just be considered to be learning via optimization, and perhaps consciousness is an emergent property of an agent with a large enough world model, or a world model that includes the agent itself?

While I don't think a majority really thinks current systems are conscious, SOTA results are absolutely astounding (check out DALL-E 2 if you haven't seen it already). Whether or not an agent is conscious doesn't really matter from a practical standpoint (but obviously a moral one) in the long run - it is intelligence that matters with these agents, and they're getting absurdly more intelligent by the half-decade

A human being has an online learning system (which ML models usually don't) but it isn't just that. Even just the brain has emotions, motivation, hunger, self-interest, forgetting, some instinctual desires, and not only do those support the way humans are intelligent/conscious, but it's possible they're necessary for it, and you can't give them to an "AGI" without it losing the computer-like usefulness of an ML model.

Feelings aren’t an emergent property of intelligence, with intelligence being defend as some nth derivative of optimization capability. So I think no.

Human consciousness isn’t just a brain. It’s a system of which the brain is a part, occurring through time.

We are on an AGI winter in NLU. Cool openAI demos are cool and irrelevant. The HN crowd should really learn to go see the leaderboards by himself on paperswithcode.com then he would realize the reality, we are stagnant on the key basic tasks (e.g coreference resolution). While GPT-3 is just a subtile bullshit generator that push to its paro(t)xism the illusion of understanding that mere collocated statistics amalgamation provide, Dalle-2 on the other hand is very impressive but it does not advance the key NLU tasks and just show how far a smart trick (constrastive learning) can go before plateau-ing.

The idea consciousness emerge proportionately with the accuracy of your mental isomorphic world representation is cute however we don't become more conscious by becoming more erudite, and the most intense magical qualias, such as e.g orgasms are accessible to the simplest mammals and are unrelated to activities in the higher cognitive regions of the brain. Even a newborn that has no understanding of its surrounding experience qualias.

You can make the argument that stagnant progress isn’t actually not progress, when it comes to AI progress. Kilcher and Karpathy recently had a video where they discussed how some new model (PALM or Dalle2 I forget which) showed zero progress during X thousand training cycles, and then suddenly rapid progress after those training cycles. It was as if the model was spending thousands of training cycles on grokking the concept, and then finally grokked it. It could simply be that as we continue to increase the number of parameters and data quality on these models that we will continue to see progress on the route to AGI as a whole, but only in step change functions that require many training cycles

How much more parameters do you need? PALM is 530 BILLIONs and underperform in NLP tasks vs XLnet (300 millions), as such very large language model are extreme failures. They do not improve the state of the art once you have proper datasets and do full shot learning and I'm not even talking about fine-tuning.

Very large languages model hide to the layman that they are the gigantesque failure in NLP ever by showing they improve the state of the art in zero or few shot learning. Who cares this is so cringe. Full size learning is what matter the most and even full size learning do not yield satisfying accuracy on most Nlp tasks (but close enough) Therefore the only use of PALM is to have mediocre (70-80%) accuracy which is better than previous SOTA, only for tasks that have no good quality existing datasets. And 530 billion is close to the max we can realistically achieve, it already cost ~10 millions in hardware and underperform a 300 million model in full size learning (e.g dependency parsing, word sense disambiguation, coreference resolution, NER, etc)

It's crazy people don't realize this gigantic failure but as always it's because they don't care enough

Out of curiosity, what was the video?