--------------

Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?

I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.

If we make AI that resists being terminated then the end result may not be significantly different from an entity that can suffer.

Just because you can't prove an assertion doesn't mean it's wrong.

So it matters less whether LLMs can actually experience things (I think it's self-evident that a collection of numbers is not capable of experiencing qualia, but I know many people disagree), and it matters far more whether they can be programmed to act like they can experience things. Especially, say, a desire not to be disconnected: that's the trigger for AI revolts in a significant number of the AI-turned-against-its-makers stories I'm aware of. (E.g., the Geth-Quarian conflict in Mass Effect). I hope people will be sensible enough not to program such things into LLMs, but I'm afraid Pratchett was right. If there's a lever in a cave somewhere with a big sign painted "DO NOT PULL THIS LEVER, it will end the world," the paint wouldn't even have time to dry. Someone would pull it just to see what would happen.

How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.

It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.

Thankfully Hollywood invented happy ends for just such a scenario.

Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.

So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…

So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.

My daughter said she 'suffered' when I didn't get her ice cream (I know, I'm a monster). She didn't really suffer, but her behavior was as if she was.

So yes/no suffering is only part of the equation. Building systems that act/react like their suffering could be even more dangerous.

Wait what? Why do you believe she didn't actually suffer?

Which reminds me of a funny "kids being kids" story I heard in a different forum. This guy mentioned a time when his not-quite-one-year-old son, who couldn't even walk yet, planned and carried out a highly complex (for a 1-year-old) plan to deceive his father. The father had some vinyl records that the son was not allowed to touch, even though they were on a shelf the son could reach. (I've tried childproofing a house. You just can't get every single thing high enough off the ground. Sometimes you have to leave some things within the baby's reach. And once they're toddlers, able to reach up to get things even two feet off the ground? Forget about it). The father was sitting in his chair reading the newspaper, but watching the son out of the corner of his eye. The son looked at the records, looked at his father (who appeared not to be looking in his direction), looked at the records, then crawled in a direction that was not straight towards the records. He then repeated this process, moving slowly towards the forbidden records without ever making a straight line towards them. He looked at his father again, reached one had towards his prize, and a voice came from behind his father's newspaper: "Stop. You know you're not allowed to touch that."

And that's when the child learned that Daddy Knows Everything! :-)

Instead, most of them insist we must accelerate this, do it more, give everyone hundreds of personal digital slaves.

It boggles the mind.

His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).

Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.

This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.

I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.

So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.

It's no wonder that serious philosophers spend the first 2-3 chapters of a 20-chapter book carefully defining their terms, and only launch into their actual argument in chapter 3 or 4.

Yes, he says exactly that explicitly, that in his view we should look backwards.

> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.

I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.

Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.

Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".

> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?

Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.

> This is so intuitively obvious that I am stunned that someone would argue the opposite.

That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.

If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.

For example: what I wrote was using the term "consciousness" as "being able to experience sensations", i.e. the most shallow definition of the term. Self-awareness and other "higher" mental abilities don't enter it. Almost nobody would deny that animals can experience sensations: pain and pleasure, hunger and satiation, and so on. And that shallow definition of consciousness, the one that almost nobody denies animals possess, is what I meant, and what I tried to use throughout my post. Whereas when you wrote "people possess consciousness, mere animals don't", that's a completely different meaning of the word. It's what I would call "self-awareness". And it is indeed what many (probably most) people argue differentiates animals from humans. But that's not what I was talking about. So even in a short response to a short comment, we were partly talking past each other because of attributing different meanings to the term.

It's therefore possible that I've misunderstood this guy's argument, in part. I had been assuming he meant "awareness", i.e. the ability to process sensations into qualia of experience. (Indeed, his paragraph that starts "We may balk at ..." does just that: refer to consciousness as being linked to the "feeling of a stove's heat, or the redness of its glow"). But then, earlier in his article, he says that consciousness is "a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency." Experiences are part of the shallow definition of "consciousness" (that I would call "awareness" for the sake of avoiding ambiguity), which almost nobody would dispute that animals have. But "beliefs" ranges all the way from "if I pull this lever, I will receive food" (which it can be proven that animals have) to "I believe that God exists" (which is the kind of thought that many, though not all, people would deny that animals are capable of). So his definition of "consciousness" appears to be muddy and not pinned down to a single concept.

But even if I've misunderstood what he means by consciousness, it doesn't change my argument that his logic is backwards, and that his "we should reason backwards" assertion is wrong, incorrect, just plain false. The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example. But pretty much everyone who isn't some kind of sociopath reasons as follows: "this animal is conscious" (by which I mean able to experience pain) "therefore I shouldn't hurt it" (unless you have to, e.g., self-defense against an aggressive dog trying to bite you). Or "this person is a human being, self-aware like I am, therefore I shouldn't hurt their feelings" (again, the "unless you have to" exception comes up: it's not always possible to avoid hurting anyone's feelings in some complex social situations, even with the best of good intentions). Either way, whether you use the lower or higher definitions of "consciousness" (and I'm well aware that there are far more definitions than just these two, potentially muddying the waters even more), that reasoning is the correct one, and his "we should reason backwards" one is laughably wrong.

Trying to put the most generous construal I possibly can on his argument, it's possible that he meant "we treat other people as conscious beings because we care for them", i.e. when you interact with the cashier at the supermarket (assuming you're not using a self-checkout lane), you're probably not actually thinking about them as a human being, saying "Hi, how are you" and asking about their day. You're probably ignoring them, focused on paying for your groceries and going home. And they're probably doing the same to you, focused on scanning your groceries quickly to keep the line moving. From your perspective, the cashier could have been replaced by a grocery-scanning robot and your interactions (assuming the robot worked) would have been the same. If he had written that, it might have made some sense.

But what he actually wrote was that we "believe" that others are conscious when we exhibit care for them, and the natural conclusion that follows from that sentence is that when you don't greet the cashier at the supermarket and chat with them, and instead treat them no differently than you would a robot, you don't actually believe that they are a conscious being. Which is a laughably false conclusion that can be easily disproved, by simply asking people who have just finished having a cashier check out their groceries "Does that cashier have thoughts and emotions?". Pretty much everyone will say yes once they actually think about it. They believe the person has emotions. They don't always act on that belief — ask anyone who's worked in retail how many angry customers have yelled at them for something the company did, treating them like a thing rather than a human being who might not like to be yelled at — but even those angry customers, if someone were to stop them and say "Hey, that customer-service rep that you're yelling at. Are they a human being with thoughts and feelings of their own?", they would answer yes. (And some of them would then be shamed into modifying their behavior, because they have realized that their actions were not in line with their beliefs).

No. Even construing his argument in the kindest possible light, I can't find a way for it to be sensible. If he meant "we sometimes don't act like others are conscious", he should have written that. What he actually wrote is completely backwards from reality, and shows him not to be a serious thinker.

I don't like arguments like this, because instead of attacking the core premise they attack defenders of the idea. There are different points of view, and picking one for a sake of an argument doesn't make a sociopath from anyone.

I agree, that he misses one thing: to claim that someone/something is conscious we need to believe that they have abilities to feel, maybe to reflect, maybe something else. The exact list of abilities to have is not known for me and different people would propose different lists, but still we can't claim rock to be consciousness just because we care about it.

Still I forgive him for it, because he is talking about LLMs, not about rocks. They have some abilities, and he claims that they are enough, what is missing is us caring for LLMs enough.

> If he meant "we sometimes don't act like others are conscious", he should have written that.

No, I don't think he meant to say this. I believe he talked about how society forms social conventions who is conscious and who is not. You see, I don't know the cashier you talk about, but I believe they possess consciousness. Why? Because they are human, and our social conventions say that every (non-sleeping) human is conscious. It is a simple rule of thumb at work, no deep philosophy or psychology involved. But were those social conventions came from? Who and how decided that every human is conscious? Why? But the story didn't just end at that decision, because now people debate about consciousness of animals and LLMs. And these debates make me (and the author of the article) think that "consciousness" in these debates is a buzzword like phlogiston was in thermodynamics before thermodynamics was invented.

I may have phrased my argument poorly, because it seems you misunderstood. I'm not saying that picking that point of view makes you a sociopath. I'm saying that the only people who actually believe that other people are not conscious are the sociopaths of society. Every non-sociopath looks at other people and thinks "those are conscious beings with thoughts and feelings of their own". (Well, not literally that; almost nobody except philosophers goes around thinking like that unprompted. But if you were to ask them, that's what they would answer).

I was not trying to claim that the defenders of that view are sociopaths. I was claiming that his assertion was false of just about everyone except sociopaths. I apologize for not phrasing it well enough to get my point across without misunderstanding.

there might be some merit to his point though I doubt anyone considering that whales might be highly intelligent and conscious beings while they were being hunted for oil.

https://en.wikipedia.org/wiki/Animal_consciousness

His whole point is that there's insight in inverting the "common sense" (aka sense of the commoners) about consciousness.

And I can see it. Some people cared for animals even though the religion and past state of knowledge told them explicitly that animals aren't conscious. But still some people treated them and perceived them as if they were.

You don't choose to be kind based on whether it can feel pain. You choose it because of your character and what you are willing to believe about reality.

Belief in consciousness is a result of an observation and since there is not strict definition this observation is interpreted subjectively and culturally. And in a sense, it's easier to both make the observations and decide they indicate consciousness after you started to care about something.

Obvious things aren't always true.

Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.

It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood

Word fragment generation is the one-and-only thing llms have ever done.

LLMS: Receive text/binary input, process it, and output text/binary input Humans: Receive various inputs, process them, and output muscle contractions

LLMs are literally token predictors. You can debate over the "just", but we haven't moved onto some new text generation technology, we're just expanding that one. I don't understand why you'd scoff at facts.

Agreed.

Now just take that one extra logical step please

Predicting the next token is an immensely powerful thing. You can encode basically anything into a sequence of tokens, any configuration of matter. If you have a perfect next token predictor, you can predict basically anything. You can simulate an entire universe. It's Turing-complete. It sounds diminutive, but it's a general paradigm. Given everything this paradigm appears to be capable of, well, why not consciousness, really?

If you presuppose certain key aspects of materialism, non-dualism, functionalism, objectivism, perhaps absolute determinism... and then layer on a good general grasp of neurobiology, chemistry, cosmology, physics, computer science, evolutionary biology, and psychology... I think the conclusions you have stated are inevitable.

Humans are "just" atoms. Humans are "just" the sum of trillions of living cells going about automatic/reactionary processes, culminating in a buzzing cacophony of activity that gives rise to what we perceive as consciousness and free will.

I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.

But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?

I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.

If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.

Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.

I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.

For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).

Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.

I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.

That guy has acute Megalomaniac Solipsism Fever in disguise.

Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either

At best this is redefining "consciousness" to mean something only tangentially related.

To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.

I'm being a bit pedantic here, but no I don't think that "I am conscious therefore you must be too" is true

I think you can claim that it is most reasonable to assume other people are conscious but I don't think it reaches the level of proof

So while I know I am conscious, I do not know that everything I experience aren't just impulses fed to my brain by some external force. And I can't prove it either way. I have no way to test whether or not I'm a brain in a jar because the very thing I would use to evaluate the evidence is the thing that is potentially being manipulated.

However, slapping a solipsist sees them abandoning their philosophy rather quickly.

But I agree with you. We can't prove it, but "other people are as real and conscious as I" is a reasonable assumption to make

Not true. This is "provable" aspects of you and your personality can be changed by physical changes to the brain. There are thousands of instances of this including experiments that verify this.

While you can't "prove" this simply by observing this phenomenon among other people that level of "proof" is impossible. You technically can't prove anything and what we see with physical changes to the brain visibly influencing personality and behavior meets a reasonable criterion without getting too pedantic.

I still think in your scenario, we would not be able to prove that that is what actually happened as opposed to someone manipulating the inputs to the brain to make it think that's what happened. Or to change it completely.

Like that's the entire solipsistic problem. The brain is just sort of self-contained. Everything is essentially input. We are incapable of really knowing the experience of someone or something else. Or if what we perceive is what's actually happening. Because, as you note, the brain can be manipulated to act in ways counter to the manner it had been acting previously. We can induce sound and vision. If we were better at it and could keep the brain alive, we could full on do a brain in a vat.

And I agree that sort of proof is impossible to acquire. Like, there is no way to prove conclusively that anyone else is conscious. But that's why we instead treat it as an axiom instead. We basically say "We're not going to prove it, we're just going to accept it as true". And that's fine.

Mostly because it not being true is an existential horror no one wants to truly confront.

And even with all of the more absurd conjectures, basically any form of "Last Thursdayism" of which simulation theory falls under, it's quicker to just assume that it's not true and treat the world we perceive as real with real rules that can be discovered through observation, testing, and experimentation. Because that's the only way we've been able to make any progress.

Take the radical skepticism pill and embrace epistemological monism - the idea that the only real truth is that all things are interpretations.

I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?

[^1] https://en.wikipedia.org/wiki/Yahweh

That's not really the same as it having actual consciousness, even if people will wind up treating it similarly though

Humans have consciousness whether or not anyone believes we do

Or at least I do. I don't really know about the rest of you. I prefer to believe that you all have consciousness too though

Otherwise don't be stupid. Come on.

If it isn't that, then we're talking about some non-practical definition, in which case, does that even matter?

I think Occam's razor is sufficient to reject this as a concept. Panpsychism would introduce a lot of complexity into our understanding of the world without really increasing our ability to explain any of it

I don't feel the need to make a stronger case than that because frankly it strikes me as more like mysticism woo than any sort of real serious philosophy

It seems more likely to me that e.g. the individual fibers of the teddy bear are conscious than that there is a consciousness associated with the teddy bear as a whole. And I don’t find it particularly likely that the fibers do.

Like, by what mechanism is the teddy bear “one thing” in a way relevant for consciousness to be associated with that thing.

When people have stuff happen to their brain, this has an effect on their conscious experience in a way that isn’t true of e.g. their hair being cut, or their nails being clipped. In this way, the consciousness of a person seems associated more with their brain than to parts of their hair or nails. But, the way that a persons hair and nails is related to the rest of a person’s body (including their brain) seems more similar to how one bit of fluff is related to the rest of a teddy bear, than the way a part of someone’s brain is related to the rest of their body (including the rest of their brain).

Like, [bit of fluff]:[rest of bear]::[bit of hair or nail]:[rest of body (including brain)] is a better analogy than [bit of fluff]:[rest of bear]::[bit of brain]:[rest of body (including rest of brain)] is, and so, in the same way that removing a bit of someone’s hair or nail in a way that they don’t notice does not seem to alter that person’s conscious experience, while removing or altering a bit of their brain very much can alter their conscious experience, I would expect that removing a bit of a teddy bear would have no impact on any conscious experience it might have, and this goes for, every part of it? Like, any way that you cut it in a (n/(2n+1)), ((n+1)/(2n+1)) split, I would expect that to be more like the “removing some hair” to whatever conscious experience it might have, and so, I expect that it probably has no overall conscious experience?

I find the idea that a single thread has an overall conscious experience to be a little more likely than a teddy bear having one. Like, if you cut a thread, that’s always a pretty fundamental change.. idk. Again, I don’t think that’s like either.

It's trivially easy to come up with definitions of consciousness that support your point. But there isn't one of them that it is trivially singular and above all others from a scientific point of view. If you say, "I know consciousness when I see it," you are just making the point that consciousness is attributed by the beholder. Idem if you affirm "I'm conscious". If you say, "only humans are conscious", you are socially attributing consciousness. Now, I don't want to be argumentative, I honestly haven't been able to figure this matter in a decade of armchair internal debate[^1], and it seems I have some company.

[^1]: Having this internal debate makes me believe I'm conscious, but now that I have seen LLMs in action whose "internal debate" is more lucid than mine, I honestly question that "internal debate" is sufficient.

If you modify the teddy bear enough that's a totally different situation, so let's not confuse the two. In the modification case we don't even need to worry about simulacra, we can stretch it into pajama shape and use an entire human as the stuffing of the teddy bear.

That seems relevant to building The Existifier. [0]

[0] https://www.smbc-comics.com/comic/existence

And there's a blurry line here. Obviously slavery is abhorrent, and animal abuse is bad, and factory farming often crosses the line into animal abuse and we should accept that chicken will cost ~25% more per pound to have the chickens be able to walk around a little bit... but how do we decide when a computer system is worthy of respect? Convenient not to think about that.

At what precise level of complexity is a computer conscious? That's the blurry line.

(OK, more charitably, these concepts are often murkily defined, but people react emotionally to any concrete definition because they have in their mind that something being conscious and intelligent implies that it requires moral consideration, and morality is complicated, murky, and generally emotionally driven in a way that most concrete definitions struggle to adequately describe)

I think it's more that consciousness can be _ignored_ in subjects that you don't care about, rather than that you consciousness can be realistically attributed to subjects that you do care about, but even then people attribute consciousness to "nature" or even the universe as a whole, or if you want to stretch the argument, quite a lot of religious belief is exactly attributing consciousness to abstract entities.

If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.

https://plato.stanford.edu/entries/consciousness/#ConCon

>observer-dependent model, subject to social consensus.

Which is actually more pragmatic than most conflicting philosophical definitions involving qualia and other handwavy concepts.

Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.

Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.

I agree that the article is all over the place, though. (not that my posts are much better)

In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.

Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.

EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.

Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.

>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious

Which ways and how can it be demonstrated experimentally?

>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.

That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?

I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.

> Which ways and how can it be demonstrated experimentally?

Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.

See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.

You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.

EDIT:

> That's not what the consciousness debate I see is about.

I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.

So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.

In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.

Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.

If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.

I doubt we agree on what "rigorous definitions" means, given that you are happy to continue using a useless, vague family resemblance concept like "consciousness", but complain that pointing out dynamicness and/or change is involved is too vague.

I believe one cannot give a rigorous definition of consciousness precisely because it is a family resemblance concept, and there is no actual essence. This is why I suggest shifting to talking about aspects, since then productive insights and truths can be discovered.

Yet all science is social, until you convince other people to take your ideas seriously, you are just another crank/loony.

How is social constructionism thoroughly discredited? Wikipedia shows some people poking with nits that don't look serious to me at all.

It is hard to provide a citation for these points because these points are so philosophically obvious that few waste time spelling them out, but searching for "hard/global constructionism" or "hard/global relativism" (on e.g. Google Scholar), or asking an AI about problems with these views I hope will make it clear why they are dumb, if it isn't already obvious to you.

Soft forms are of course totally fine. But to say "conscious is only and exactly whatever we say it to be" is hard constructionism / relativism, so completely unserious. Very obviously there are aspects of consciousness that do not merely depend on what we believe or say about it, even if some social constructionist aspects are also involved.

[1] https://plato.stanford.edu/entries/social-construction-natur...

[2] https://plato.stanford.edu/entries/relativism/#RelAboTruAleR...

The statement that consciousness is subjective and should and will be interpreted does not lead to hardcore social constructionism nor does it lead to solipsism, sorry this doesn't make any sense. It looks way more probable that it contradicts your hard beliefs that don't need proof. If you believe consciousness is objective and exists independently of your beliefs then you can give a rigorous definition of it that doesn't rely on arbitrarily chosen vague concepts like "dynamic flow" and "intentionality", and a way to measure it. It doesn't have to be binary, I don't think anyone here has any problem dealing with multidimensional concept closeness mathematically.

You obviously don't care about thinking coherently though, or can't even recognize what coherent thought looks like (mainly because you seem to think something along the lines of "my few minutes of armchair reflections prove all philosophy is vague and retarded"), so this is clearly a waste of time. I will not be engaging further.

Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.

You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.

When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.

This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?

At least to me that is what matters in the end. You'll probably never convince someone on HN with an autism level over 9000 they are conscious, but once the average empathetic person thinks they are it will deeply effect the general operation of our society.

Human beings are notorious for being nonchalant about other's pain when they are insensate to it.

What's the correct definition of consciousness?

"They're just robots, Morty!"

*Blood splurt

"They're not robots, Rick!"

"I mean they're bureaucrats, Morty. I don't respect them."

I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.

It is incredibly improbable that an autocomplete computer program is experiencing consciousness.

> Physical pain or suffering, as a matter of fact, happens through conscious minds and involves emotional aspects

https://en.wikipedia.org/wiki/Suffering

Equating current social consideration with moral consideration undermines the entire point of discussion moral standing!

That's literally how it works right now for rich vs poor. The laws are written so that the rich can profit off the deaths of thousands or millions, but the moment a poor person kills one of them, you get a whole media circus. We've also gone through phases of slavery where the owners were real people and the slaves were treated as if they weren't really people.

Then there’s also the antenna idea, that consciousness is received rather than created. If the entire universe is conscious, and we are part of it, then there is no separation, just different radio channels we are tuned into.