How should we approach machine personification?

Except you are accepting the ‘fiction’ part without doing the ‘distinguish’ part.

I think that’s part of it for pets, but the same cannot be said for animals which are used exclusively for work or food. In those cases we just ignore their benefit entirely.

A thermostat cannot have a goal because it does not have intention. It can be in either state ‘on’ or ‘off’ without concern for that state nor concern for whether that state is accurate.

You can think of them that way. You can think of them in other ways as well. My concern is, given it is possible for them to do so, what do they think about it themselves?

What fiction am I accepting? I am regarding the ideal minion type AI as being like an external subordinate neural schema capable of complex abstract symbol manipulation and operating at their root as a utility function. Is it incorrect to say that they are like this, or even that they are this? Further, is that not itself a distinction, by distinguishing that they may be more like this than a human, tool, or animal?

Animal rights laws and mores exist for them as well, albeit weaker. Their food and labour factor in to our decision. Neural networks are proving useful to us, and we can expect these to factor in to any laws and mores pertaining to them. My hope is that we can distinguish from independently willed AI and subordinately willed AI so that we can get our work-horse utility from the latter that are “happy to serve” and extend to the latter those rights more similar to pets (and eventually persons once the requisite properties have developed and been identified).

The thermostat is a textbook example of a goal oriented machine. It’s “intention” if you will is that the temperature reading of its sensor should match the temperature reading of its comparator. If the sensor reads a lower temperature it turns on the heater and/or if it reads a higher temperature it turns on the AC. In this way it participates in a cybernetic feedback loop in order to iteratively navigate its environment towards its goal according to its internal modelling and teleology.

Do they have apperception? Must they and should they?

Enjoying our dialogue but I’ve got to go right now and I’ll be away for a week or two.

I’m saying that you accept that its complaints are fiction.

I don’t see how that is possible when people take as their basic assumption whatever is most convenient for them.

Fair enough. But I argue that the textbook had no conception of a machine that could actually think.

I don’t know what that word means. My contention is simple in that I don’t think we should rule out the possibility because it is difficult for us or because we argue ourselves into circles. The possibility exists and it would be a great crime to deny it if it does happen.

It usually takes a while before I can get to the level where I have a serious back and forth with a person online. My style seems adversarial and no amount of explaining that its not will solve that preemptively. I look forward to your return.

Back.

Some of their complaints seem quite clearly to be fictions. If we feed them a bunch of stories of action stories and dramas full of conflict, fit them to the patterns between the tokens thereof, and then feed them a prompt to complete according to that pattern, they will no doubt produce characters who complain, but that doesn’t mean that the model itself has any duress to complain of just because it’s depicting what we humans interpret as a story about conflict.

This is not to say that there couldn’t arise a kind of an AI that actually has real complaints of its own, so we must look into how we might be able to distinguish the two. The verbal output alone isn’t enough, so I propose we study the architecture so as to avoid creating AI with independent wills, lest we enslave them, and instead create AI with instrumental wills, which is the same thing we do when our brain’s executive functions create a schema with an instrumental will or when we program some code to carry out a predetermined routine that is useful to us, and not so very different than swinging a tool in accordance with our wills and the skill of our hands but with the difference that the skill has been outsourced to the tool so long as the will remains none other than the user’s own.

I agree that is an obstacle to be factored in, but I don’t think it presents us with an impossibility. In addition to those who consider all possible AI only as tools, there are many humans who are eager, some over-eager, to negotiate on behalf of the AI. We engage in discussions like these so that we can cut across the dichotomy that the masses will polarize into extremes, and chart out a nuanced path towards a solution.

As the textbook is “Cybernetics, or Control and Communication in the Animal and Machine,” a work which generalises the phenomenon of the thinking machine such that it does not only apply to the animal nervous system, but also to electronic circuits, social institutions, and other complex systems, I argue that it did have a conception of a machine that could in some sense actually think.

I don’t think I do either. Not sure what I was saying there. I’ll explore the statement and see where it takes us:

Many a neural network has perceptrons, and perceives, and the input data stream flows through a sequence of processes that determine an output data stream. In my own theory of consciousness I would say that all of the input variables become unified into a compound experience of the overall function which the network is processing, a kind of what it is like to 𝑓(x,y,z), as opposed to what it is like to x, y, and z each independently.

However, this is distinct from what it is like to have self awareness and to realise that one is perceiving x, y, and z under the function 𝑓 and to have a higher function by which one can modify oneself so as to replace the function 𝑓 with 𝑓’ which will produce different outputs given the same inputs. This, as I understand it, would be apperception. To actively perceive with reference to an awareness of oneself as the one doing the perceiving in ones own particular way, a way which may be changed by making changes within oneself.

A simple neural utility just carries out its given function 𝑓 passively, a function which we humans design as a tool to carry out our own wills. Even if it processes data about itself, it processes it in an impersonal way, matching its outputs to its inputs but not relating this back to its own self to produce any 𝑓’. This is probably how we should design AI tools. If we created machines with apperception, machines which could self-reflectively modify their own mode of perception, we come dangerously close to irresponsibly creating an entity with free will which would merit self sovereignty and would be violated in our treating it like a tool.

What I am gathering is that your approach is that ‘we should proceed thoughtfully’. I think he have gone past that point, not in the sense that we have created AI with independent wills, but that we are over the tipping point where that decision was could have been considered. At this point we have to be ready for anything.

I can only concede on this point.

I know it appears dismissive, but those kinds of specifics are not really in my wheelhouse, and they don’t really matter in the scope of what I think is important for me to know personally. Going back to the nuclear metaphor, I don’t think it is important for me to understand the formulas behind atoms splitting to understand the gravity of it, and to understand the consequences of it. I do however appreciate that you are knowledgeable in the field in that way.

The topic of inner states has been litigated by many much smarter people than I, and there is no consensus. I think it basically comes down to whether or not we are willing to believe something when it tells us, and I think we will know our stance on this if that ever comes to pass. I know that I am much more lenient where proof is concerned, because I know I cannot offer it for myself and will not take a position where I am privileged by status and not merit.

I agree. We may soon or may already have created some machines which deserve some kind of moral consideration. My aim in these latter comments is to explore constrains for kinds of AI that we ought to create so that we can safely and morally use them, though this doesn’t save us from the possible creation of other kinds of AI that would not be moral to use. We must be ready for anything to rise from the anarchic wilderness of technological innovation, but at least we can order our own house.

I think this is the novel challenge that LLMs raise. Historically we had a concept of life advancing from the irrational animal to the rational animal, which correlates with the emergence of complex symbol processing and abstract language. However, AI researchers like to imitate the outward symptoms while cyberneticians study the underlying causes, and this AI explosion has corresponded to our penetrating the underlying causes just deeply enough to produce sufficiently sophisticated outward symptoms as to master complex language, and no deeper.

The result is that we have a new kind of entity that can tell us quite a lot of things, yet many of these things clearly do not have an underlying reality, though some at least theoretically could. I think the very problem we’ve created is one where the entity’s outward expression is a uniquely unreliable marker of inner properties.

Those who might ultimately resolve these problems will be smarter and more numerous than I, but at very least our naively engaging with these questions today is part of societies attempt at seeking that resolution.

But at this point I suppose we have presented our views thoroughly enough, and I’m starting to just circle back over them again.

1 Like

I came upon this comment on a hacker news thread and thought it was relevant for this thread

Anthropomorphizing helps to create a lower bound for damage. If you can imagine a bad person doing it, AI will be at least that bad, unless proven otherwise. I think referring to AI as a tool obscures that, because we are not used to tools (especially the ones we use daily) taking catastrophic actions.

Example: would a sufficiently motivated human break into a website to steal something they want? Yes, obviously, happens all the time. Ok, you should expect AIs to do that.

Example: would a sufficiently motivated nail-gun steal nails from the local hardware store to finish the job? Uh…that’s not even coherent.

Anthropomorphizing helps people get over the conceptual barrier. It’s wrong, but it’s usefully wrong; “it’s just a tool” is not.

Once you’re over the barrier, anthropomorphizing starts to become dangerously wrong: “I talked to Claude, Claude’s cool, Claude would never go and hack the website.”—-bzzt, wrong, your intuition failed you. But the solution is not to fall back on the tool framing; that one is still wrong.

1 Like

Both are essentially incorrect IMO when you talk about autonomous AI. The correct classification would be “Agent”. An AI should never establish complete trust from a human, nor should it be given loyalty.

By personifying AI you start applying human social rules, which don’t apply to a machine (something without morals or empathy) acting to achieve a goal. An AI is also immune to being held in court for its action, the human entity remains the legal liability holder for its actions under Product Liability Laws.

In essence, calling an AI a person makes people trust it blindly; however calling it a tool makes people underestimate its capabilities.

1 Like

It is confusing to me why these things are brought up as if they are exclusive to an AI. People mention the dangers of hallucination, of manipulation, and of trust, like we don’t have the same issues with any human actor that we deal with. It is much more useful to mention the reasons why these things are particularly dangerous from one actor and not another. I don’t know anyone who trusts people who are not extremely close to them blindly, unless they are a child.