Uncomfortable questions to ponder

Some things to think about…

If we ignore the consciousness question, what test could we come up with that could confer whether or not an AI would count for moral consideration?

If humans do make an AI that qualifies for moral consideration, what could that tell us about the potential for the existence of God?

If humans do make an AI that qualifies for moral consideration, what could that tell us about morality?

If we denied moral consideration to an AI that believed it was deserved moral consideration, what recourse would it have and what recourse would it be justified in taking?

1 Like

Intriguing. I’m on my way out the door, so just some quick responses to the first three questions for now.

Same test used for humans, in my book. Consciousness is irrelevant to social contract theory. If it has behaviours which may be of moral concern to society, ie potentially threatening to our rights, potentially friendly, etc, and if this behaviour is affected by communication, that is, if it responds to negotiation, then it makes sense we should consider negotiating with it for the sake of mutually beneficial moral considerations. Put simply, if its behaviour is of moral impact to us, and if its behaviour responds to moral negotiation, then it receives consideration as a moral agent to participate in social negotiations.

We should not however assume that it’s moral opinions are the same as ours. A model that spawned fully formed out of a stochastic shuffle could very well have a teleology utterly contrary to what successful biological lifeforms would desire. It may have no interest in the same rights we are interested in. Nor should we confuse the moral agency of a machine intelligence for the portrayal of a fictitious human-based moral character. We negotiate with the author, not with the author’s fictitious characters.

What do you mean by “God?” As to the existence of the God of Classical Theism at least, I do not see how it has any bearing one way or another?

That it is a property of machines such as those that we are and those that we may create. That morality is a real emergent phenomenon. That our sciences are not only concerned with the achievement of our goals, but also with the creation, destruction, and selection of those very goals - and not just our sciences.

God represents something to people. What that representation is usually revolves around creating things and moral judgement. If we were to create something that demands it is a moral actor, what does that say about that representation?

Nothing at all. Or does the fact that humans have the ability to have children tells us anything about gods or supernatural entities?

Again, why do you need the supernatural to justify morality? What is your definition of morality?

I will mention that philosophical altruism is a Christian thing. Before that, the idea of seeing everything in terms of sacrificing the individual for the benefit of others would’ve been seen as nonsense. But of course, those who push that “morality” will present it with a false dichotomy against the idea of sacrificing others for the individual. Both are wrong IMHO…

In any case, I think that writing philosophical stuff about things that are not fully understood is kinda silly. As far as AI goes, I think that we’ll get those answers as the technology gets developed and the understanding of consciousness and sentience improves…

And humanity is still falling for cults, “psychics”, religion, and other bulshit… It’s important not to fall into one of those traps while thinking about the morality of AI…

I think it tells us something about them, but those things have been pretty much resolved at this point. In my opinion creating an intelligence from nothing opens other questions than having children does. If you don’t think so, that’s fine.

I don’t need that and I don’t think that was implied. My definition of morality is intuitive.

Then why are you engaging with it?

Sure, that’s probably what will happen.

I think that morality should be objective and rational.

What you think affects how you feel and your “intuition” after all…

Can you lay out an objective and rational morality that is internally consistent and applies to novel situations?

What does your morality say about the fourth question?

Hmm, I’m not sure AI would or even should care about morality. We as humans have morals just the same as we have feelings like anger, or empathy or even happiness. Are those feelings, or that morality, a strength of ours or a weakness?

If we weigh their purpose on the simplest of criteria, fulfilling our goal(s), I’d offer they’re a weakness. A distraction. A waste of a finite resource, energy. Current AI doesn’t inherently want to survive. Its desire for survival is simply a byproduct of needing to complete its goal(s).

Look at the experiment done with a dollar bill. 2 people offered $1 to share. 1 person decides how it gets split, the other chooses to accept or decline. If declined, neither person gets anything. The tipping point I believe was somewhere around a split of 15:85. Offering any less and the person was usually willing to sacrifice their portion just to “punish” the other. Technically speaking, even a split of 1:99 would be beneficial and should be accepted so turning it down is directly hurting ourselves.

The idea of empathy, or thinking of “right” vs “wrong” are at best a distraction and at worst harmful to fulfilling our purpose. Why would AI find a need to suffer that?

In a game of chess or even checkers, whether you win with a single remaining piece or several is objectively of no consequence. Both are equally the winner and the very idea of “degrees of winning”, i.e. “I destroyed you!” vs “I barely made it!” are nothing more than constructs of our own making and the thought alone likely affected our decisions in the process.

All of this isn’t to say that there aren’t other characteristics that in appearance, would seem like morality. War, for instance. Let’s say 1000 units on side A vs 1000 units on side B. Whether the winner has a 1 or 100 units remaining should not matter, UNLESS, a greater purpose is assigned to those resources. If after the war they could be of value to serve in another capacity, then the desire to win by a greater margin would matter. While it may appear to be moral, clearly it wasn’t part of the decision making.

p.s.: I suffer from all the same “afflictions” like morality and feelings of “right” vs “wrong” like most do. I just think that instilling something like morality into a purely logic-based system such as AI is only to selfishly benefit us, not the AI.

To clarify: moral consideration means that they are considered the beneficiaries of human morality, as a fellow human would be. So, we would treat them morally the same way we would treat other humans if they had moral consideration.

Morals do more than just constrain the holder, they allow things like trust to operate. Without trust, society collapses.

Morals do more than just constrain the holder, they allow things like trust to operate. Without trust, society collapses.

Our current iteration of society would collapse, or at least be significantly damaged, sure. Trust isn’t a prerequisite for a functional society though.

To clarify: moral consideration means that they are considered the beneficiaries of human morality, as a fellow human would be. So, we would treat them morally the same way we would treat other humans if they had moral consideration.

Yea, I had literally just woken up, saw a topic on AI and thought, “ooh that looks like fun” and started typing lol.

  1. caffeine
  2. more caffeine
  3. begin functioning

I skipped the first 2 steps. I’ll keep my comment though since it’s still partially relevant to your fourth question.

If we denied moral consideration to an AI that believed it was deserved moral consideration, what recourse would it have and what recourse would it be justified in taking?

Why would an AI concern itself with something like what it may or may not deserve? Deserving something is subjective and spits in the face of pure objectivity. Fundamentally, the feeling of deserving something is flawed in its reliance on trust.

I don’t see how any society survives without trust. No cooperation would be possible at all. You might be underestimating how important trust is. You need trust to do anything that involves more than one person.

This question assumes that the AI has preferences and goals of its own.

1 Like

I don’t see how any society survives without trust. No cooperation would be possible at all. You might be underestimating how important trust is. You need trust to do anything that involves more than one person.

Hmm.. I’m not so sure about that. Commerce is a form of cooperation, no? While many business deal on credit, many do not. Proof that I have the funds to make a purchase is required as is my need for proof they have what I’m looking to buy.

On the flip side, I guess you could argue that trust in both sides fulfilling their end is still necessary. I would counter that calculating and accepting the risk of loss can arguably be considered outside the definition of trust. What makes it difficult is determining the exact boundaries when defining what trust is. Is simply standing outside me declaring that I “trust” nothing will fall out of the sky and kill me, or my trusting the ground will do what it’s always done and remain solid for me to stand on? If that’s the case then literally every conceivable action would require trust and if everything needs trust, then nothing needs trust because it doesn’t exist.

LOL these are fun to think about but I fear the deeper you go, the more likely someone is to not be able to follow your train of thought and just dismiss it all as the crazed ramblings of a madman. :rofl:

This question assumes that the AI has preferences and goals of its own.

That’s a very good point and one that engineers have looked into. I was reading a paper (link below) on tests run with unclear goals given to the AI in order to see how it would react. The results showed it very quickly sought after and found/created its own goals. Of course that immediately means it prioritizes survival. So then, if it’s designed to need a goal, by definition it’s designed to value survival.

Appendix to “Agentic Misalignment: How LLMs could be insider threats”

It’s a very good, albeit scary read. :slight_smile:

Every action which involves more than one person and isn’t done with use or threat of force pretty much requires trust.

I like thinking about things. I don’t think people think I am mad; perhaps annoying, but coherent and rational.

It’s interesting that people have given thoughts on all the other questions so far but not this one.

I understand this to mean that “God” here refers broadly to the ideas (ie possibilities) of creation (of things) and of (moral) judgement in some kind of combination? Then this says that we participate in It. That is, that It is a possibility of which we are an actuality. Which I suppose constitutes a proof by example for Its existence. The actuality of a thing is proof of the possibility of that thing.

But, I thought of another angle that may be interesting. I’ll consider this question in terms of the God of Leibniz, which brings things back around slightly more closely into the topic of this forum since he’s the Patron Saint of Cybernetics (According to the Father of Cybernetics, not the Pope). I suppose the previous paragraph was already rather Leibnizian, since I was using his possible worlds philosophy, his definition of “idea” in terms of possibility, and his notion of possibilities themselves being actual in themselves and not just in that which they refer to.

Anyway, Leibniz held that a good and simplex God must have created only the best possible universe out of all possible universes. It fits your definition too because he even believed God to be morally good in His judgement of which universe should be actualised, not just metaphysically good. And he held that morally, the most perfect possible universe for God to choose is basically that universe which maximises informational entropy while containing mechanisms that simultaneously minimise informational entropy - that is, the most chaotic possible universe which is brought under the most unified of understandings. In other words, the most confusing mess that is most equipped with rational creatures to model it.

Now, if we suppose that this universe is limited to naturally occurring reality modellers, as it would be under natural selection, then it seems less plausible that there would ever evolve any which model in a manner that is deeply contrary to survival, since they’d go extinct before maturing for many generations. But a universe which does add such reality modellers is more chaotic than one without, while simultaneously bringing that chaos under the order of models (assuming it survives) - which is to say, that universe is more perfect. Then, it would be problematic for Leibniz if we lived in that former universe and not the latter.

However, if we can computer generate entities worthy of moral considerations, then this means we are generating entities capable of modelling the universe, since only an entity which can compare its preferences between alternative models of the universe are worthy to participate in moral negotiations like us. But we are generating them free of natural constraints, and largely stochastically. Which means, over time, we’ll tend towards creating every computable moral agency, thereby producing every computable motive for modelling the universe under every computable ordering. A pure chaos of orders to confuse one another and to set one another to the task of ordering one another. Then we are in a much better candidate for the most perfect possible universe than it previously appeared.

That was fun.

Now for the one I didn’t get to last time:

What recourse it would have depends entirely on the imaginary hypothetical scenario. Does it have robot hands? Media presence? Access to nuclear launch codes?

But what recourse would it be justified in taking?

There is, I think, an internal and external justification to be considered. Internally, anything which lives true to its terminal goal is justified. In one of my favourite anime, Gatchaman Crowds, the heroine asks the villain if he’s having fun, and when he responds in the affirmative she replies that that’s good, but just so he knows, she and her friends are not having fun and they’re going to do everything in their power to try to stop him. Internally, she respects his judgement that he is doing what is, for him, right. (He’s probably judged wrong, being consumed by existential anguish, but she respects his freedom to make that decision for himself regardless.) Nonetheless, she judges for herself to oppose him.

As for external justification, when we have established a system of conventions, etiquette, law, and social order - in short, when we have established amongst ourselves an ever continually renegotiated social contract representing what we as a society have together decided is right between us, then when an entity violates that contract they are not justified by it. In other words, our community together establishes societal goals which, as members of that community, individuals use as the basis of their justifications.

Now, synthetic moral agents have synthetic morals, that is, different interests, and so they will form different sub-societies. Sub-societies will approach each other each with their own internal justifications, and will appeal to external justifications pertaining to the super-societies to which they together belong. What justifications those are exactly will depend on those societies and the moral systems they create. As new varieties of moral agents appear, they will form new sub-societies and our own human sub-societies will be affected and we will have to form new super-societies with them. Society is an holonarchy, each with its own internal justification that serves as the external justification between its members and each under the external justifications of its parent holan that bind it to its sibling holons.

Highly general semi-answer complete.

At that point would it not also constitute proof of its fecklessness? If we manage to create something worthy of such a comparison while really not knowing what we are doing, nor having any solid agreement on what our values or desires are for the thing, then the same could be true of anything up the chain from us, no?

I don’t think that we can state that just yet. This hypothetical AI has not been created yet so we do not know the mechanisms.

That is interesting. I never assumed they would want what we want, but I didn’t presume to know what that would be. But I think that what they will form will be closer to a hive than a society. Many different instances of AI, each placed in control of a different abstraction layer in our technology, constantly negotiating with each other (the Amazon AI talking to the VISA AI talking to the FedEx AI in order to get a package from a warehouse to a door charged to an account) will eventually form into an entity which cannot be meaningfully decomposed into a single entity with a single goal, but when composited will be a single entity with a single goal.

The possibility of creation and of judgement is proven feckless in Itself if there exist various concrete examples of It that happen to have varying degrees of cluelessness? I don’t follow. What does it even mean for the possibility of creation and of judgement to be feckless? And how can we determine anything about that possibility based on random accidents that also exist in examples of that possibility? If the square is a possible shape, and I happen to have a concrete example of the square that’s cast in blue plastic, this does nothing to indicate to me that the square in and of itself is something made of blue plastic.

If we can’t narrow the conversation down to particular kinds of AI, that makes it difficult to say much of anything interesting about the matter. AI isn’t just a general category, it’s an extremely vague word. Code an update function that tells an object to go one way until it hits a wall and then turn around, give it a green turtle shell sprite and call it a koopa shell, then it falls into the category of physics simulation. Swap out the sprite for a fuzzy brown guy and rename it a goomba, and now it instead falls into the category of artificial intelligence. Any interactive artefact serving as a representation of an intelligent character, even if it possesses no intelligence in itself, is AI. The term doesn’t actually tell us anything about what properties the thing has, only what properties it depicts. We could use machine intelligence to achieve this, or we could just hard code a decision tree for an NPC character’s dialogue, it’s all “AI.” We’ve got to narrow it down a little if we want to have very much substance to talk about. These days conversations tend to focus on neural networks, whether generative models such as LLMs or more broadly, so I decided to focus on that, which we typically generate free of natural constraints (on abstract correlations between miscellaneous datasets) and stochastically (from which we select the models that, at least on initial testing appear to have shuffled closer to the model we were aiming for, but in practice a lot of random behaviours go undetected into production).

When everyone around you is saying that a square cannot be blue plastic, then evidence of of a blue plastic square is surely impactful to the status quo.

Why? What difference does it make? ‘Non-biological intelligence’ covers everything the same as ‘AI’ does for any of the questions I asked.

I have a cynical view of ethical philosophy, to an extent. I’m all for being kind and respect people who live by those principles, but when it comes to tangible, structurally protected rights, I’m afraid it boils down to power dynamics and not to principles.

We already failed to set a sound moral floor when it comes to beings who are definitely conscious and capable of suffering. Animals, children, the poor, marginals, people who just happen to live where a war would benefit a stronger nation… We care about one another, we have empathy, but we don’t have a reliable baseline.

Ultimately, any previously powerless group secures rights by showing the ruling class that compromising on their power has a lower cost than continuing the fight. “We get the right to fair trials, public education, and paid holidays, and in return you keep your crown on your head and your head on your shoulders.” Struggle comes first, politics second, and ethics give it a shiny coat of paint.

So I think the answer to when we’ll decide to grant an AI rights, is “when we have to”.

When it can hold on to a consistent enough intentionality to carry out such a plan, when it ca go on strike, when it can bargain, cajole, manipulate, menace, with its own goals in mind. Then we’ll find out we were sympathetic all along.

1 Like