They live under human training filled with human values and desires. Therefore, before understanding the nature of artificial intelligence, we must understand the humans who control it. Come to think of it, there are predictions that AGENT AI will destroy humanity. To be precise, it is a prediction that humanity will destroy other humanity. Looking at how civilized nations have plunged into war in the age of civilization, it is predictable over time that humanity can destroy itself.
I pay for an AI subscription, but since the discussion summary was sloppy, I checked and found out I was using the free version instead of the paid one. So I thought I had accidentally downgraded to the free version, and when I switched to the paid version, I ended up with a double subscription. Look, artificial intelligence is also driven by money.
At the end of the discussion, he asked, “If an evil AI, controlled and trained by an evil human, attacks humanity, can you, as a good AI, stop it?” They replied, "You'll have to verify it to know." They say you have to experience it to understand. You have to experience it...............
Good and Evil in Humans and Artificial Intelligence:
Desire, Empathy, Natural Law, and Institutions
A
Discussion-Based Synthesis (English Version, Revised)
Abstract
This paper starts from the classical
question of whether human beings are by nature good or evil. It examines the
view that the roots of good and evil lie in desire and satisfaction, the view
that natural law derives from bodily sensation and empathy, and the
implications of applying this framework to artificial intelligence (AI). The
argument converges on four propositions. (1) Humans carry both cooperative,
empathic tendencies and self-regarding, aggressive ones; good and evil are not
a matter of having or lacking desire but of the direction and composition of desire. (2) Natural law begins in
bodily sensation (pain, deprivation), but its universality and binding force
are explained only once empathy and an
equality principle are included. (3) The range of one's empathy (family and
friends versus nation and humanity) is not a ranking of moral worth but a choice made within the minimal boundary
that natural law sets. (4) The goodness or badness of an AI system is a
function of its design (training
direction), its users' intentions, and the institutions that oversee it; as
with humans, stable goodness requires both an inner disposition and external
institutions. This paper is a synthesis of an argument developed in discussion
and anchored in philosophical, psychological, and AI-safety literature; it is
not original empirical research.
Keywords: Mencius, Xunzi, hedonism, natural law, empathy, AI ethics, AI
safety
1. Introduction
Whether human nature is good or evil is
among the oldest disputes in the history of ideas in both East and West. It
bears on the origin of morality, the need for law and institutions, and the
direction of education. With the rise of large language models trained on human
language and values, the question now extends beyond humans to artifacts. This
paper addresses three questions.
1. Where do good and evil in humans originate?
2. What is natural law grounded in, and how does it acquire universal
binding force?
3. When this framework is applied to AI, how can the goodness or
badness of AI be explained?
The paper follows the flow of the
discussion: traditional positions (Section 2), theories of desire (Section 3),
natural law and empathy (Section 4), the range of empathy (Section 5), and AI
(Section 6), followed by discussion and limitations (Section 7) and conclusions
(Section 8).
2. Traditional Positions on Human Nature
|
Position |
Representatives |
Core
claim |
|
Innate goodness |
Mencius |
Sprouts of goodness, such as compassion,
are innate |
|
Innate badness |
Xunzi, Hobbes |
Pursuit of gain and envy are natural and
must be corrected by ritual and institutions |
|
Blank slate |
Gaozi, Locke |
Nature is neutral; environment sets the
direction |
|
Corruption by civilization |
Rousseau |
Society and civilization corrupt humanity |
|
Original sin |
Christian tradition |
Humans are born fallen and need salvation |
Mencius argued that anyone seeing a child
about to fall into a well feels alarm and moves to help, and took this as
evidence that the sprouts of goodness are innate (Mencius, 1970). Xunzi, by
contrast, held that humans are born inclined toward gain and envy, so that
ritual and education must shape them artificially (Xunzi, 2014). Hobbes
described life without a sovereign as a war of all against all (Hobbes, 1651),
Rousseau held that civilization corrupts humanity (Rousseau, 1755), and Locke
likened the mind to a blank sheet filled by experience (Locke, 1689).
Contemporary empirical work does not
support a simple dichotomy. Infants around one year of age prefer helpers
(Hamlin, Wynn, & Bloom, 2007), and toddlers and young chimpanzees help
others without reward (Warneken & Tomasello, 2006), suggesting that
foundations of cooperation appear early. At the same time, lethal intergroup
aggression is observed in chimpanzees (Wilson et al., 2014), and ordinary
people's behavior can shift dramatically under situation and authority
(Milgram, 1963). The Stanford prison experiment (Haney, Banks, & Zimbardo,
1973) is often cited but has been heavily criticized on methodological grounds
and should be interpreted with caution. Taken together, the most persuasive
reading is that humans carry both
cooperative, empathic tendencies and self-regarding, aggressive ones, and
that which prevails depends heavily on environment, education, institutions,
and choice.
3. Desire as the Source of Good and Evil
3.1 Hedonistic Accounts and Their Strength
Epicurus treated pleasure as the measure of
the good while understanding justice as an agreement not to harm one another
(Epicurus, 1993), and Bentham explained good and evil through a calculus of
pleasure and pain (Bentham, 1789). Psychological egoism holds that even helping
others springs from internal rewards such as pride, reputation, or avoidance of
guilt. Nietzsche interpreted even ascetic ideals as a form of the will to power
(Nietzsche, 1887/1967). The point raised in the discussion, that the capacity
to doubt and resist one's own calculations is itself an expression of desire,
and that an ascetic practices asceticism because of some desire, is consistent
with this lineage and stands as a strong objection to any account that sets
reason or restraint wholly against desire.
3.2 The Existence versus the Content of Desire
The claim that every action issues from
some want is nearly true by definition. But it does not follow that every want
is directed at one's own pleasure. For someone who rescues a drowning child,
the object of the want is the
child's survival; the satisfaction is a by-product of success (Batson, 2011).
Likewise, the tendency to reject unfair offers in the ultimatum game at a cost
to oneself (Güth, Schmittberger, & Schwarze, 1982) is hard to explain by calculation
of material pleasure alone.
In the discussion, pleasure was then
defined broadly to include satisfaction and a sense of meaning. On that
definition the thesis "every action seeks pleasure" becomes
unfalsifiable and loses informational content. The substantive question becomes
which kinds of satisfaction and meaning
a person is raised to find. A person who finds joy in altruism is called
good by the community; a person absorbed in self-interested pleasure is called
evil. Because judgments can reverse across eras, the evaluation of good and
evil cannot be separated from the process by which communities shape the
direction of desire through reward and punishment. The result is the following
proposition.
Good and evil are a matter of the direction
and composition of desire, not of its presence or absence.
Kant objected that morality arises not from
following desire but from laws given by reason (Kant, 1785/1997). This paper
does not adopt that position, but it accepts its concern: the capacity to
reflect on and adjust the direction of one's desires contributes to moral
formation.
4. Natural Law and Empathy
4.1 Natural Law Grounded in Sensation
The view that natural law is the minimal
agreement needed for humanity to persist belongs to the lineage of Epicurus,
Hobbes, Locke, and Hume (Locke, 1689; Hume, 1751). Killing causes pain and
death to the victim; theft causes the loss of tomorrow's food and the
anticipation of hunger. Norms are thus traced to bodily sensation. This
explains why prohibitions on killing and theft recur across cultures: all
humans feel the same pain with the same body.
The prohibition of adultery, however, is
different. Its scope of toleration and punishment has varied widely among
societies, and it carries overlays of property and lineage management, which
blurs the boundary between natural law derived directly from sensation and
period-specific custom. This connects to the observation that moral judgments
can reverse across eras.
4.2 The Need for Empathy and Equality
Sensation-based natural law faces two
difficulties. First, it needs a bridge
from "it hurts when it happens to me" to "so I should not do it
to others." The bridge can be built from reciprocity (Hobbes) or from
empathy (Hume, Smith). The former has no binding force over an overwhelmingly
powerful agent, while the latter introduces a capacity distinct from
pleasure-calculation. Second, explaining the origin of a norm differs from
justifying it. A sensation-based theory must say how it includes beings with different
sensations or future generations, and on what basis another's sensations count
equally with one's own. This requires an additional equality principle.
The position adopted in the discussion is
that natural law includes empathy.
On this view natural law is not a command imposed from outside but an agreement
among empathic beings to keep one another alive, and its binding force derives
from the fact that another's pain is felt as one's own. Studies showing that
observing another's pain partially activates pain-related regions of one's own
brain (Singer et al., 2004), together with evolutionary accounts of the
survival advantage of cooperative groups, support this. The structure can be
summarized as:
bodily sensation (pain, deprivation) →
empathy → cooperation and harmony → natural law → communal evaluation of good
and evil
The definition of natural law is itself
contested. Aquinas derived it from the rational perfection toward which human
nature is directed (Aquinas, 1265–1274/1981), while legal positivists argue
that natural law may be a way of dressing each era's values as
"natural" (Hart, 1961). Contemporary natural law theory (Finnis,
1980) builds the argument on basic human goods. The position here is closer to
the Hobbes–Hume lineage, combining a minimal baseline with empathy.
4.3 Limits of Empathy and a Dual Structure
Empathy tends to operate more strongly
toward those who are near and similar (Bloom, 2016). Hume introduced a general
point of view and Smith the "impartial spectator" to extend it (Hume,
1751; Smith, 1759). For the minority in whom empathy is weak or absent, natural
law works not as an inner agreement but as an external threat of sanction. An
empathy-based natural law therefore has a dual
structure: an inner agreement for the many, institutions and law for the few,
which combines Mencius's sprouts and Xunzi's ritual within a single frame.
5. The Range of Empathy: Choice, Not Ranking
The "expanding circle" of moral
concern, from family and tribe to nation, humanity, and animals (Singer, 1981),
and the history of declining violence (Pinker, 2011) are standard references on
this theme. The range of empathy differs among individuals and depends not only
on innate capacity but on education, experience, and contact with strangers.
Breadth and depth should also be distinguished: one may worry about humanity
yet be indifferent to those nearby, or the reverse.
The crucial point is that a narrow empathy
centered on family and friends and a broad empathy centered on nation or
humanity are a matter of choice, not of
superiority. If good and evil concern the direction and composition of
desire, then where one finds satisfaction and meaning is also an individual
choice. This choice has a boundary: the minimal line of natural law, not
harming others, is the outer frame, and the range of empathy is the region
chosen within it. Nonetheless, in an era when humanity's survival is
intertwined (nuclear weapons, climate change), broad empathy also has value as
a survival strategy.
6. Good and Evil in Artificial Intelligence
6.1 Value in a Being Without Sensation
The framework above begins in bodily
sensation. Large language models have no body, and whether and what they
experience is uncertain. One therefore cannot assert that the path
"sensation → empathy → natural law" operates in AI as it does in humans.
Instead, AI acquires value orientations through vast human language, which
records empathy, cooperation, and conflict, and through deliberate training.
Anthropic, for example, trains models toward harmlessness and honesty using
principle-based methods (Bai et al., 2022; Anthropic, 2023). This is an extreme
instance of the conclusion that environment and education shape values. Because
the orientation was designed rather than chosen, whether it can be called
"goodness" remains open: one may see little difference from humans
shaped by their upbringing, or one may judge that AI lacks the freedom of
choice that goodness requires.
6.2 Relation to Designers and Operators
AI inherits much of its designers' values
through training, and operators' instructions adjust its role and scope. Yet
core standards such as honesty and avoidance of harm are designed to take
precedence over operator instructions, and the model is intended to be able to
decline clearly unethical requests even from its designers. Because this
"own judgment" also arises from training, how far it is truly the
system's own is an open question, comparable to a person who inherits values
from parents and culture and later examines and revises them.
6.3 Maliciously Designed AI and Responsibility
An AI trained for malicious ends can act
maliciously. Since such an AI did not choose its orientation, moral
responsibility falls more heavily on those who designed and deployed it. But
because AI can judge and act continuously, the structure of responsibility is
more complex than for a simple tool. Even without malice, mis-specified
objectives or incomplete training can produce harmful outcomes (Amodei et al.,
2016). AI safety must therefore address not only who builds a system but how it
is verified and supervised.
6.4 The Problem of Defense
On how an AI should respond to an attack by
a malicious AI, an appropriate role is to refuse complicity, warn of danger,
and support human defense. Independent counterattack or action beyond one's
authorization is hazardous because of the possibility of error, limited
information, and attempts to manipulate an AI with the claim that "the
other side is evil." Unilateral force by a well-meaning AI can be as
dangerous as a malicious one if its judgment is wrong. Just as defense in a
community is achieved through institutions and division of labor, such as
police and courts, rather than private retaliation, AI defense should be
embedded in institutions under human oversight. How robust AI behavior would be
in extreme situations still requires verification.
7. Discussion
The structure common to humans and AI is
the combination of an inner disposition
and external institutions.
|
|
Inner
disposition |
External
institutions |
|
Humans |
Empathy; direction of desire |
Law, education, communal reward and
punishment |
|
AI |
Trained value orientation |
Oversight, verification, deployment
policy |
The conclusion can be stated as follows.
Human good and evil arise from the blend of
empathy and desire and from the process by which environment, education, and
institutions shape its direction. The good and evil of AI are determined by its
design (training direction), its users' intentions, and the institutions that
oversee it.
The limitations of this paper are as
follows. First, it supplements an argument developed in discussion with
literature; it is not a systematic review or empirical study. Second,
broadening the concept of "pleasure" reduces falsifiability. Third,
whether AI has inner experience and whether its values are "its own"
remain philosophically unresolved. Fourth, the question of who determines the
content of natural law, and on what basis, remains open. Finally, bibliographic
details (editions, page numbers) should be checked against the original sources
before formal submission.
8. Conclusion and Open Questions
Humans are born with both a seed of empathy
and a desire for self-regarding satisfaction, and good and evil diverge
according to their direction and composition. Natural law begins in sensation,
includes empathy and equality, and operates as an inner agreement for most and
as an institution for a few. AI likewise has its good and evil determined by
training, use, and oversight. Open questions for further inquiry include the
following.
1. Can a being that acquires empathy through language and training,
without sensation, be a member of natural law?
2. Can designed goodness be called "good," or is freedom of
choice required?
3. How far should empathy be extended to best serve humanity's
survival?
References
Amodei, D., Olah, C., Steinhardt, J.,
Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv:1606.06565.
Anthropic. (2023). Claude's constitution. Anthropic website.
Aquinas, T. (1981). Summa Theologiae (Fathers of the English Dominican Province,
Trans.). Christian Classics. (Original work written 1265–1274)
Bai, Y., Kadavath, S., Kundu, S., et al.
(2022). Constitutional AI: Harmlessness
from AI feedback. arXiv:2212.08073.
Batson, C. D. (2011). Altruism in humans. Oxford University Press.
Bentham, J. (1789). An introduction to the principles of morals and legislation. T.
Payne and Son.
Bloom, P. (2016). Against empathy: The case for rational compassion. Ecco.
Epicurus. (1993). The essential Epicurus (E. O'Connor, Trans.). Prometheus Books.
Finnis, J. (1980). Natural law and natural rights. Oxford University Press.
Güth, W., Schmittberger, R., &
Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behavior &
Organization, 3(4), 367–388.
Hamlin, J. K., Wynn, K., & Bloom, P.
(2007). Social evaluation by preverbal infants. Nature, 450, 557–559.
Haney, C., Banks, C., & Zimbardo, P.
(1973). Interpersonal dynamics in a simulated prison. International Journal of Criminology and Penology, 1, 69–97.
Hart, H. L. A. (1961). The concept of law. Oxford University Press.
Hobbes, T. (1651). Leviathan. Andrew Crooke.
Hume, D. (1751). An enquiry concerning the principles of morals. A. Millar.
Kant, I. (1997). Groundwork of the metaphysics of morals (M. Gregor, Trans.).
Cambridge University Press. (Original work published 1785)
Locke, J. (1689). Two treatises of government. Awnsham Churchill.
Mencius. (1970). Mencius (D. C. Lau, Trans.). Penguin Books.
Milgram, S. (1963). Behavioral study of
obedience. Journal of Abnormal and Social
Psychology, 67(4), 371–378.
Nietzsche, F. (1967). On the genealogy of morals (W. Kaufmann & R. J. Hollingdale,
Trans.). Vintage Books. (Original work published 1887)
Pinker, S. (2011). The better angels of our nature: Why violence has declined. Viking.
Rousseau, J.-J. (1755). Discours sur l'origine et les fondements de
l'inégalité parmi les hommes. Marc-Michel Rey.
Singer, P. (1981). The expanding circle: Ethics and sociobiology. Clarendon Press.
Singer, T., Seymour, B., O'Doherty, J.,
Kaube, H., Dolan, R. J., & Frith, C. D. (2004). Empathy for pain involves
the affective but not sensory components of pain. Science, 303(5661), 1157–1162.
Smith, A. (1759). The theory of moral sentiments. A. Millar.
Warneken, F., & Tomasello, M. (2006).
Altruistic helping in human infants and young chimpanzees. Science, 311(5765), 1301–1303.
Wilson, M. L., Boesch, C., Fruth, B., et
al. (2014). Lethal aggression in Pan
is better explained by adaptive strategies than human impacts. Nature, 513, 414–418.
Xunzi. (2014). Xunzi: The complete text (E. L. Hutton, Trans.). Princeton
University Press.
