Document
What does “meaningful human control” mean in the context of AI and lethal decision-making, and why is it essential for protecting human dignity and civilian life? Follow-up Question: If an autonomous system causes harm, who should ultimately be held accountable—the programmer, the military commander, the government, or the machine itself?
The World Council of Churches (WCC) affirms the vast potential that such a transformative technology as artificial intelligence has for enhancing the wellbeing of humanity and the living planet. From researching and developing new treatments for previously incurable diseases, to finding new ways of averting climate catastrophe and preventing biodiversity loss, to supporting the elderly and the vulnerable, the positive use cases for AI are as wide as the technology is adaptable.
At the same time, it is both urgent and critical that we collectively recognize and mitigate against the risks posed by an emerging technology so powerful that it represents potential civilizational change more than mere technological advance. Unconstrained, the profit motive has rarely proven to be an engine for humanity’s best instincts; more often for its worst. We are therefore deeply concerned by the lack of uniform global regulation of a technology that even many of its pioneers have warned carries potentially existential risks.
Our concern is only increased by some of the uses that have been earliest and most avidly pursued by private sector purveyors and governmental adopters alike – including enhanced public surveillance by authoritarian states, and deployment of AI-driven autonomous weapons systems. Moreover, the profound impacts of AI technology in society are being felt already now in the form of spiralling job losses and reduced job prospects, corrosive effects on mental health and relationships for children and young people, amplified distrust in politics and society due to AI-enhanced mis- and dis-information, accelerated environmental and climate degradation due to AI’s power and water consumption, and the massively increased concentration of wealth in the hands of a minuscule minority of technological and political ultra-elites.
AI could be a major force for good for people throughout the world. However, without careful regulation by responsible governments cooperating at the international level, it seems much more likely to be a profoundly disruptive, destabilizing, and dehumanizing technology, widening rather than reducing existing divisions and injustices. AI can have harmful and dangerous consequences if not kept in proper checks and balances. This is where religious communities can play a very important role.
"Meaningful human control" is a phrase which was first used by campaigners, and later adopted by some Governments, calling for limits to be placed on the development of weapons technologies with AI capabilities. It refers to the ethical imperative that autonomous systems should never be allowed to have ultimate authority over life and death decisions. If respected, the principle would ensure that weapons systems would not operate beyond human oversight, accountability, and moral judgment.
Machines can never be developed to the extent that they could have the uniquely human characteristics of moral reasoning, empathy, and the contextual understanding necessary to make decisions that respect human dignity and the laws of war. Without ensuring meaningful human control over a weapons system, the risk of unintended escalation, civilian casualties, and the erosion of ethical boundaries in conflict situations increases significantly.
The World Council of Churches, as an active member of the Campaign to Stop Killer Robots, has for several years advocated for the ethical governance of AI in weapons systems. In particular, the Campaign calls for a global pre-emptive ban on fully autonomous weapons. This initiative underscores that the lack of human control not only endangers civilian lives but also dehumanizes warfare, reducing it to a calculation of algorithms – a “digital dehumanisation” - rather than a reality with profound moral implications. It is the most concrete expression of Pope Leo’s call for AI to be ‘disarmed’.
We consider that delegating decisions - which could lead to death or injury - to machines undermines the God-given dignity of every human being and contradicts the principles of Just Peace and Christian social ethics. The use of AI in warfare must align with international humanitarian and human rights law, which prioritizes the protection of civilians and protects the right to life. Meaningful human control ensures that human values—such as love, compassion, justice, and the sanctity of life—remain central to decision-making, which is even more critical in times of conflict.
Meaningful human control requires that humans retain three particular roles with respect to the use of weapons with AI systems. The first of these is oversight – Humans must have the ability to intervene and interrupt the deployment of autonomous systems at any stage. The second is understanding – The soldiers or other operators of such weapons must fully comprehend the capabilities, limitations, and potential consequences of the systems they are using. Thirdly, accountability is key. If an autonomous system would cause harm, the question of responsibility becomes complex. Machines cannot be held accountable, they cannot suffer punishment or reflect on their wrongdoing. Accountability must remain with human actors - the manufacturers or programmers of the weapons, the military commanders or operators, or governments who design, deploy, and oversee these systems. We believe that without clear lines of responsibility, the use of autonomous weapons could create a situation where no one is held accountable for violations of international human rights or humanitarian law. There must be clear legal and ethical frameworks that assign human responsibility for the actions of autonomous systems.
AI is here to stay but it raises the question: What kind of world do we want to create. Let us create a world of justice, peace, reconciliation and unity. A world for everyone, every person and all creation, where all are respected and valued. A world not besotted by war, violence and conflict but a world in which both humans and machines can lead to the advancement of life and respect the sovereignty of God. Let us build such a world together!
Rev. Prof. Dr Jerry Pillay
General Secretary
World Council of Churches
