Chairman McGovern, Chairman Smith, and Members of the Commission, thank you for the opportunity to participate in today's hearing. I am speaking on behalf of Human Rights Watch, an international, independent organization that takes no government funding. I have worked on technology and human rights issues for more than 15 years. Human Rights Watch has been a leader in research and advocacy on the issue of autonomous weapons systems for 14 years.
The explosion of headlines warning about AI threats and cybersecurity events is not happening in isolation. It directly echoes United Nations and national debates on autonomous weapons systems and military AI, where Human Rights Watch has actively pressed for international restrictions. What was once a science fiction idea is now a live question of international law and civilian protection.
Autonomous weapons systems, which select and engage targets without human intervention, increasingly incorporate AI and are thus important examples of the use of AI in the military domain. They raise a host of legal, moral, security, and other concerns. In addition to their consequences for the law of armed conflict, the systems implicate international human rights law in at least six ways, beginning with the foundational right to life.
To avoid arbitrarily depriving someone of their right to life, use of force needs to be grounded in domestic law, necessary to achieve a legitimate aim, and applied in a proportionate manner. Lethal force may only be used as a last resort to protect human life. Autonomous weapons systems would face difficulties assessing the necessity and proportionality of force given that they would not have the capacity to interpret complex contexts and would lack distinctly human judgment. In addition, they would be unable to defuse a situation and ensure lethal force was only used as a last resort.
The use of force by autonomous weapons systems in law enforcement situations would present risks to the right to peaceful assembly. Police officers have a duty to protect peaceful protests and use non-violent means before resorting to force. Autonomous weapons systems would be a poor tool to serve this purpose. The use or threat of use of autonomous weapons systems could strike fear among protesters and have a chilling effect on free expression and peaceful assembly, rights protected under the US Constitution as well as under international law.
Autonomous weapons systems would undermine two fundamental human rights principles: human dignity and non-discrimination. Human dignity means that all people have an inherent worth. Autonomous weapons systems would kill without the uniquely human capacity to understand or respect the true value of life and the significance of its loss. Furthermore, they would dehumanize their targets by relying on algorithms that reduce people to data points.
The principle of non-discrimination calls for the protection of every person's human rights, irrespective of race, sex and gender, ability, or other status under the law. Biases of system designers or datasets could influence the design and decision-making of an autonomous weapon system using AI. Once such a system is deployed, the inability to understand how and why the system makes determinations could prevent a human operator from intervening to correct discriminatory errors.
The development and use of autonomous weapons systems that are based on AI technology could violate the right to privacy when they entail mass surveillance. Such data-gathering can be unlawfully arbitrary because it fails the requirements of being necessary for reaching a legitimate aim and proportionate to the end sought.
Finally, the use of autonomous weapons systems could create an accountability gap contrary to the right to remedy. There are obstacles to holding individual operators criminally liable for the unpredictable actions of a machine they cannot understand. There are also legal challenges to finding manufacturers responsible under civil law.
A new legally binding international instrument that ensures meaningful human control over the use of force and prohibits the use of autonomous weapons systems that target people would help address human rights as well as other legal, moral, and security concerns. In August, the Group of Governmental Experts for the Convention on Conventional Weapons adopted proposed elements of an instrument on autonomous weapons systems. Although the text would need strengthening to adequately address the problems of autonomous weapons systems, Congress should urge the US government to join the 76 states that have said the text is a sufficient basis for negotiation of an instrument. It should further urge the US government to join the 130 states that have called for a legally binding instrument on autonomous weapons systems.
Militaries are also adopting AI rapidly for a range of other uses, and many of the same human rights concerns apply. Although these systems fall outside the common definition of autonomous weapons systems because they do not select and engage targets, they may inform decisions on the use of force and target selection, or assess expected harm to civilians during armed conflict. For example, such systems may depend on data collection from mass surveillance incompatible with the right to privacy. Algorithmic bias in military AI systems may lead to discriminatory impacts on people with vulnerable and marginalized identities. The opaque design of these systems presents distinct challenges to transparency, oversight, and accountability that are compounded in military and classified contexts when they are not subject to adequate scrutiny and regulatory safeguards.
Congress should urge the US government to provide immediate transparency on the role of AI in the use of force and not to use any AI system, including generative AI, as the basis for targeting decisions, unless certain conditions are met. Specifically, Congress should take the appropriate action and oversight to ensure that US systems:
- Do not use data gathered in violation of international human rights and other international law;
- Are operated with meaningful human control; and
- Are predictable, reliable, explainable, and traceable.
Thank you. I look forward to your questions.
