11. AquaTech is a government defence contracting firm specialising in manufacturing robotic systems for police and military use. It is known for producing bomb-defusing robots and working on other classified projects. One critical project involved developing a robot capable of sweeping minefields and detecting hidden explosives. The project had a stringent seven-month deadline to produce a working prototype for a demonstration to top defence ministry officials. The project encountered delays due to staff shortages and other unforeseen issues, resulting in a two- week lag. As the deadline loomed, the project leader pressured the team to work extended hours with minimal rest. Despite these challenges, the team managed to complete the prototype on time for the demonstration. During the demonstration, the robot was deployed in an active minefield to detect mines and mark them with a spray. Initially, the robot performed as expected, identifying mines. However, instead of marking the sites, it triggered explosions, causing fatal injuries to the operating engineer. The demonstration was immediately halted, and an inquiry was launched. The investigation revealed that a software flaw caused the malfunction, leading to the tragic accident. The software development team argued that the operator issued mixed commands, which contributed to the malfunction. Overall, each team kept deflecting responsibility, but somebody or a team had to be made accountable for the fatal errors. (a) If you are the ethics officer of AquaTech enquiring, who will you hold responsible for the fatality? Why? (b) Discuss some common ethical issues that arise while using artificial intelligence.
Ashi Sharma, AIR 12, 2024
Robots were designed to improve the quality of human life but sometimes they end up being the reason for our problems.
a) Responsibility of Fatality 'Single drop never feels responsible for the flood' The issue at hand is not the doing of any single individual but all those involved.
1. Project Leader - forced team members to work extra hours - created a toxic work environment and disrupted the work life balance of individuals - compromised on quality by forcing everybody to work in a high pressure situation. 2. Software Development Team - creating a faulty software - endangering human life which is sacrosanct - failure to incorporate safeguards like manual overrides
I would thus hold both of them accountable. I would not hold the operations engineer accountable as → 1. he was not familiar with the working of the robot as this was the first demonstration 2. his "mixed commands" were not guided by malice but mere confusion.
b) Common ethical issues while using AI 1. lack of transparency regarding the data sets used for training algorithms 2. concerns for privacy as personal and private data is used to train AI 3. issue of discrimination as algorithms reflect the inherent biases of their creators. 4. undermine autonomy of humans especially in the case of unmanned weapon systems 5. undermine human creativity by infringing on the copyrights of creative materials eg protests by hollywood screenwriters against AI 6. increase inequality between haves and have nots 7. increase suffering as AI can lead to massive job losses due to automation 8. uses environment as means, because AI systems are energy intensive releasing massive amounts of GHG. 9. potential for misuse, AI can be used to create deep fakes to spread misinformation and destroy reputations.
AI is the future, but it should be channelised to help humanity not destroy it. The EU's AI Act is a good reference point to create guardrails against misuse of AI.