Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction

Engaging Science, Technology, and Society

View Publication Info
 
 
Field Value
 
Title Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction
 
Creator Elish, Madeleine Clare
 
Subject STS; science and technology studies; information studies; sociology; law and technology studies
autonomous vehicles; responsibility; machine learning; human factors; accidents; social perceptions of technology; self-driving cars, robot; human-in-the-loop; human-robot interaction
 
Description As debates about the policy and ethical implications of AI systems grow, it will be increasingly important to accurately locate who is responsible when agency is distributed in a system and control over an action is mediated through time and space. Analyzing several high-profile accidents involving complex and automated socio-technical systems and the media coverage that surrounded them, I introduce the concept of a moral crumple zone to describe how responsibility for an action may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system. Just as the crumple zone in a car is designed to absorb the force of impact in a crash, the human in a highly complex and automated system may become simply a component—accidentally or intentionally—that bears the brunt of the moral and legal responsibilities when the overall system malfunctions. While the crumple zone in a car is meant to protect the human driver, the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator. The concept is both a challenge to and an opportunity for the design and regulation of human-robot systems. At stake in articulating moral crumple zones is not only the misattribution of responsibility but also the ways in which new forms of consumer and worker harm may develop in new complex, automated, or purported autonomous technologies.
 
Publisher The Society for Social Studies of Science
 
Contributor John D. and Katherine T. MacArthur Foundation
 
Date 2019-03-23
 
Type info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
Peer-reviewed Article
Comparative textual analysis
 
Format application/pdf
 
Identifier http://estsjournal.org/index.php/ests/article/view/260
10.17351/ests2019.260
 
Source Engaging Science, Technology, and Society; Vol 5 (2019); 40-60
2413-8053
 
Language eng
 
Relation http://estsjournal.org/index.php/ests/article/view/260/177
 
Coverage United States
20th century
NA
 
Rights Copyright (c) 2019 Madeleine Clare Elish
http://creativecommons.org/licenses/by-nc-nd/4.0
 

Contact Us

The PKP Index is an initiative of the Public Knowledge Project.

For PKP Publishing Services please use the PKP|PS contact form.

For support with PKP software we encourage users to consult our wiki for documentation and search our support forums.

For any other correspondence feel free to contact us using the PKP contact form.

Find Us

Twitter

Copyright © 2015-2018 Simon Fraser University Library