ARTIFICAL INTELLIGENCE
Can a Machine Be Moral? A Q&A with Jean-Francois Bonnefon
By: SARAH HOFFMAN | April 13, 2022
FCAT recently hosted a presentation by psychologist and author Jean-Francois (JF) Bonnefon on his latest book, “The Car That Knew Too Much”. The book discusses a groundbreaking experiment, the Moral Machine, that allowed millions of people from over 200 countries and territories to make choices about life-and-death dilemmas posed by driverless cars. Should they sacrifice passengers for pedestrians? Save children rather than adults? Kill one person so many can live? Following his presentation, FCAT’s Sarah Hoffman caught up with JF to ask a few additional questions about this largest experiment in moral psychology.
  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print

Sarah Hoffman: Why do you think the Moral Machine went viral? Why do you think this topic got the attention of so many people?

Jean-François Bonnefon: I think that for many people, the Moral Machine was the first time they realized how challenging moral dilemmas can be. Even people who knew about dilemmas such as 'is it ok to sacrifice one life to save five lives' and thought they had a response to that, realized that they had a hard time making decisions on the complex scenarios offered by the Moral Machine. And the Moral Machine had millions of scenarios to offer, which ensured that no two people saw the same ones. In retrospect, this played a big role in the virality of the Moral Machine, because it made it Youtube-friendly: everybody could post a different reaction video to the Moral Machine, because they all got different scenarios to react to.

SH: Is there anything you wish you had done differently when designing the Moral Machine?

JF: If I were to do it all over again, I would think harder about including a homeless person as one of the potential crash victims. We did this because we believed that, sadly, people would sacrifice the homeless character more; and we wanted to use this as an example of why you cannot simply follow the preferences of the crowd, given that some of these preferences are unethical. But as the Moral Machine went viral, we lost control about the way it was presented in traditional and social media; and in some occasions the presence of the homeless character was perceived as a sign that it was an actual possibility that self-driving cars could be programmed to crash into the poor. I don't like that the Moral Machine may have indirectly contributed to this belief.

SH: What are your next steps for this research?

JF: The crash scenarios in the Moral Machine were intentionally simplified, so that people would not have to absorb too much information about road safety statistics. This approach was instrumental in the success of the website, but a shortcoming was that the scenarios lacked realism. So we have been developing another version of the platform, which uses realistic scenarios and actual traffic statistics. While this data-collection platform is not as simple or engaging as the Moral Machine, its results will be more directly relevant to policymaking.

SH: Do you think there’s anything we can learn from all this beyond self-driving cars that may apply to other technologies or use cases?

JF: Beyond self-driving cars, the data we collected address one huge question, which is whether people put different values on different lives when you cannot save everyone; and as a corollary, which kind of policy they are more likely to accept if governments need to be explicit about the lives they prioritize in a crisis. We have been through this situation at least twice during the pandemic: when ventilators became scarce, many countries had to explain which patients would be prioritized; and when the first vaccines became available, governments had to explain who would be first in line to get a shot. In cases like these, the data of the Moral Machine or comparable studies can help anticipate how citizens will react to high-stake policies decided under close public scrutiny.

Sarah Hoffman leads AI and Machine Learning (ML) research for FCAT, helping the firm understand trends in these technologies and their potential impact on Fidelity.

 
  • Facebook.
  • Twitter.
  • LinkedIn.
  • Print
1022651.1.0
close
Please enter a valid e-mail address
Please enter a valid e-mail address
Important legal information about the e-mail you will be sending. By using this service, you agree to input your real e-mail address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an e-mail. All information you provide will be used by Fidelity solely for the purpose of sending the e-mail on your behalf.The subject line of the e-mail you send will be "Fidelity.com: "

Your e-mail has been sent.
close

Your e-mail has been sent.

This website is operated by Fidelity Center for Applied Technology (FCAT)® which is part of Fidelity Labs, LLC (“Fidelity Labs”), a Fidelity Investments company. FCAT experiments with and provides innovative products, services, content and tools, as a service to its affiliates and as a subsidiary of FMR LLC. Based on user reaction and input, FCAT is better able to engage in technology research and planning for the Fidelity family of companies. FCATalyst.com is independent of fidelity.com. Unless otherwise indicated, the information and items published on this web site are provided by FCAT and are not intended to provide tax, legal, insurance or investment advice and should not be construed as an offer to sell, a solicitation of an offer to buy, or a recommendation for any security by any Fidelity entity or any third-party. In circumstances where FCAT is making available either a product or service of an affiliate through this site, the affiliated company will be identified. Third party trademarks appearing herein are the property of their respective owners. All other trademarks are the property of FMR LLC.


This is for persons in the U.S. only.


245 Summer St, Boston MA

© 2008-2024 FMR LLC All right reserved | FCATalyst.com


Terms of Use | Privacy | Security | DAT Support