References
This page contains the list of all references that we will using in this course.
Under Construction
This page is still under construction. In particular, nothing here is final while this sign still remains here.
A Request
I know I am biased in favor of references that appear in the computer science literature. If you think I am missing a relevant reference (outside or even within CS), please email it to me.
Other Courses
A list of tech ethics syllabi
Casey Fiesler maintains an list of syllabi of courses that teach about ethics in technology . The list is very comprehensive.
Below we list some courses that are somewhat similar to ours (see the above for a more comprehensive list):
- A Course on Fairness, Accountability and Transparency in Machine Learning , crash course sponsored by the GIAN program of the Government of India.
- Ethical and Policy Dimensions of Information, Technology, and New Media , U. of Colorado.
- Fairness in Machine Learning , U. of Berkeley, 2017.
- Human-centered Machine Learning , U. of Colorado.
- Science of Data Ethics , UPenn..
- When Machines Decide: The Promise and Peril of Living in a Data-Driven Society , U. of Utah.
Technical Books
Below are some books that talk about the technical issues that we will consider in this course:
- Solon Barocas, Moritz Hardt and Arvind Narayanan, Fairness and machine learning . Online textbook (in progress).
- Michael Kearns and Aaron Roth, The Ethical Algorithm . This book is targeted at a non-technical audience though it does go into some of the high level ideas of the underlying algorithms.
Research Papers
Below is a list of research papers that we will refer to in this course roughly categorized by the broad topics we will cover in this course:
Discrimination and Fairness
- Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford and Hanna Wallach, A Reductions Approach to Fair Classification . In ICML 2018. [Full version ]
- Alexandra Chouldechova, Fair prediction with disparate impact: A study of bias in recidivism prediction instruments . In FAT/ML 2016.
- Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold and Rich Zemel, Fairness through awareness . In ITCS 2012. [Full version ]
- Cynthia Dwork and Christina Ilvento, Fairness Under Composition . In ITCS 2019. [More Details ]
- Sorelle A. Friedler, Carlos Scheidegger and Suresh Venkatasubramanian, On the (im)possibility of fairness . 2016.Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton and Derek Roth, A comparative study of fairness-enhancing interventions in machine learning . In FAT* 2019. [Arxiv version ]Michael Kearns, Seth Neel, Aaron Roth and Zhiwei Steven Wu Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness . In ICML 2018 [Arxiv version ]Jon Kleinberg, Sendhil Mullainathan and Manish Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores . In ITCS 2017. [More Details ]Latanya Sweeney, Discrimination in Online Ad Delivery . In CACM, May 2013 . [Full version ]
Bias
- Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove and Aaron Rieke , Discrimination through optimization: How Facebook's ad delivery can lead to skewed outcomes . In CSCW 2019.
- Joy Buolamwini and Timnit Gebru, Gender Shades: Intersectional Accuracy Disparities inCommercial Gender Classification . In FAT* Conference, 2018. [More Details ]
- Aylin Caliskan, Joanna J. Bryson and Arvind Narayanan, Semantics derived automatically from language corpora contain human-like biases . In Science April 2017.
- Le Chen, Anikó Hannák, Ruijin Ma and Christo Wilson, Investigating the Impact of Gender on Rank in Resume Search Engines . In CHI 2018. [Author copy of the paper ]
- Thomas Davidson, Debasmita Bhattacharya and Ingmar Weber, Racial Bias in Hate Speech and Abusive Language Detection Datasets . In AWL 2019.
- Batya Friedman and Helen Fay Nissenbaum, Bias in computer systems . In ACM Transactions on Information Systems 1996. [Full version ]
- Hila Gonen and Yoav Goldberg, Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them . In NAACL 2019.
- Ziad Obermeyer, Brian Powers, Christine Vogeli and Sendhil Mullainathan, Dissecting racial bias in an algorithm used to manage the health of populations . In Science 2019.
- Ronald E. Robertson, David Lazer and Christo Wilson, Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages . In WWW 2018. [Author copy of the paper ]
- Ronald E. Robertson and Shan Jiang and Kenneth Joseph and Lisa Friedland and David Lazer and Christo Wilson, Auditing Partisan Audience Bias within Google Search . In CSCW 2018. [Author copy of the paper ]
- Gabriel Stanovsky, Noah A. Smith and Luke Zettlemoyer, Evaluating Gender Bias in Machine Translation . In ACL 2019. [More Details ]
- Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan IV, Mark DM Leiserson and Adam Tauman Kalai, What are the biases in my word embedding? . In AIES 2019. [More Details ]
Explainable AI
- Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Dino Pedreschi and Fosca Giannotti, A Survey of Methods for Explaining Black Box Models . In ACM Computing Surveys 2018. [More Details ]
- Zachary C. Lipton, The Mythos of Model Interpretability . In ICML 2016 Workshop on Human Interpretability in Machine Learning. [More Details ]
- Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin, "Why Should I Trust You?": Explaining the Predictions of Any Classifier . In KDD 2016.
Feedback loops
- Eszter Bokányi and Anikó Hannák, Ride-share matching algorithms generate income inequality . Manuscript 2019.
- Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian, Runaway Feedback Loops in Predictive Policing . In FAT* 2018.
- Lily Hu and Yiling Chen, Fairness at Equilibrium in the Labor Market . In FAT/ML 2017.
- Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz and Moritz Hardt, Delayed Impact of Fair Machine Learning . In ICML 2018. [More Details ]
- Kristian Lum and William Isaac, To predict and serve? . In Significance 2016.
AI and Law
- Jack M. Balkin, The Three Laws of Robotics in the Age of Big Data . 2017.
- Solon Barocas and Andrew D. Selbst, Big Data's Disparate Impact . In California Law Review 2016.
- Ryan Calo, Artificial Intelligence Policy: A Primer and Roadmap . 2017.
- Alice Xiang and Inioluwa Deborah Raji, On the Legal Compatibility of Fairness Definitions . In Workshop on Human-Centric Machine Learning 2019.
Interfacing with Society
- Susan Athey, Beyond prediction: Using big data for policy problems . In Science 2017.
- Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon and Iyad Rahwan, The Moral Machine experiment . In Nature 2018.
- Sebastian Benthall and Bruce D Haynes, Racial categories in machine learning . In FAT* 2019. [More Details ]
- David Danks and Alex J. London, Algorithmic bias in autonomous systems . In IJCAI 2017. [Author copy ]
- Sina Fazelpour and Zachary C. Lipton, Algorithmic Fairness from a Non-ideal Perspective . 2020.
- Marion Fourcade and Kieran Healy, Seeing like a market . In Socio-Economic Review 2016. [Author copy ]
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume III and Kate Crawford, Datasheets for Datasets . In FAT/ML 2018.
- Bruce Glymour and Jonathan Herington, Measuring the Biases that Matter: The Ethical and Casual Foundations for Measures of Fairness in Algorithms . In FAT* 2019. [Author copy ]
- Ben Green, “Fair” Risk Assessments: A Precarious Approach for Criminal Justice Reform . In FAT/ML 2018.
- Anna Hoffmann, Where fairness fails: On data, algorithms, and the limits of antidiscrimination discourse.. In Information, Communication, and Society 2019.
- Zachary C. Lipton, The Mythos of Model Interpretability . In ICML 2016 Workshop on Human Interpretability in Machine Learning.
- Samir Passi and Solon Barocas, Problem Formulation and Fairness . In FAT* 2019. [More Details ]
- Andrew D. Selbst, danah boyd, Sorelle Friedler, Suresh Venkatasubramanian and Janet Vertesi, Fairness and Abstraction in Sociotechnical Systems . In FAT*2019. [More Details ]
- Harini Suresh and John V. Guttag, A Framework for Understanding Unintended Consequences of Machine Learning . 2019.