Mapping Part of the Fair Machine Learning Debate

In this research project we developed a system for mapping debates and then applied that system to map part of the debate about fairness in algorithmic decision-making systems developed via machine learning. Debate mapping, unlike argument analysis, is concerned primarily with relations between arguments, the dialectic of claims, objections, responses, rebuttals, etc. The main logical relations in a debate map are support and dispute, rather than deductive ones establishing validity, and every argument in a debate map is defeasible in the sense that its conclusion might be defeated by some further objection. In the first part of the project, we developed a fairly rigorous system for the theory and practice of debate mapping with an eye toward its eventual computerization. We tested and critiqued some existing systems. In the second part, we applied this system to manually map a fragment of the debate concerning the fairness of algorithmic decision-making systems developed via machine learning: the debate over the fairness of the recidivism-risk predictor, COMPAS. Other predictive algorithms developed by machine learning include systems such as automated resume screeners, loan application reviewers, recidivism risk predictors, etc. Many AI ethicists and others worry that these systems may make predictions that are unfair because they are created by training on historical data that may generate biases against traditionally disadvantaged groups. The relevant debate about fairness in philosophy and in the law is extensive, but much of it remains disconnected from the fair machine learning debate.