Ethical Machine Learning

Disciplines
AI
Machine Learning
Author

Marcos M. Raimundo

Published

June 3, 2024

This page consolidates the two editions of the Ethical Machine Learning course I taught: the first, in 2022/2023, in Portuguese and with the full content; and a second, in 2024, in English and with condensed content. The course equips students to identify and mitigate the negative impacts of data and machine learning tools on vulnerable groups, promoting transparency and ethical solutions. It involves data analysis, application of machine learning models, and understanding of optimization problems.

Final Objectives

Students should be able to identify uses of data and machine learning tools that cause harm to individuals and vulnerable groups, and to use and propose tools that reduce the impact and increase the transparency of such data and tools.

Prerequisites

Below are the skills desirable for students; they are not mandatory, but they increase the likelihood of good absorption of the content.

  • Ability to perform descriptive data analysis (basic statistics, data visualization).
  • Ability to apply and describe the characteristics of machine learning models (logistic regression, decision trees, and ensembles) on tabular data.
  • Ability to apply deep learning models to data of diverse nature.
  • Ability to understand and model optimization problems in the context of machine learning.

Diagnostic Assessment

In the first edition (2022/2023), the beginning of the semester included a diagnostic assessment of the class’s prerequisites:

  • A project carried out between the first and second weeks, with tasks designed to elicit a demonstration of each of the skills listed above.
  • An individual practical exercise, over the same period, with the same goal.
  • Four additional classes, by videoconference/hybrid format, one for each topic, to level up the minimum performance in each required skill.

Course Content

  • Unit 1. Introduction — Data laws, building trust, the ML lifecycle, and security. (2 weeks; only in the 2022/2023 edition)
    • Objective: enumerate the main desirable attributes of an ethical, transparent, and trustworthy machine learning model.
  • Unit 2. Data — Data sources, modalities, biases, privacy, and consent. (1 week)
    • Objective: prepare trustworthy and less biased data while preserving user privacy and consent.
  • Unit 3. Basic Modeling — Basic machine learning concepts, supervised learning, causal models. (1–2 weeks)
    • Objective: choose, and potentially create, machine learning models better suited to the prepared data.
  • Unit 4. Trustworthiness and Fairness — Detecting and promoting fairness in learning machines, maintaining trustworthiness under distribution shift or attack. (3 weeks in the 2022 edition; 5 in the 2024 edition)
    • Objective: choose, and potentially create/modify, more trustworthy and fair machine learning models.
  • Unit 5. Interaction — Interpretability in machine learning, and the design of intervention mechanisms. (3 weeks in the 2022 edition; 5 in the 2024 edition)
    • Objective: detect and modify a learning machine that fails to empower decision-makers to understand it and to detect failures in its trustworthiness and fairness.
  • Unit 6. Purpose — Social good, the influence of decision-makers on the design of machine learning products, challenges in machine learning. (1 week; only in the 2022/2023 edition)
    • Objective: enumerate the ethical problems in the application context, harmonize with the environment it is embedded in, and design a framework that reduces such problems in pursuit of social good.

In the 2024 edition, condensed and taught in English, Units 1 and 6 were folded into the others, and the course was reorganized into three parts: Data and Models (Units 2–3), Fairness (Unit 4), and Explainability (Unit 5).

References

  • General audience:
    • Weapons of Math Destruction, by Cathy O’Neil: a book presenting numerous examples of unethical algorithms (also available in Portuguese as Algoritmos de Destruição em Massa).
    • The Ethical Algorithm: The Science of Socially Aware Algorithm Design — introduces the main areas of ethics in ML (fairness, transparency, and privacy) for a general audience.
    • Coded Bias: a Netflix documentary that can serve as motivation for the course.
  • Main reference:
  • Additional references:
    • Fairness and Machine Learning: a freely available book, co-authored by three professors from different universities who teach Fairness or ML Ethics courses; the site also links to the pages of their respective courses.
    • Practical Fairness: an O’Reilly Fairness book included only as a caution that it’s not particularly worthwhile — it uses different terminology from what’s standard in the literature and doesn’t present the mathematical side well.
  • General resources:
    • Trustworthy ML Initiative: a range of Trustworthy ML resources, such as general-audience books, textbooks, documentaries, and course websites.