Definitions of fairness in machine learning explained through examples

By David Griffin

Algorithms dominate many aspects of our lives. They play largely unseen roles in everything from determining which advertisements target us online to who will be hired for a job. Since these algorithms are based in mathematics, it is understandable to assume they are fair and unbiased. However, this is rarely the case. Since they are predominantly written either directly or indirectly by humans, they are often filled with unidentified biases which can negatively impact women and minorities.

A recent paper by Verma and Rubin (2018) sought to explore definitions of fairness for classification in machine learning (ML). In this context, classification refers to the labelling or categorising of data based on predictions made from a large exemplary dataset. Common definitions of fairness found in artificial intelligence (AI) and ML literature were then illustrated by the authors using a single example; the German Credit Dataset (GCD) (Lichman, 2013). This dataset, used in many papers to explore fairness through example, contains information about 1,000 German loan applicants from the 1990s with their individual characteristics as applicants. The 20 characteristics listed include details such as marital status, age, gender, number of dependents, credit history and occupation. In their paper, Verma and Rubin (2018) illustrated the various definitions of fairness used in AI and ML literature, with an emphasis on gender bias and discrimination. An overview of this work is provided in the following section.

Definitions of Fairness

‘Group fairness’, ‘Statistical parity’, ‘Benchmarking’ or ‘Equal acceptance rate’ - This definition of fairness is met if individuals with both protected and unprotected characteristics are equally likely to receive a positive classification prediction. Protected characteristics are those which should not be used to discriminate between people, such as gender or disability. In the example of the GCD, this would mean that male and female applicants would have equal likelihood of receiving a good credit score prediction.

‘Conditional statistical parity’ – This definition of fairness implies that subjects of both protected and unprotected characteristics are equally likely to receive a positive classification prediction, when other legitimate factors are taken into account. When using the GCD example, legitimate factors would include credit history and employment. Thus, male and female applicants, given parity in credit history and employment credentials, would be equally likely to be predicted a good credit score.

‘Predictive parity’ or ‘Outcome test’ - This definition of fairness is satisfied if subjects of both protected and unprotected characteristics have equal probability of receiving a positive classification when predicted one due to their characteristics. Using the GCD example, this would mean that both male and female applicants would actually receive a good credit score when predicted a good credit score. That is to say they have equal positive predicted value (PPV).

‘False positive error rate balance’ or ‘Predictive equality’– This definition is true of a classifier if subjects with protected and unprotected characteristics are equally likely to receive a positive classification when predicted a negative one. In the GDC example, this would mean that both male and female applicants with bad credit scores would be equally likely to be predicted a good credit score by the classifier.

‘False negative error rate balance’ or ‘Equal opportunity’ – This definition is the reverse of the last; it is true of a classifier if subjects with protected and unprotected characteristics are equally likely to receive a negative classification when predicted a positive one. Intuitively, in the GDC example, this would mean that both male and female applicants with good credit scores would be equally likely to be predicted a bad one by the classifier.

‘Equalised odds’, ‘Conditional procedure accuracy equality’ or ‘Disparate mistreatment’ – This definition can be deemed true if both of the previous two definitions are true.

‘Conditional use accuracy equality’ – This definition of fairness is met by a classifier if the probability of those who receive a positive classification and are deserving of one is equal to the probability of those who receive a negative classification and are deserving of one, for subjects with both protected and unprotected characteristics. Using the GDC example, this would suggest that both male and female applicants would be equally likely to receive the appropriate credit score classification.

‘Treatment equality’ – This definition is met by a classifier if both subjects with protected and unprotected characteristics have an equal ratio of false positive classifications to false negative classifications. In the GDC example, this would mean that the ratio of those receiving incorrectly positive predicted credit scores to those receiving incorrectly negative predicted credit scores would be the same for both male and female applicants.

‘Test fairness’, ‘Calibration’, ‘Matching conditional frequencies’ or ‘Well-calibration’ – This definition of fairness is met if subjects with protected and unprotected characteristics are both equally likely to belong to the positive classification. Within the GDC example, this would mean that both male and female applicants who apply for a loan would be equally likely to actually hold a good credit score.

‘Balance for positive class’ – This definition is met if subjects with protected and unprotected characteristics have, on average, the same predicted probability of being classified positively. In the GDC example, this would mean that both male and female loan applicants with a good credit score would have equal probability of being predicted a good credit score.

‘Balance for negative class’ – This definition of fairness is essentially the opposite of the last. It is met if subjects with protected and unprotected characteristics have, on average, the same probability of being classified as negative. Within the context of the GDC example, this would mean that both male and female loan applicants with a bad credit score would have equal probability of being predicted a bad credit score.

‘Causal discrimination’ – A classifier meets this definition of fairness if all subjects with identical attributes are classed as equal. In the GDC example, this would mean that both male and female loan applicants with identical attributes would both either receive a good or bad predicted credit score.

‘Fairness through unawareness’ – This definition of fairness is similar to the last. It is met if a classifier does not use protected characteristics in its classification. In the context of the GDC example, this would mean that whether a subject was male or female would not be considered when applying for a loan.

Conclusions

Verma and Rubin (2018) provide a concise overview of definitions of fairness in classification through ML. It is hoped that this summary overview will provide readers with an introduction to the many and varied designations of fairness. It is clear from their work that measuring fairness is a complex and challenging task. The authors stress that the applicable definitions will vary with the intended use of each and every system. They also highlight the fact that while verifiable outcomes are available for training data, it is challenging to ascertain whether real data will follow the same distribution. Consequently, inequality may exist in a system despite its training to avoid this.

FAQ

Q: How do the definitions of fairness discussed in Verma and Rubin's paper align with the perceptions of fairness among students, particularly those from diverse backgrounds?

A: The definitions of fairness outlined by Verma and Rubin provide a mathematical and theoretical framework for understanding and addressing bias in machine learning algorithms. However, the alignment of these definitions with the perceptions of fairness among students, especially those from diverse backgrounds, may vary significantly. Students' perceptions of fairness are shaped by their personal experiences, cultural contexts, and the social dynamics they observe or are subjected to. Incorporating student voice into the conversation about fairness in algorithms can bridge the gap between theoretical definitions and lived experiences. It allows for a more nuanced understanding of fairness that considers the complexity of human diversity and the subtleties of discrimination. By engaging with students and valuing their perspectives, researchers and developers can ensure that the definitions of fairness they employ are more holistic and reflective of the realities of all individuals, particularly those from underrepresented groups.

Q: What methodologies or approaches can be utilised in text analysis to identify and mitigate biases in datasets like the German Credit Dataset, especially concerning underrepresented groups?

A: Identifying and mitigating biases in datasets, such as the German Credit Dataset, requires a multifaceted approach that combines technical methodologies with an understanding of social contexts. Text analysis, as a tool, can play a crucial role in this process. Techniques such as sentiment analysis, thematic analysis, and the use of natural language processing (NLP) algorithms can help uncover patterns, biases, and underlying themes in the data that may not be immediately apparent. For instance, text analysis can reveal discriminatory language or bias in the descriptions of applicants' profiles. Additionally, incorporating student voice into the analysis process can provide insights into the nuances of bias and fairness from diverse perspectives. Engaging with students from underrepresented groups to review and interpret the findings of text analyses can help validate the results and ensure that the biases identified are meaningful and accurately reflect the experiences of those most affected. This collaborative approach not only enhances the rigour of the analysis but also promotes inclusivity and fairness in the development of machine learning models.

Q: How can student input be incorporated into the development and evaluation of fair machine learning models, particularly in applications that impact student life directly, such as admissions or financial aid algorithms?

A: Incorporating student input into the development and evaluation of machine learning models is essential for ensuring the fairness of applications that directly impact student life, like admissions or financial aid algorithms. One effective way to involve students is through participatory design sessions, where students from diverse backgrounds can share their experiences, concerns, and suggestions for the algorithms that affect them. These sessions can help identify potential biases and areas where the model might not align with the students' perceptions of fairness. Additionally, surveys and focus groups can be used to gather a broader range of student voices and perspectives. Involving students in the evaluation phase through user testing and feedback collection can also provide insights into the real-world implications of the models and highlight areas for improvement. By actively engaging with students throughout the development and evaluation process, institutions can ensure that their machine learning models are more equitable, transparent, and responsive to the needs and values of all students, thereby fostering trust and fairness in technological applications that play a significant role in their academic and personal lives.

References

[Source Paper] Verma, S., Rubin, J. 2018. Fairness Definitions Explained. 2018 ACM/IEEE International Workshop on Software Fairness (FairWare), 2018, pp. 1-7.
DOI: 10.1145/3194770.3194776

[1] Lichman, M. 2013. UCI Machine Learning Repository.
Available Here.

Related Entries