16.2. Key Ethical Principles in AI

AI models can inherit and amplify human and historical biases, especially in areas like hiring, lending, and criminal justice.

• Example: COMPAS algorithm used in US courts was found to be biased against Black defendants.
• Mitigation: Bias audits, balanced datasets, fairness constraints in training.

Fairness in AI refers to the need to ensure that algorithms do not perpetuate existing inequalities or create new ones. AI systems learn from data, and if this data reflects biases (like racial or gender biases), the AI can unintentionally repeat those biases in its decisions. The COMPAS algorithm, used in U.S. courts for assessing the risk of recidivism, has been criticized for being biased against Black defendants, leading to unjust outcomes. To combat this issue, strategies such as conducting bias audits, using datasets that are diverse and balanced, and applying fairness constraints during the training of AI models can help create more equitable AI systems.

Imagine a hiring software that learns from past hiring decisions, favoring applicants with similar profiles to current employees. If those employees are predominantly from one demographic, the software may unfairly prioritize candidates who look like them, potentially missing out on truly qualified candidates from different backgrounds. By implementing checks and diversifying the training data, the software could help level the playing field for all applicants.

The "black-box" nature of many machine learning models, especially deep learning, makes it hard to explain decisions.

• Solution: Use explainable AI (XAI) tools like SHAP, LIME, and model-agnostic techniques.
• Importance: Crucial in high-stakes applications (e.g., healthcare or finance).

Many AI systems operate as 'black boxes', meaning their internal workings are not transparent and their reasoning is often difficult to understand. This lack of clarity can be particularly problematic in critical areas such as healthcare or finance, where understanding the basis for a decision can have significant consequences. To address this, explainable AI (XAI) tools like SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) can be employed. These tools help provide insights into how and why a model arrived at a specific decision. Transparent AI systems foster trust and allow users to audit and validate AI outcomes.

Consider a doctor relying on an AI system to recommend treatments for patients. If the AI suggests a medication without explaining why it made that choice, the doctor might feel uneasy. By using XAI tools, the AI could explain which patient data points led to the recommendation, thereby empowering the doctor to make an informed decision that they can trust.

AI systems often require large amounts of personal data, raising concerns about surveillance and consent.

• Practices: Data minimization, anonymization, federated learning, differential privacy.

The privacy of individuals is at risk due to the vast amounts of personal data needed to train AI systems. This data can include sensitive information that, if leaked or mishandled, could lead to breaches of privacy. To protect user privacy, practices such as data minimization (only collecting necessary data), anonymization (removing identifiable information), federated learning (training models without centralizing data), and differential privacy (adding noise to data to protect individual identities) can be utilized. These methods help ensure that user consent is respected and that personal information is safeguarded.

Think of how you share your personal information online. If you're filling out a health app, you'd want to ensure that it only collects the data required for it to function effectively, like your exercise routines and dietary habits, without storing sensitive information like your address. By implementing privacy-preserving methods, the app could learn from the data without ever needing to know exactly who you are or where you live.

Who is responsible when an AI system fails? The developer? The organization? The user?

• Frameworks: Model documentation (e.g., Model Cards), ethical AI committees, regulatory oversight.

With AI systems playing critical roles in many sectors, understanding accountability when things go wrong is essential. If an AI model makes an erroneous decision, it raises the question: who is liable? Is it the developer who created it, the organization that deployed it, or the user relying on it? To establish clear accountability, frameworks such as model documentation (like Model Cards that detail a model's intent and functionality), ethical AI committees to oversee decisions, and regulatory oversight can help clarify responsibilities and ensure accountability is integrated into AI processes.

Imagine an autonomous vehicle gets into an accident. Is the car manufacturer responsible for the design? Is the software developer accountable for the algorithm? Understanding these relationships is vital, much like knowing who is at fault in a car accident. Clear regulations and frameworks can guide stakeholders on how to proceed when failures occur, shifting the focus from blame to improvement.

AI systems must be secure from adversarial attacks and robust in unpredictable environments.

• Risks: Adversarial examples, data poisoning, model inversion attacks.
• Approaches: Secure model training, red teaming, robustness testing.

AI systems are vulnerable to various security threats, including adversarial attacks—deliberate attempts to trick the model into making incorrect predictions. For instance, slight modifications to input data can lead to significant errors. Additionally, attackers might use data poisoning (injecting malicious data into training sets) or model inversion (recovering sensitive information from model outputs). To counter these risks, practices like secure model training, red teaming (where teams simulate attacks), and robustness testing (evaluating how models perform under adverse conditions) can help bolster the security and dependability of AI systems.

Think of a door lock for your home. Just like you wouldn’t use a flimsy lock that could easily be picked, AI systems should be designed with robust security features to withstand potential intrusions. Conducting 'what-if' scenarios during the development process can help identify vulnerabilities before the AI system is deployed in the real world, ensuring it can handle unforeseen challenges.

--