Automated Decision Making and Profiling

Automated Decision Making and Profiling is regulated by the The General Data Protection Regulations (GDPR). These have redefined how organisations handle personal data, refining and reforming rules to protect individuals’ privacy. In an era where automated decision making and profiling are becoming increasingly prevalent, understanding the rules and rights under the GDPR is crucial. Because these types of data processing can significantly impact individuals, they receive stricter oversight to ensure fairness, transparency, and accountability.

Contents

• Understanding Automated Decision Making and Profiling

• Legal Framework Under The GDPR

• Conditions for Lawful Automated Decision Making

• Explicit Consent

• Authorisation by Law

• Rights of Data Subjects

• Transparency and Accountability

• Implementing Safeguards

• Challenges and Risks

• Future Directions and Technological Advancements

• Conclusion

Understanding Automated Decision Making and Profiling

Automated decision making involves decisions made solely by automated means without any meaningful human involvement.

Profiling, on the other hand, refers to the automated processing of personal data to evaluate certain aspects of an individual, such as their behaviour, preferences, or performance. Common examples include credit scoring, targeted advertising, and recruitment algorithms. While these technologies offer efficiency and personalisation, they also pose risks to individual rights and freedoms.

Important Considerations:

• Transparency: people should be informed that automated decision making or profiling is used and have access to the criteria considered. People should also understand the legal effects that could arise from processing their data in this way.

• Fairness: The algorithms must be free from bias that could unfairly discriminate against certain groups.

• Human Oversight: There should be a mechanism for human review of automated decisions, particularly when significant consequences are involved (e.g., loan denial).

• Right to Explanation: Individuals should have the right to understand why an automated decision was made about them.

Legal Framework Under The GDPR

The GDPR addresses automated decision making and profiling primarily through Article 22, which prohibits decisions based solely on automated processing, including profiling, that produce legal effects or significantly affect individuals. However, there are exceptions to this prohibition, governed by stringent conditions and safeguards to protect data subjects’ rights.

Conditions for Lawful Automated Decision Making

Automated decision making is permissible under GDPR if it is necessary for the performance of a contract, based on the individual’s explicit consent, or otherwise authorised by law. Each condition requires careful adherence to ensure lawful processing:

Necessity for Contract

Automated decisions must be essential to fulfil contractual obligations with the data subject. Here are some examples:

1. Creditworthiness Assessments:

• Financial institutions often use automated algorithms to assess creditworthiness when evaluating loan applications.

• The algorithm might analyse factors like income, credit history, and debt-to-income ratio to determine eligibility and interest rates.

This is an important example because the contractual basis for processing personal data includes deciding whether or not to enter into a contract at all.

2. Employment:

• Increasingly employers are using automated decision making to sift through CVs and employment applications. They will then select a sub-set of applications received for human review.

This example helps to highlight the difference between automated decision making and profiling. It is appropriate to automatically remove applications from people without degrees where having a degree is an essential requirement. It is absolutely not appropriate to remove applicants based on age or gender etc.

Explicit Consent

Individuals must give clear and informed consent, understanding the implications of automated decisions. You can use explicit consent for any kind of automated decision making or profiling but consent is not always the best lawful basis to use because people have the right to withdraw consent at any time.

Also consent must be freely given. That means you cannot deny access to other products or services if people refuse consent for automated decision making or profiling.

Also consent must be granular. That means you must separate out consent for automated decision making from other consent-based processing of personal data.

As with the contractual basis, where automated decision making and profiling are done it must be clearly and separately set out in your privacy statement and captured in your record of processing activity.

Authorisation by Law

Specific laws may explicitly allow automated decision making and profiling in some circumstances. These laws will also typically require appropriate safeguards.

Identity Verification and Fraud Detection

• During the onboarding process, automated systems can verify a user’s identity using data like government IDs or perform fraud checks based on financial information.

Rights of Data Subjects

The GDPR empowers individuals with specific rights regarding automated decision making and profiling:

The Right to Be Informed

As noted above individuals must be informed about the existence of automated decision making, its significance, and consequences.

Right to Obtain Human Intervention

Data subjects can request human intervention to review and potentially alter automated decisions.

Right to Contest Decisions

Individuals have the right to challenge decisions made by automated processes and present their case to a human reviewer.

This is an important point to understand. People have the right to challenge the process not the outcome.

Transparency and Accountability

Organisations must maintain transparency in their automated decision making and profiling activities. This includes providing clear information about the logic involved, the significance, and the consequences of such processing. Accountability is achieved through comprehensive documentation, regular audits, and adherence to GDPR principles, ensuring that individuals’ rights are upheld at every stage.

Implementing Safeguards

To protect data subjects, organisations must implement technical and organisational measures. These safeguards include:

Data Minimisation

Limiting data collection to what is necessary for the intended purpose is a key GDPR principle. To get this right organisations must plan for the data they need and the purposes they need it for.

Anonymisation and Pseudonymisation

Protecting personal data by making it less identifiable is an important safeguard.

Regular Monitoring and Assessment

Continuously evaluating automated systems to ensure compliance and address potential issues.

Data Privacy Impact Assessments

Data Privacy Impact Assessments (DPIAs) are almost certainly necessary for identifying and mitigating risks associated with automated decision making and profiling. DPIAs involve:

• Evaluating whether the processing is necessary and proportionate to the purpose.

• Systematically assessing potential risks to data subjects’ rights and freedoms.

• Developing strategies to minimise identified risks.

Challenges and Risks

Automated decision-making (ADM) and profiling using personal data offer efficiency and convenience, but they come with inherent risks. Here’s a breakdown of some key concerns:

Bias and Discrimination:

• Data Bias: Algorithms can inherit and amplify biases present in the data they’re trained on. This can lead to discriminatory outcomes, denying opportunities or resources to certain groups based on factors like race, gender, or socioeconomic background.

• Algorithmic Bias: The design and development of ADM systems can introduce unintended biases. For example, facial recognition algorithms have been shown to have higher error rates for certain ethic groups, leading to disproportionate legal effects.

Transparency and Explainability:

• “Black Box” Problem: Complex algorithms can be opaque, making it difficult to understand how they arrive at decisions. This lack of transparency can make it challenging to assess fairness and identify potential biases.

• Right to Explanation: Individuals may have difficulty understanding how their data is used in automated processes, hindering their ability to exercise their right to explanation under regulations like GDPR.

Accuracy and Fairness:

• Data Quality: The accuracy of outputs depends heavily on the quality of the data used to train and run the algorithms. Errors or inconsistencies in data can lead to unfair or inaccurate decisions.

• Algorithmic Errors: Algorithms can make mistakes, especially when dealing with complex or nuanced situations. This can have negative consequences for individuals if not properly addressed.

Privacy Concerns:

• Data Collection and Use: ADM often relies on vast datasets, raising concerns about data minimization and potential misuse of personal information.

• Surveillance and Social Scoring: Extensive use of ADM for profiling can lead to intrusive surveillance and the creation of social scoring systems, impacting individuals’ freedoms and opportunities.

By acknowledging these risks and taking proactive steps to mitigate them, organisations can leverage the benefits of automated decision making and profiling while ensuring responsible and ethical use of personal data.

Future Directions and Technological Advancements

The landscape of automated decision making and profiling is rapidly evolving with advancements in AI and machine learning. As technologies advance, GDPR regulations must be understood to address new challenges and opportunities. Emerging trends, such as explainable AI and enhanced transparency mechanisms, offer promising solutions to balance innovation and data protection.

To balance this it is likely that there will be an increased emphasis on engagement with people to understand their personal preferences and provide assurance of compliance with their legal rights.

Conclusion

Automated decision making and profiling present both opportunities and challenges in the realm of data protection. GDPR provides a comprehensive framework to ensure these practices are conducted fairly and transparently, safeguarding individuals’ rights. By implementing robust safeguards, conducting impact assessments, and maintaining accountability, organizations can harness the benefits of automation while respecting data subjects’ rights. Balancing innovation with privacy protection remains a pivotal goal in the evolving landscape of data processing.