AI systems can use personal data, in particular to train machine learning models, and also work with it in the actual operation and processing of queries and prompts. The use of personal data then makes the task more specific and the results more accurate. At the same time, there may be AI systems specifying built on working with and processing personal data. It is essential that personal data is processed responsibly and in accordance with privacy regulations.
EDPB Opinion
On 17 December 2024, the European Data Protection Board (“EDPB“) adopted Opinion No. 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models (“Opinion“)[1]. This Opinion was prepared following a request from the Irish Data Protection Commission, pursuant to Article 64(2) of the EU General Data Protection Regulation (“GDPR“).
This is currently a fairly fundamental statement in the area of data protection in relation to AI given the focus on key aspects of personal data processing in the development and deployment of AI models. While not legally binding, EDPB opinions are strongly advisory and provide a consistent interpretation of GDPR rules across EU Member States.
Data Protection Authorities (“DPAs“) in EU Member States often follow these opinions as they reflect a harmonised approach to data protection in the EU, so it is certainly worth paying attention to this Opinion when monitoring the evolution of the relationship between data protection and AI.
The opinion responds to four areas of specific questions:
(i) What about the anonymity of AI models when processing personal data – can AI models be anonymous?
(ii) Can the legal basis of legitimate interest be relied upon for AI-related processing of personal data?
(iii) What are the reasonable expectations of individuals regarding the processing of their personal data in AI models?
(iv) What is the impact of unlawful processing of personal data at an early stage of development on the lawfulness of subsequent processing or operation of the AI model?
Before discussing the answers to each of the question headings, it is interesting to point out some very interesting concepts and interpretations by the EDPB that are new to the field of AI in relation to the GDPR: the EDPB presents its own concept of certain phases of the AI model life cycle, such as “development” and “deployment”.
The EDPB views the development of an AI model as all the phases leading up to deployment, including the creation of code, the collection of personal training data, the preprocessing of that data, and the training process itself
The deployment phase, on the other hand, refers to all phases where the AI model is actively used after development is complete.
The EDPB points out for the different development phases that development and deployment may constitute separate processing activities with different purposes, which require individual assessment for each legal basis for processing, compliance with data protection requirements and the responsibilities of the data controller. The EDPB notes, of course, that the AI lifecycle is not strictly divided into these two phases in practice and that some processing activities included, for example, in the development phase may also occur in the deployment phase (for example, for model retraining).
Now to the specific comments on the questions raised in Opinion:
Anonymity of AI models
The EDPB states that the assessment, either by the controller or the DPA, of whether an AI model developed using personal data can be considered anonymous should be carried out on a case-by-case basis
The EDPB considers that an AI model can only be considered anonymous if it cumulatively meets the following assumptions:
- the likelihood of direct (including probabilistic) acquisition of personal data about individuals was negligible; and
- the likelihood of obtaining this data from interviews, whether intentionally or unintentionally, was also negligible.
The EDPB stresses the need for a thorough assessment of the likelihood of identification of the data subject on the basis of the data collected in order to classify the model as anonymous. This assessment is based on Recital 26 of the GDPR, which emphasises the reasonable means that could be used by the controller or another person and should take into account the potential unintended re-use or disclosure of the model.
According to the Opinion, when assessing anonymisation, DPAs should require documentation from data controllers that includes technical and organisational measures, attack resistance model tests and identification risk analyses. For example, protection against so-called data regurgitation, where a model incompletely reproduces training data, is key.
Development and deployment of AI model based on legitimate interest
The EDPB emphasises that legitimate interest under Article 6(1)(f) GDPR cannot be the ‘default’ legal basis for the processing of personal data when training and using AI models.
It is acceptable only if the controller demonstrates, through the three-step proportionality test of legitimate interest, that the processing of personal data related to the AI model is proportionate, necessary and efficient to achieve the intended purpose.
The Opinion sets out rather general considerations that should be taken into account when assessing whether legitimate interest is an appropriate legal basis for the processing of personal data in the development and deployment of AI models. The EDPB refers to its earlier guidance on legitimate interest, which contains a three-step test to assess the appropriateness of this legal basis. As an example, the Opinion cites the use of legitimate interest for conversational agents or to improve cybersecurity.
So the three-step test proceeds as follows:
- Is the legitimate interest legal, clearly defined and real?
- Is the processing really necessary to achieve a legitimate interest?
- Does the legitimate interest not outweigh the rights of data subjects?
Reasonable expectations of individuals
The Opinion sets out specific criteria that DPAs may consider when assessing whether an individual could reasonably expect certain processing of their personal data in AI models.
These criteria include: (a) the public availability of the data; (b) the nature of the relationship between the data subject and the controller; (c) the nature of the service provided; (d) the context of the data collection; (e) the source of the data; (f) the possible future use of the AI model; and (g) the data subject’s awareness that their data is online.
The opinion underlines that transparency is indeed key, especially for technologies involving web scraping or the processing of publicly available data. Organisations using AI in this way must therefore ensure that data subjects have access to clear and comprehensible information about the processing of their personal data.
Impact of illegal processing on subsequent AI model operations
The Opinion examines various situations where unlawful processing of personal data in the development of an AI model could affect its subsequent use.
According to the EDPB, there are three likely scenarios of illegal processing:
- The unlawfully processed personal data is still part of the model and is used by the same controller. The assessment depends on whether the processing is separate and whether the principle of the legal basis has been respected.
- A model with illegally processed personal data is used by another controller. This controller must carry out its own assessment of compliance with the GDPR.
- The model has been anonymised before further use. In this case, the GDPR does not apply to the further use, unless there is further processing of personal data.
The EDPB also focuses on how unlawful processing during the development phase may affect the lawfulness of subsequent processing or operation of the AI model (i.e. compliance with Article 5(1)(a) and Article 6 of the GDPR).
In cases of violation of these articles, corrective measures may be ordered by the supervisory authority, such as:
- fines,
- temporary processing restrictions,
- deletion part or all of the dataset,
- or retraining the AI model.
The Opinion does not address how these obligations of the administrator affect other participants in the AI chain (developers, providers or operators) as defined in the now oft-cited AI Act. According to the EDPB, these cases continue to be assessed on a case-by-case basis.
Conclusion
The opinion underlines the EDPB’s commitment to ensuring that AI developments are in line with the principles of the GDPR and the balance between innovation and privacy.
AI technologies raise a large number of ethical and legal question marks. Organisations must therefore ensure that the development and deployment of AI models are GDPR compliant. Transparency, data minimisation, ensuring data subjects’ rights and responsible handling of personal data are key factors.
The implications for data controllers are twofold in practice: documentation and risk assessment. Data controllers must therefore keep comprehensive documentation that demonstrates that measures have been taken to anonymise personal data, assess legitimate interest, comply with the data protection principles.
The Opinion underlines the need for regular risk assessments, the assessment of potential impacts on the fundamental rights of data subjects and, last but not least, constant consideration of technological developments The EDPB itself points out that AI technologies are evolving rapidly and that its recommendations should be interpreted in the context of these technological advances. This flexible approach aims to ensure that the opinion remains relevant, while providing clear principles to ensure consistency. The Opinion represents an important step towards clarifying the application of the privacy principles to AI models, providing practical guidance while maintaining the necessary flexibility to address future technological developments.
If you have any questions about the legal regulation of AI and/or the processing of personal data, we at PEYTON legal are at your disposal.
Mgr. Tereza Pechová, junior lawyer – pechova@plegal.cz
Mgr. Jakub Málek, managing partner – malek@plegal.cz
16. 1. 2025
[1] The original version of the opinion is available here: https://www.edpb.europa.eu/our-work-tools/our-documents/opinion-board-art-64/opinion-282024-certain-data-protection-aspects_en