AI Agents – When agency and subordination go hand in hand

This is an article that I wrote in the context of a Chatham House rules roundtable on questions to be tackled more extensively in the future regarding the interaction between AI systems & data protection. This particular one relates to AI agents and the notion of authority, notably as regards (i) the user’s perspective (subordination?), (ii) the AI agent provider (how to limit issues?) and (iii) the publisher (what to do with bot identification in the era of AI agents?). Do share your thoughts on the topic too!

For more content on AI & data protection, read my two other articles on (i) whether AI training data = (non-)personal data? And is consent really relevant? and (ii) AI training & legitimate interest as a GDPR legal ground.

---

If you let an AI system do anything on your behalf, to what extent is that system acting under your authority? Up until recently, the focus would have been on placed on the issue of liability (and the question would have talked about “robots” rather than “AI systems”). When using a robot, is the robot’s end-user liable? What about the operator or manufacturer? Today, the question appears to have broadened, covering also data protection compliance.

Fundamentally, an AI agent is an AI system that is granted a certain degree of autonomy over how to achieve a particular task defined by an end-user or operator. It is a kind of AI system (following the OECD definition*) with a greater focus on the interaction with a broader environment.

* OECD AI Principles (as updated in May 2024), definition of “AI system”: “An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.”

---

The travel AI agent example

The example of an AI agent asked to prepare a trip illustrates this well.

The AI agent is given a broad instruction (such as “I want a weekend trip for two adults in a romantic city we haven’t yet visited”), the AI agent then interprets the uses information at its disposal (such as travel history, user interests, travel origin [to determine what makes sense for a weekend trip], etc.) and researches options for the user.

Depending on its autonomy, the AI agent might either prepare a detailed plan with recommendations, with a “book or not” choice presented to the user, or actually make the necessary bookings without requesting the user’s input.

Throughout the process, the AI agent interacts with other systems dealing with mobility (e.g. map systems to look up public transport and car transport options, booking systems for flights/trains/etc.), accommodations (e.g. if a hotel is needed before an early flight out, a hotel or bed & breakfast on location), tourist attractions, restaurants and more.

The AI agent might interact purely with internal systems, in the case of an undertaking with own systems for the various stages, just as it might act with external systems, whether through emulating the web browsing behaviour of a human or through interacting with systems through publicly usable application programming interfaces (APIs). Many variations in functionality and many degrees of autonomy are possible for one same type of AI agent.

This autonomy creates a range of (surmountable) challenges from a data protection perspective (and also regarding privacy), as the user in effect permits the AI agent to carry out a range of operations on the user’s behalf, some of which involve disclosing personal data regarding the user to a third party. It is important to examine this from three perspectives: (i) the user, (ii) the AI agent provider and (iii) the publisher with whose website or app the AI agent interacts.

---

i) The user’s perspective: subordination subject to clarification

The user wishes to delegate as much as possible to the AI agent, but the AI agent must not cause unlawful acts such as the provision of personal data in a manner that the user cannot foresee.

When a user inputs information in a web form, there are mechanisms to help the user to become aware of the website publisher’s data protection practices (such as disclaimers, privacy notices).

These do not have to be seen by a human at all times. For instance, other tools for the automatic filling out of forms (such as “auto-fill” functionality for contact details, or password managers), require at least one action of human intervention for the actual submitting of the form. There are also other tools for the consultation of websites (such as for automatic monitoring of changes to a website), yet they require that the user specifies which website. The user is thus aware or at the very least ought to have been aware of the data protection practices of the website in question.

Through the intervention of an AI agent, on the other hand, the user is not directly confronted with such disclaimers or notices and might not even be aware in advance of which websites will be consulted. A checkbox for acceptance of terms or for consent might be checked without the user knowing there is one on the relevant website, and the explanations on which a controller might have spent considerable time may never be made available specifically to the user.

Yet the user chooses to ask the AI agent to carry out a certain (set of) operation(s). The processing would not occur without the user’s general instruction, such that the AI agent – a mere tool, not a natural or legal person – must be considered as an extension of the user in this regard.

Just as a controller is deemed to have authority over processors and other persons acting on the controller’s instructions, so it might be with the user and AI agents. While instructed third parties are considered as controllers in some cases (such as consultants and lawyers), this controllership is due to the fact that the third party in question has its own purposes for processing. Even if one were to consider an AI agent as similar to a natural or legal person, it has no own purposes for processing the user’s personal data and does so only for the purposes set by the user.

The AI agent thus has autonomy, but within the framework of subordination.

For a user to understand what the AI agent has done with his or her data, it may thus be important for the user to request from the AI agent a record of all websites to which the user’s personal data was submitted. In addition, it is better for the user to verify the precise data being used by the AI agent, as any issues in terms of data accuracy in the input being provided will be the responsibility of the user, not of the publisher who uses the data as provided.

Could the AI agent be used also to make privacy notices and other information accessible, or even more intelligible?

Just as users have started to use AI systems such as LLMs to summarise documents, so can AI agents be requested to summarise privacy notices, or to help select whichever privacy-related settings are most in line with the user’s expectations. Yet the use of the AI agent remains the user’s sole responsibility in this respect: if an important part of the privacy notice is not reflected in the summary generated by the AI agent, the user will still be deemed to be aware of the entire, original privacy notice, not merely the AI-generated summary.

---

ii) The AI agent provider: actions to improve experience and limit issues

Given the concerns mentioned above and the perspective of the user, it is worth considering the perspective of the AI agent provider.

While there may be cases in which the AI agent provider’s liability is invoked from a data protection perspective (for instance, regarding own use of personal data for customer management purposes), there are limited reasons to invoke the AI agent provider’s liability in relation to processing carried out by the AI agent on behalf of the user (save possibly for unlawfulness of which the AI agent provider ought to have been aware before a given use by the user).

Instead, the focus for the AI agent provider may be on ways to improve the ability of the AI agent to help the user maintain the autonomy or information that matters the most.

For instance, the provider could consider building into the functionality of the AI agent the recording of every website interaction, with the precise URL of the relevant page(s) (as is already the case in some such tools, often under a “sources” or “tasks” type of section).

In addition, it may be worthwhile from a risk mitigation perspective to ensure that before any action is undertaken that involves the transmission of personal data to a third party, the user is presented with a confirmation request, with the possibility to review the filled in form on the actual webpage (and thus to manually submit the form or to block the contemplated action from being completed).

In other words, the AI agent provider should ensure that the autonomy of the AI agent still reflects subordination.

---

iii) The publisher: moving away from human vs bot, to legitimate vs illegitimate user

For publishers, the first concern with AI agents is simply the fact that this is not a website visit by a human but a bot. This has an impact on monetisation (for ad-funded businesses), costs and security (due to implications of bot traffic and spam). The security aspects create a need for collaboration and a good faith approach between AI agent providers and publishers, so that legitimate use (by bots & humans) can be recognised. This assessment, though, requires data.

Another key concern for publishers is how to accommodate for the fact that an increasing proportion of the personal data they receive may come from AI agents. While some may claim that it is worthwhile making documentation more “LLM-friendly”, the truth is that “LLM-friendly” is “human-friendly”, due to a focus on clarity and structure (which contribute to intelligibility – a statutory requirement for any information vis-à-vis data subjects). As a rule, then, AI agent usage is another opportunity to test intelligibility and accessibility of information. This way, whether the interaction comes from a human or an AI agent, the information can be properly and adequately brought to the attention of the human user.

More generally, the use of AI agents provides another excuse to review the method for recording user actions and for considering whether the action should lead to a confirmation by e-mail, a change in user account settings, an in-app notification or no notification whatsoever. The added benefit is that this becomes useful evidence that legitimate users are empowered.

---

The use of AI agents presents an opportunity to rethink controls, transparency and evidence of compliance. It is not without its challenges (notably regarding monetisation and security) but taking this use into account helps ensure that the user experience is as smooth as possible.