So, about the Dutch DPA scraping guidelines: based on my experience, they are just wrong. I know they have gone viral in the data protection & AI circles, but they are based on premises about scraping that might not align with best practices. Why?
First, much in the scraping guidelines of the Autoriteit Persoonsgegevens (AP) is based on the premise that the only logical legal ground for many cases – legitimate interests – will be unavailable, because the AP continues to claim that commercial interests cannot be legitimate. [Nearly everyone else says that “legitimate” = “not prohibited by law”.]
Second, and more importantly, these guidelines are framed as “scraping involves the copying of all information on a website, so lots of personal data will be processed”.
Based on the scraping cases I have worked on, this is *wrong*. [Disclaimer: some were cases regarding Dutch companies or came to me through Dutch lawyers.]
Why? Excluding search engines (indexing is a very specific use case), let’s look at two scenarios:
Scenario 1: I do the scraping myself
If I am doing scraping on my own, it’s too resource-intensive (processing power + storage) to do so indiscriminately, so I am selective: which websites are relevant, and within them, which sections.
To illustrate: while I may load the page in full, I might not want the Javascript code, the website headers and navigation, etc., so I may wish to only store the relevant sections (e.g. content within an
Do we really want to say that there is processing by virtue of the loading of the page? That’s opening a can of worms.
Scenario 2: I rely on a repository (e.g. CommonCrawl or LAION-5B)
In this case, I download a massive repository (several terabytes of data). I don’t yet know what is in there and start my analysis. [Let’s leave aside for now whether this is lawful or not]
I will then probably *also* want to be selective – if I am training an LLM, I want to give more weight to better sources (e.g. Stackoverflow > Reddit for coding; The New York Times > 4chan for a professional manner of expression). [This happens if you aren’t that selective: https://lnkd.in/dcd2gSa7 ]
It is therefore already *less likely* that there will be actual processing of personal data.
Next, in both cases, I will likely use pattern recognition to exclude some information. If I don’t care who the author of something is or what their date of birth is, that can be largely excluded.
And that’s without counting the question of whether an LLM actually processes personal data. [Not quite the “yes” that e.g. noyb claims]
So is scraping “almost always illegal” under the GDPR? Not necessarily if you approach it carefully.
If you need help with that, reach out!
Did this analysis get you thinking? Reach out!
DataLaws.net is entirely open-access, and instead of getting your data in exchange for this content, how about another trade? If this commentary saved you research time or sparked an idea, feel free to invite me over for tea, chai or a hot chocolate next time you are around Brussels or Antwerp - or invite me over to your offices for a chat!
Get in touch ↗ Let's connect on LinkedIn ↗