English
Romana
In Romania, a web crawling project is legal when it satisfies three conditions at the same time: the target site's robots.txt allows access to the pages you want, its Terms and Conditions don't explicitly forbid automated extraction, and the data collected doesn't include personal information without a documented legal basis. The three conditions work independently — meeting one does not replace the other two.
The most common mistake isn't deliberately ignoring the rules; it's treating robots.txt and GDPR as if they were the same check. Robots.txt answers "am I technically allowed to access this page?"; GDPR answers a completely different question — "am I allowed to store the data found on that page?". A crawling project can pass the robots.txt test and still breach GDPR, if the allowed pages contain personal data collected without a legal basis.
This article puts the two checks into a single decision matrix, applied to the data categories a typical eCommerce or market-research crawling project encounters in Romania, and explains where Romania-specific legislation (ANSPDCP, Law 8/1996 on copyright) applies on top of the general GDPR and robots.txt framework.
The decision matrix: robots.txt + GDPR combined per data category
The table below combines the two filters into a single practical outcome, for the data categories most commonly encountered in a business-oriented crawling project.
| Data category | Does robots.txt usually allow it? | GDPR risk | Can you extract it? |
|---|---|---|---|
| Title, price, product code, stock | Yes, if the product page isn't in Disallow | Low | Yes, if the Terms and Conditions also allow it |
| Technical description, specifications | Usually yes | Low | Yes, watch for copyright on the text |
| Product images | Depends on the image directory | Low | Conditional on copyright (Law 8/1996) |
| Reviews with full name or username | Usually yes | Medium-high | Only with the author anonymized or excluded |
| Publicly listed emails, phone numbers | Usually yes | High | No, without a documented legal basis |
| Admin pages, internal search, account pages | Usually blocked via Disallow | Variable | No — respect the technical block |
This classification is indicative and depends on the specific target site; the matrix shows the general pattern, not a legal guarantee for every case.
Robots.txt confirms technical access, not full legal compliance
Robots.txt is a public text file, placed at the root of a domain, through which a site owner indicates which pages robots may access and which pages it prefers to keep unaccessed. It's the first check, not the only one — for a full explanation of the directives and the risks of ignoring it, see the dedicated guide on robots.txt and legal crawling.
For this article, keep one thing in mind: a permissive robots.txt only confirms technical access to a page. It says nothing about what you can legally do with the data found there, especially if that data includes information about individuals.
GDPR decides what you do with the data you find, regardless of robots.txt
GDPR applies to any processing of personal data belonging to individuals in the European Union, regardless of where the collecting company is based and regardless of whether the information is public on a website or not. Publicly visible information doesn't automatically lose its status as personal data.
For a detailed breakdown of legal bases, data categories, and risk-reduction steps, see the HappyWeb guide on GDPR and personal data in crawling. This article doesn't repeat that analysis in detail; it focuses on how the GDPR outcome combines with the robots.txt outcome into a single per-project decision.
Romania-specific: ANSPDCP and Law 8/1996
On top of the general GDPR framework, applicable across the entire European Union, a crawling project run from Romania or targeting a Romanian site intersects two additional local elements:
- ANSPDCP (the Romanian National Supervisory Authority for Personal Data Processing) is the national authority that enforces GDPR in Romania and can investigate complaints related to unauthorized data collection, including data obtained through crawling.
- Law 8/1996 on copyright and related rights protects, under Romanian law, original descriptions, texts, and images published on a site, regardless of robots.txt or GDPR — reproducing this content identically without right can constitute a separate breach, distinct from data-protection issues.
In practice, this means a crawling project that respects both robots.txt and GDPR can still run into a legal problem if it reproduces protected descriptions or images identically, instead of processing or adapting them for its own use.
How to decide, step by step, whether you can extract a data category
For each type of data you want to extract from a site, run through the sequence below:
- Check whether the page containing the data is allowed in robots.txt for the user-agent your script uses.
- Read the target site's Terms and Conditions for clauses about automated extraction, content reproduction, or explicit scraping bans.
- Ask whether the data identifies, directly or indirectly, a natural person — if so, you need a documented legal basis before collecting it.
- If the data is original text or images (not simple facts like price or code), decide whether you intend to reproduce it identically or to process/adapt it.
- Document internally the outcome of this check for every new source, before launching collection at scale.
- For large data volumes or sensitive sources, request a legal review before building the final script.
Common risks when the two checks are treated as one
The most common project error is stopping the check after robots.txt, assuming that "if I can technically access it, I can store whatever I find there." The practical consequences of this confusion:
| Common mistake | Possible consequence | Mitigation |
|---|---|---|
| Robots.txt checked, GDPR ignored | Unintended collection of names/emails from reviews or profiles | Explicitly filter personal fields in the script, at the source |
| Identical reproduction of product descriptions | Risk of breaching Law 8/1996 on copyright | Process/adapt the text, don't copy it identically |
| Ignoring the Terms and Conditions | Cease-and-desist request from the site owner | Read the automated-extraction section before the project |
| No internal documentation of the decision | Hard to demonstrate good faith if a complaint is raised | Note, per source, what was checked and when |
In-house crawling vs an external service: who runs the decision matrix
An in-house team can technically build a crawler, but rarely has a systematic process for checking every new source — robots.txt, Terms and Conditions, GDPR, and copyright, all four, every time. A custom crawling service treats this decision matrix as a standard part of the project, not an optional check added after the script is already running in production.
Frequently asked questions about legal crawling in Romania
If a site has no robots.txt, can I extract anything from it?
No. The absence of a robots.txt file only means the site hasn't published explicit access restrictions for robots; it doesn't remove the need to check the Terms and Conditions, GDPR obligations for any personal data found on the site, or copyright protection for original content.
Can I extract competitor prices without breaking the law?
Usually yes, if robots.txt allows access to the product pages and the Terms and Conditions don't explicitly forbid automated extraction. Price, by itself, is not personal data.
What if the data I want to extract also contains people's names?
Explicitly exclude those fields from the crawling script or anonymize them before storage. The practical recommendation is to filter personal data at the source, not collect it and protect it afterward.
Can ANSPDCP fine a company for data collected through crawling?
Yes, if a complaint or an audit shows that personal data was processed without a legal basis, ANSPDCP can investigate and apply sanctions, under its authority to enforce GDPR at the national level.
Is respecting robots.txt enough to be legally safe?
No. Robots.txt only confirms allowed technical access. Full legal compliance also depends on the site's Terms and Conditions, correctly handling personal data under GDPR, and, for original content, respecting copyright.
Conclusion: check all three filters, not just one
A legal crawling project in Romania isn't reduced to a single file or a single rule — it's the intersection of the technical access allowed by robots.txt, the permissions in the target site's Terms and Conditions, and the correct handling of personal data under GDPR, with extra attention to copyright for original content. The decision matrix above shows you quickly where each data category you want to extract stands.
See the full web data extraction service, with legal review included from the planning stage, or discuss your project with the HappyWeb team for an evaluation applied exactly to your data source.
Sources
- Google Search Central — Introduction to robots.txt: developers.google.com/search/docs/crawling-indexing/robots/intro.
- GDPR.eu — What is GDPR: gdpr.eu/what-is-gdpr.
- ANSPDCP — Romanian National Supervisory Authority for Personal Data Processing: dataprotection.ro.
- Law 8/1996 on copyright and related rights (consolidated text, Legislatie.just.ro): legislatie.just.ro.
- HappyWeb's web data extraction (crawling) service: happyweb.ro/servicii/servicii-servicii-preluare-date-online-crawling.
Note: the legal information in this article is indicative and does not constitute legal advice. For crawling projects involving large data volumes or sensitive sources, we recommend checking the target site's Terms and Conditions directly and consulting a legal specialist or a Data Protection Officer (DPO).
Image generated with AI, used for illustrative purposes.
Write a comment