Google will stop taking into account the noindex, the nofollow and the crawl-delay in the robots.txt files. Google wants an official standard for the robots.txt file interpretation protocol , used by webmasters to set the Philippines WhatsApp Number List rules followed by crawlers (REP, Robots Exclusion Protocol). Google has also decided to make the C ++ library used to analyze robots.txt files open source. In parallel, Google announces the end of the interpretation , by its bots, of several statements used by certain webmasters – in particular crawl-delay, noindex and nofollow. Google specifies that it has analyzed the robots.txt offered by the sites, and notices that certain declarations do not correspond to the standard imagined by Martijn Koster in 1994. The crawl-delay, nofollow and noindex rules have not been applied either. documented by Google.

So few webmasters use them, but there are – and they will have to change their practices by the end of August 2019. How to avoid indexing web pages on Google Indeed, Google will no longer interpret the rules not defined in the Robots Exclusion Protocol from September 1. This concerns in particular crawl-delay, nofollow and noindex. Google recalls that other means allow webmasters to declare a page or a directory in noindex, to refuse indexing: Add the noindex in the meta tags of the page (best solution) HTTP codes 404 and 410 mean that the page does not exist, Google crawlers understand this and remove these pages from its index. Pages placed behind a paywall – and those that require a password – are “usually” deleted from Google’s index. Google only indexes the pages that it knows

The Context: The Standardization Of The Robots.txt Interpretation Protocol

Putting a page or a directory in disallow via robots.txt therefore remains a possibility. Google will still be able to index a page if it finds a link to this page, but specifies that these disallow pages via robots.txt will be less visible on Google in the future. You can also use Search Console to remove a URL that has nothing to do with Google. For the CNIL, “online advertising targeting is a priority subject”: the action plan to regulate the sector has just been unveiled. Last year, 21% of complaints received by the CNIL concerned marketing. It has therefore decided to make the regulation of advertising targeting its priority for 2019. But before unveiling its action plan, the CNIL clarified two important points for marketing professionals and publishers.


Commercial prospecting: the compliance period is over Regarding commercial prospecting (or “opt in partner”), the CNIL recalls that the applicable rules have been clearly explained in the past. “[The CNIL] has left players with a compliance period of 6 months, which has now expired” . Companies that do not comply with these rules can therefore be penalized. Remember that they differ slightly between BtoC (for individuals) and BtoB (for professionals), but that BtoB commercial prospecting is also regulated. We sometimes hear that professional emails can be collected and used freely, this is not true. BtoC: explicit consent is required. Email advertising is subject to the explicit consent of the data subjects when the address is collected. Exceptions: customers, for similar products or services; non-commercial prospecting (charity, for example).

Google Will Stop Interpreting Certain Declarations In Robots.txt

Prior information is necessary. You must inform professionals, at the time of collection, of the use of their email address for prospecting purposes – and like individuals, they must be able to object easily. The prospecting must be linked to the position of the person contacted. No, the scroll is not “a valid expression of consent”, but … Another very interesting subject: what is explicit consent? In this regard, the CNIL recalls that the application of the RGPD, reinforced by the guidelines of the European Data Protection Committee on consent, “[excludes] explicitly that the continued navigation on a site may constitute a valid expression of consent ” . In other words: no, scrolling is not valid consent, just like moving your mouse or clicking on a link. The fact that a user scrolls a web page has no relation to his explicit consent

It is not a sufficient reason to place cookies on his computer or mobile. You must wait for explicit consent to place these cookies. Despite this very clear reminder, the CNIL will consider that the scroll is an “acceptable” mode of expression of consent until the end of 2020. We will see this in the details of the action plan. Let us recall all the same that the RGPD has been applied since May 25, 2018. For more than two years, site editors will therefore be able to use a non-compliant practice, despite the very clear interpretation of the supervisory body on this subject. The CNIL presents a 2019-2020 action plan to support stakeholders Beyond these reminders, the CNIL therefore presented its action plan to ensure the compliance of players who develop or integrate targeting solutions. New guidelines in July 2019 The CNIL will repeal its 2013 cookie recommendation,

Leave a Reply

Your email address will not be published.