Meta Just Deleted the Clause Letting It Scrape News Sites for AI

A wave of backlash from legal scholars and media observers finally forced Meta’s hand. According to reporting by The Verge, the tech giant quietly removed ambiguous clauses from its commercial terms page that many experts believed allowed the company to bypass site restrictions and harvest web data for AI training.

What the original terms changed for publishers

As legal and digital information experts like Cornell professor James Grimmelmann pointed out, the previous phrasing suggested Meta could pull content from any web domain to develop and train its AI products. Critics argued this wording effectively gave the company the legal right to override traditional blocks set up by publishers through a site's robots.txt file.

Meta initially pushed back against the criticism. Company representatives, including communications VP Andy Stone, insisted that web crawler operations had not changed and that claims to the contrary were false. However, mounting pressure resulted in a total rewrite of the policy language.

How the updated policy works in practice

In a follow-up statement to The Verge, Meta spokesperson Daniel Roberts clarified that the consolidated crawling terms were originally meant strictly for ads and business products. The new update explicitly separates those functions and removes the controversial wording entirely.

Respect for robots.txt: Meta officially reaffirmed that content owners retain the absolute right to use robots.txt files to signal their preferences.

No new harvesting rights: The updated terms do not grant Meta any special permissions to supersede webmaster blocking instructions.

Narrower scope: Commercial terms apply strictly to designated business tools rather than open-ended AI model harvesting.

Verdict: does this protect your content?

For creators and publishers worried about automated data collection, this retreat offers a temporary sigh of relief. While web scraping remains a broader industry challenge, Meta's move demonstrates that public pushback and expert scrutiny can successfully force big tech firms to clean up ambiguous legal language regarding intellectual property.