Bright Data Web Scraping Prevailing Against X's Efforts to Sue

A US District judge dismissed a lawsuit brought by Elon Musk's X Corp (formerly Twitter) against Israeli web scraping company Bright Data, marking a significant victory for Bright Data and setting a crucial precedent in the realm of web scraping and data accessibility.

The case, which was filed last July, saw X Corp accusing Bright Data of unlawfully copying and selling content from the social media platform formerly known as Twitter, as well as providing tools for others to do the same.

“Bright Data's victory over X makes it clear to the world that public information on the web belongs to all of us, and any attempt to deny the public access will fail," stated Bright Data's CEO, Or Lenchner. "As demonstrated in several recent cases including our win in the Meta case."

US District Judge William Alsup, presiding over the case in San Francisco, ruled that X Corp failed to substantiate its claims that Bright Data had violated its user agreement by facilitating scraping activities and circumventing X's anti-scraping measures. Judge Alsup emphasized that the use of scraping tools is not inherently fraudulent and cautioned against granting social media companies unchecked authority over public data usage, highlighting the potential for creating information monopolies that would not be in the public interest.

"What is happening now is unprecedented, and has profound implications in business, research, training of AI models, and beyond," Lenchner added.

Musk has been vocal about his battle against web scraping and his efforts to alter his platform's rules of engagement accordingly.

To address extreme levels of data scraping & system manipulation, we’ve applied the following temporary limits:

- Verified accounts are limited to reading 6000 posts/day
- Unverified accounts to 600 posts/day
- New unverified accounts to 300/day

— Elon Musk (@elonmusk) July 1, 2023

But the decision reinforces the principle that public data should remain accessible to everyone.

This ruling follows a string of legal battles involving Bright Data and other entities, including a recent case with Meta Platforms (formerly Facebook), where Bright Data was similarly vindicated. In that case, Meta had sued Bright Data for scraping data from Facebook and Instagram, but the court ruled in favor of Bright Data, stating that Meta had not provided sufficient evidence to support its claims of non-public data scraping.

The implications of these rulings may extend beyond the realm of business, affecting research, AI model training, and other fields that rely on web data. The New York Times is suing OpenAI and Microsoft for allegedly using millions of its articles to train chatbots that now compete with the news outlet. This lawsuit, filed in Federal District Court in Manhattan, accuses the companies of tortious web scraping and copyright infringement, and seeks "billions of dollars in statutory and actual damages." The Times had previously approached Microsoft and OpenAI in April to address its concerns but failed to reach a resolution.

The legal battle between Bright Data and X Corp is ongoing, with X Corp having the option to amend its complaint. However, the court's ruling sets a strong precedent that could have far-reaching effects on the future of web scraping and data accessibility, potentially affirming the legality and permissibility of openly accessible public information.