web scraping

Subscribe to web scraping via RSS

On May 9, 2024, a California district court dismissed, with leave to amend, the complaint brought by social media platform X Corp. (formerly Twitter) against data provider Bright Data Ltd. (“Bright Data”) over Bright Data’s alleged scraping of publicly available data from X for use in data products sold…

Late last year, Chegg Inc. (“Chegg”), an online learning platform, obtained a preliminary injunction based on allegations that the various operators of the Homeworkify website (“Defendants”) – which allows users to view Chegg’s paywalled solutions without creating an account – violated the Computer Fraud and Abuse Act (CFAA). (Chegg …

On January 23, 2024, a California district court released its opinion in a closely-watched scraping dispute between the social media platform Meta and data provider Bright Data Ltd. (“Bright Data”) over Bright Data’s alleged scraping of publicly-available data from Facebook and Instagram for use in data products sold to third…

  • Flight and travel data has always been valuable for data aggregators and online travel services and has prompted litigation over the years.
  • Latest suit from Air Canada against a rewards travel search site raises some interesting liability issues under the CFAA.
  • The implications of this case, if the plaintiffs are successful, could impact the legal analysis of web scraping in a variety of circumstances, including for the training of generative AI models.

In a recent post, we recounted the myriad of issues raised by recently-filed data scraping suits involving job listings, company reviews and employment data.  Soon after, another interesting scraping suit was filed, this time by a major airline against an award travel search site that aggregates fare and award travel data.  Air Canada alleges that Defendant Localhost LLC (“Localhost” or “Defendant”), operator of the Seats.aero website, unlawfully bypassed technical measures and violated Air Canada’s website terms when it scraped “vast amounts” of flight data without permission and purportedly caused slowdowns to Air Canada’s site and other problems. (Air Canada v. Localhost LLC, No. 23-01177 (D. Del. Filed Oct. 19, 2023)).[1]   

The complaint alleges that Localhost harvested data from Air Canada’s site and systems to populate the seats.aero site, which claims to be “the fastest search engine for award travel.” 

It also alleged that in addition to scraping the Air Canada website, Localhost engaged in “API scraping” by impersonating authorized requests to Air Canada’s application programming interface.  

UPDATE: On February 5, 2024, the California district court granted the defendant Aspen Technology Labs, Inc.’s motion to dismiss Jobiak LLC’s web scraping complaint for lack of personal jurisdiction, with leave to amend. The court found that Jobiak had not adequately alleged that its copyright and tort-related claims arose out of the defendant’s forum-related activities and that there were no allegations that Jobiak’s database or website was hosted on servers in the California forum.  On March 8, 2024, the court dismissed the action with prejudice, as Jobiak did not submit an amended complaint within the time allowed by the court.  

In recent years there has been a great demand for information about job listings, company reviews and employment data.   Recruiters, consultants, analysts and employment-related service providers, amongst others, are aggressively scraping job-posting sites to extract that type of information. Recall, for example, the long-running, landmark hiQ scraping litigation over the scraping of public LinkedIn data.

The two most recent disputes regarding scraping of employment and job-related data were brought by Jobiak LLC (“Jobiak”), an AI-based recruitment platform.  Jobiak filed two nearly-identical scraping suits in California district court alleging that competitors unlawfully scraped its database and copied its optimized job listings without authorization. (Jobiak LLC v. Botmakers LLC, No. 23-08604 (C.D. Cal. Filed Oct. 12, 2023); Jobiak LLC v. Aspen Technology Labs, Inc., No. 23-08728 (C.D. Cal. Filed Oct. 17, 2023)).

On October 24, 2022, a Delaware district court held that certain claims under the Computer Fraud and Abuse Act (CFAA) relating to the controversial practice of web scraping were sufficient to survive the defendant’s motion to dismiss. (Ryanair DAC v. Booking Holdings Inc., No. 20-01191 (D. Del. Oct. 24, 2022)). The opinion potentially breathes life into the use of the CFAA to combat unwanted scraping.

In the case, Ryanair DAC (“Ryanair”), a European low-fare airline, brought various claims against Booking Holdings Inc. (and its well-known suite of online travel and hotel booking websites) (collectively, “Defendants”) for allegedly scraping the ticketing portion of the Ryanair site. Ryanair asserted that the ticketing portion of the site is only accessible to logged-in users and therefore the data on the site is not public data.

The decision is important as it offers answers (at least from one district court) to several unsettled legal issues about the scope of CFAA liability related to screen scraping. In particular, the decision addresses:

  • the potential for vicarious liability under the CFAA (which is important as many entities retain third party service providers to perform scraping)
  • how a data scraper’s use of evasive measures (e.g., spoofed email addresses, rotating IP addresses) may be considered under a CFAA claim centered on an “intent to defraud”
  • clarification as to the potential role of technical website-access limitations in analyzing CFAA “unauthorized access” liability

To find answers to these questions, the court’s opinion distills the holdings of two important CFAA rulings from this year – the Supreme Court’s holding in Van Buren that adopted a narrow interpretation of “exceeds unauthorized access” under the CFAA and the Ninth Circuit’s holding in the screen scraping hiQ case where that court found that the concept of “without authorization” under the CFAA does not apply to “public” websites.

On May 19, 2022, the Department of Justice (DOJ) announced that it had revised its policy regarding prosecution under the federal anti-hacking statute, the Computer Fraud and Abuse Act (CFAA). Since the DOJ last made changes to its CFAA policy in 2014, there have been a number of relevant developments in technology and business practices, most notably related to web scraping.  Among other things, the revised policy reflects aspects of the evolving views of this sometimes-controversial statute and the outcome of two major CFAA court decisions in the last year (the Ninth Circuit hiQ decision and the Supreme Court’s Van Buren decision), both of which adopted a narrow interpretation of the CFAA in situations beyond a traditional outside computer hacker scenario.

While the DOJ’s revised CFAA policy is only binding on federal CFAA criminal prosecution decisions (and could be amended by subsequent Administrations) and does not directly affect state prosecutions (including under the many state versions of the CFAA) or civil litigation in the area, it is likely to be relevant and influential in those situations as well, and in particular, with respect to web scraping. It seems that even the DOJ has conceded that the big hiQ and Van Buren court decisions have mostly (but not entirely) eliminated the threat of criminal prosecution under the CFAA when it comes to the scraping of “public” data. Still, as described below, the DOJ’s revisions to its policy, as written, are not entirely consistent with the hiQ decision.

UPDATE: On December 23, 2021, the parties reached a settlement, as Southwest filed an unopposed motion for entry of final judgment and a permanent injunction containing the same restrictions as the temporary injunction issued in September. Under the proposed permanent injunction, Kiwi would be barred from scraping flight and fare information from Southwest’s site, publishing any Southwest flight or fare information on kiwi’s site or app (or selling any Southwest flights), or otherwise using Southwest’s site for any commercial purpose or in a manner that violates Southwest’s site terms.

UPDATE: On November 1, 2021, the parties filed a Joint Notice of Settlement indicating that they have reached a settlement agreement in principle.  The terms of the settlement were not disclosed.

UPDATE: On October 28, 2021, the defendant Kiwi.com, Inc. filed a notice of appeal to the Fifth Circuit seeking review of the district court’s ruling granting Southwest Airlines Co.’s motion for a preliminary injunction.

On September 30, 2021, a Texas district court granted Southwest Airline Co.’s (“Southwest”) request for a preliminary injunction against online travel site Kiwi.com, Inc. (“Kiwi”), barring Kiwi from, among other things, scraping fare data from Southwest’s website and committing other acts that violate Southwest’s terms. (Southwest Airlines Co. v. Kiwi.com, Inc., No. 21-00098 (N.D. Tex. Sept. 30, 2021)). Southwest is no stranger in seeking and, in most cases, obtaining injunctive relief against businesses that have harvested its fare data without authorization – ranging as far back as the 2000s (See e.g., Southwest Airlines Co. v. BoardFirst, LLC, No. 06-0891 (N.D. Tex. Sept. 12, 2007) (a case cited in the current court opinion)), and as recently as two years ago, when we wrote about a 2019 settlement Southwest entered into with an online entity that scraped Southwest’s site and had offered a fare notification service, all contrary to Southwest’s terms.

In this case, the Texas court found that Southwest had established a likelihood of success on the merits of its breach of contract claim. Rejecting Kiwi’s arguments that it did not assent to Southwest’s terms, the court found that Kiwi had knowledge of and assented to the terms in multiple ways, including by agreeing to the terms when purchasing tickets on Southwest’s site. In all, the court found the existence of a valid contract and Kiwi’s likely breach of the terms, which prohibit scraping Southwest’s flight data and selling Southwest flights without authorization. The court also found that Southwest made a sufficient showing that Kiwi’s scraping and unauthorized sale of tickets, if not barred, would result in irreparable harm. In ultimately granting Southwest’s request for a preliminary injunction, the Texas court also found that Southwest also demonstrated the threatened injury if the injunction is denied outweighed any harm to Kiwi that will result if the injunction is granted and that the injunction would be in the public interest.

What made this result particularly notable is that the preliminary injunction is based on the likelihood of success on the merits of Southwest’s breach of contract claim and Kiwi’s alleged violation of Southwest’s site terms, as opposed to other recent scraping disputes which have centered around claims of unauthorized access under the federal Computer Fraud and Abuse Act (CFAA).