Operators of public-facing websites are typically concerned about the unauthorized, technology-based extraction of large volumes of information from their sites, often by competitors or others in related businesses.  The practice, usually referred to as screen scraping, web harvesting, crawling or spidering, has been the subject of many questions and a fair amount of litigation over the last decade.

However, despite the litigation in this area, the state of the law on this issue remains somewhat unsettled: neither scrapers looking to access data on public-facing websites nor website operators seeking remedies against scrapers that violate their posted terms of use have very concrete answers as to what is permissible and what is not.

In the latest scraping dispute, the e-commerce site QVC objected to the Pinterest-like shopping aggregator Resultly’s scraping of QVC’s site for real-time pricing data.  In its complaint, QVC claimed that Resultly “excessively crawled” QVC’s retail site (purpotedly sending search requests to QVC’s website at rates ranging from 200-300 requests per minute to up to 36,000 requests per minute) causing a crash that wasn’t resolved for two days, resulting in lost sales.  (See QVC Inc. v. Resultly LLC, No. 14-06714 (E.D. Pa. filed Nov. 24, 2014)). The complaint alleges that the defendant disguised its web crawler to mask its source IP address and thus prevented QVC technicians from identifying the source of the requests and quickly repairing the problem.  QVC brought some of the causes of action often alleged in this type of case, including violations of the Computer Fraud and Abuse Act (CFAA), breach of contract (QVC’s website terms of use), unjust enrichment, tortious interference with prospective economic advantage, conversion and negligence and breach of contract.  Of these and other causes of action typically alleged in these situations, the breach of contract claim is often the clearest source of a remedy.

This case is a particularly interesting scraping case because QVC is seeking damages for the unavailability of their website, which QVC alleges to have been caused by Resultly.  This is an unusual theory of recovery in these types of cases.   For example,  this past summer, LinkedIn settled a scraping dispute with Robocog, the operator of HiringSolved, a “people aggregator” employee recruiting service, over claims that the service employed bots to register false accounts in order to scrape LinkedIn member profile data and thereafter post it to  its service without authorization from Linkedin or its members.  LinkedIn brought various claims under the DMCA and the CFAA, as well as state law claims of trespass and breach of contract, but did not allege that their service was unavailable due to the defendant’s activities.  The parties settled the matter, with Robocog agreeing to pay $40,000, cease crawling LinkedIn’s site and destroy all LinkedIn member data it had collected.  (LinkedIn Corp. v. Robocog Inc., No. 14-00068 (N.D. Cal.  Proposed Final Judgment filed July 11, 2014).

However, in one of the early, yet still leading cases on scraping, eBay, Inc. v. Bidder’s Edge, Inc., 100 F. Supp. 2d 1058 (N.D. Cal. 2000), the district court touched on the foreseeable harm that could result from screen scraping activities, at least when taken in the aggregate.  In the case, the defendant Bidder’s Edge operated an auction aggregation site and accessed eBay’s site about 100,000 times per day, accounting for between 1 and 2 percent of the information requests received by eBay and a slightly smaller percentage of the data transferred by eBay. The court rejected eBay’s claim that it was entitled to injunctive relief because of the defendant’s unauthorized presence alone, or because of the incremental cost the defendant had imposed on operation of the eBay site, but found sufficient proof of threatened harm in the potential for others to imitate the defendant’s activity.

It remains to be seen if the parties will reach a resolution or whether the court will have a chance to interpret QVC’s claims, and whether QVC can provide sufficient evidence of the causation between Resultly’s activities and the website outage.

Companies concerned about scraping should make sure that their website terms of use are clear about what is and isn’t permitted, and that the terms are positioned on the site to support their enforceability. In addition, website owners should ensure they are using “robots.txt,” crawl delays and other technical means to communicate their intentions regarding scraping.  Companies that are interested in scraping should evaluate the terms at issue and other circumstances to understand the limitations in this area.

UPDATE: Subsequently, QVC brought a motion for a preliminary injunction to enjoin Resultly from selling or other distributing any of its non-cash assets during the pendency of the litigation, based upon Resultly’s purported liability under the CFAA for intentionally causing damage to QVC’s servers when it crawled QVC’s website.  Resultly argued that it has not violated the CFAA in this way because it did not “intentionally cause damage” to QVC’s server.  In March 2015, the court denied the motion for a preliminary injunction, finding Resultly, a non-competitor, lacked intent to cause damage to QVC’s servers or take down its site and that Resultly’s crawl rate was implemented according to procedures that had been in place for a period of time and had never caused a problem (and, according to the court, QVC could have implemented a crawl delay for unknown bots while still allowing the programs it wanted to crawl its site to do so at higher speeds, thus Resulty’s crawl rate did not evidence any intent to harm).  (QVC, Inc. v. Resultly, LLC, 99 F.Supp.3d 525 (E.D. Pa. 2015)).  Moreover, the court found that QVC failed to show a likelihood of irreparable harm because, among other things, evidence showed QVC’s ability to protect itself against any future outages caused by unknown bots.

In August 2015, QVC filed an amended complaint claiming, among other things, that VigLink, Inc., an approved participant in QVC’s affiliate marketing program, allegedly sublicensed (contrary to its agreement with QVC) its rights under the program to Resultly.

In February 2016, in ruling on the defendants’ motion to dismiss, the court allowed some contract and CFAA claims to go forward, while dismissing a number of state law claims, including conversion and negligence claims (on the latter claim, the court found Resultly and other publishers did not owe a legal duty of care to the companies whose websites they crawl). (QVC, Inc. v. Resultly, LLC, 159 F.Supp.3d 576 (E.D. Pa. 2016)).  While the court dismissed claims against Resultly for breach of the QVC Publisher Agreement because it was not a party to the contract, it allowed breach of contract claims against Viglink, which was an approved participant in QVC’s affiliate marketing program and allegedly allowed Resultly to crawl QVC to earn commissions from Viglink (the QVC Publisher Agreement stated VigLink was “responsible for all usage…through its account” and prohibited sublicenses). (In 2016, QVC and Viglink settled).  The court also allowed QVC’s CFAA claim against Resultly for accessing QVC’s networks “without authorization” – the court stated that while QVC did not expressly revoke QVC’s permission to crawl its site by sending a cease and desist letter, QVC made plausible allegations that the QVC Publisher Agreement prohibits web crawling and that Resultly was arguably alerted to that prohibition by virtue of being a sub-publisher of Viglink, which was required to comply with such Agreement.  As the decision was made on a motion to dismiss, the court was obliged to draw inferences in the light most favorable to the plaintiff, though one might wonder whether this decision on the CFAA claim might have been different had it been decided after the Power Ventures and related decisions of the Ninth Circuit.

The matter was settled in 2017.