The practice of search engine crawling and caching of Web site content has infrequently been litigated. (The Perfect 10 case is a significant exception.) This may be because most Web site operators want their content to be indexed and available on search engines. Those Web site operators that do not want their content copied and indexed can stop crawling and caching by deploying a robots.txt file. By generally accepted convention, a robots.txt file is consulted by most search engine crawlers for instructions from a Web site operator as to whether, and to what extent, Web site content may be copied and indexed. The inclusion of the "noindex" metatag in a robots.txt file instructs a crawler that the content may not be copied and indexed.

In Parker v. Yahoo!, Inc., 2008 U.S. Dist. LEXIS 74512 (E.D. Pa. Sep. 26, 2008), the district court held that a Web site operator’s failure to deploy a robots.txt file containing instructions not to copy and cache Web site content gave rise to an implied license to index that site. The court did say, however, that the implied license in the case may have, at some point, been terminated by the operator.


The court found that plaintiff Parker’s acknowledgment that he deliberately chose not to deploy a robots.txt file on the Web site containing his work was conclusive on the issue of implied license. In so ruling the court followed Field v. Google, Inc., 412 F.Supp. 2d 1106 (D. Nev. 2006), a similar copyright infringement action brought by an author who failed to deploy a robots.txt file and whose works were copied and cached by the Google search engine:

The district court [in Field v. Google found] that the plaintiff’s conscious choice was "reasonably interpreted" by Google to be the grant of a license to Google for that use. Id. The court also noted that the opt-out meta-tag was a "well-known industry standard," and that it would be impossible for Google to personally contact every website owner to ascertain whether the owner wanted to have her pages listed in search results or be accessible through cached links. Id. at 1112, 1116. The court thus concluded that Google had sufficiently established the defense of implied license.

The Court is persuaded that Parker’s complaint conclusively establishes the affirmative defense of implied license. At the very least, paragraph 24 of his complaint suggests that Parker knew that as a result of his failure to abide by the search engines’ procedures, the search engines would display a copy of his works. From Parker’s silence and lack of earlier objection, the defendants could properly infer that Parker knew of and encouraged the search engines’ activity, and, as did the defendants in Field, they could reasonably interpret Parker’s conduct to be a grant of a license for that use.

The court further ruled, however, that a nonexclusive implied license, because it is not supported by consideration, may be terminated. Parker may have terminated the implied license by the institution of the litigation, the court concluded, and he alleged that the search engines failed to remove copies of his works from their Web sites even after the litigation was instituted. If proved, "the continued use over Parker’s objection might constitute direct infringement." That issue, which had not been briefed by the parties, was Left for resolution at a later time.

Another interesting point left unresolved was whether the existence of an implied license is a question of federal or state law. In the absence of an argument to the contrary, the court applied "federal common law." For further discussion of this question see Foad Consulting Group, Inc. v. Musil Govan Azzalino, 270 F.3d 821, 832 (9th Cir. 2001) (existence of implied copyright license is an issue of state law) and the concurring opinion of Judge Kozinski (some, but not all, implied copyright licenses are governed by state law). The issue is also touched on in a recent article, Afori, Implied License – An Emerging New Standard in Copyright Law, 25 Santa Clara Computer and High Technology Law Journal (2008)

By the way, this is not the first legal challenge brought by the plaintiff Web site operator, Gordon Roy Parker, against search engines. Parker has taken on the system with a series of pro se lawsuits against Google, Yahoo! And Microsoft, claiming infringement of his copyright-registered works by the companies’ respective Web crawlers and search engines.

Parker struck out with the first of these lawsuits, Parker v. Google, 422 F.Supp.2d 492 (E.D.Pa., Mar. 10, 2006) (dismissing complaint), aff’d, 242 Fed.Appx. 833 (3d Cir. 2007) (unpublished). The district court ruled, and the appeals court agreed, that the automatic excerpting of Web sites in search results did not constitute direct infringement because the activity lacked the "necessary volitional element to constitute direct copyright infringement." In the instant case, the court found that Parker was precluded from relitigating this issue.