standard for robot exclusion

Computing Dictionary

standard for robot exclusion definition

World-Wide Web
A proposal to try to prevent the havoc wreaked by many of the early World-Wide Web robots when they retrieved documents too rapidly or retrieved documents that had side effects (such as voting). The proposed standard for robot exclusion offers a solution to these problems in the form of a file called "robots.txt" placed in the document root of the web site.
W3C standard (http://w3.org/TR/html4/appendix/notes.html#h-B.4.1.1).
(2006-10-17)
The Free On-line Dictionary of Computing, © Denis Howe 2010 http://foldoc.org
Cite This Source
Explore Dictionary.com
More from Thesaurus.com
Synonyms and Antonyms for standard for robot exclusion
More from Reference.com
Search for articles containing standard for robot exclusion
Dictionary.com Word FAQs

Dictionary.com presents 366 FAQs, incorporating some of the frequently asked questions from the past with newer queries.

Copyright © 2014 Dictionary.com, LLC. All rights reserved.
  • Please Login or Sign Up to use the Recent Searches feature