web crawler

noun Digital Technology.
a computer program that automatically retrieves Web pages for use by search engines. Compare bot3.
Also, Web crawler, webcrawler.
Also called spider.
Dictionary.com Unabridged
Based on the Random House Dictionary, © Random House, Inc. 2014.
Cite This Source Link To webcrawler
Computing Dictionary

WebCrawler definition

World-Wide Web
A free World-Wide Web search engine developed by Brian Pinkerton at the University of Washington and now moved to America Online, Inc. WebCrawler collects URLs by searching the Internet and allows users to perform keyword searches through a World-Wide Web browser.
The Free On-line Dictionary of Computing, © Denis Howe 2010 http://foldoc.org
Cite This Source
Copyright © 2014 Dictionary.com, LLC. All rights reserved.
  • Please Login or Sign Up to use the Recent Searches feature