web crawler
Americannoun
Other Word Forms
Derived Forms
Etymology
Origin of web crawler
First recorded in 1990–95
Example Sentences
Examples are provided to illustrate real-world usage of words in context. Any opinions expressed do not reflect the views of Dictionary.com.
He went on to explain how one could use a Web crawler to capture masses of data and mark voters as having been contacted by the campaign in order to alter its metrics.
From The New Yorker • Oct. 1, 2019
When their Web crawler crashed, there was no informative diagnostic message—just the phrase “Whoa, horsey!”
From The New Yorker • Dec. 3, 2018
Yet Snowden himself reduced the complicated social question of when and how the disclosure of sensitive information strengthens democratic government to a technical one: how many classified files can I touch with my Web crawler?
From The New Yorker • Dec. 11, 2016
The Wayback Machine collects every Web page it can find, unless that page is blocked; blocking a Web crawler requires adding only a simple text file, “robots.txt,” to the root of a Web site.
From The New Yorker • Jan. 19, 2015
Definitions and idiom definitions from Dictionary.com Unabridged, based on the Random House Unabridged Dictionary, © Random House, Inc. 2023
Idioms from The American Heritage® Idioms Dictionary copyright © 2002, 2001, 1995 by Houghton Mifflin Harcourt Publishing Company. Published by Houghton Mifflin Harcourt Publishing Company.
