web crawler

[ web-kraw-ler ]

/ ˈwɛbˌkrɔ lər /

Save This Word!

noun Digital Technology.

a computer program that digitally surveys websites, as in order to index web pages for a search engine.

QUIZ

QUIZ YOURSELF ON AFFECT VS. EFFECT!

In effect, this quiz will prove whether or not you have the skills to know the difference between “affect” and “effect.”

Question 1 of 7

The rainy weather could not ________ my elated spirits on my graduation day.

Meet Grammar Coach

Sometimes Web crawl·er .

Also called crawl·er .

Compare bot¹, spider.

Origin of web crawler

First recorded in 1990–95

web crawling, noun

He was referring to web censorship behind the Great Firewall.
China’s Internet Is Freer Than You Think|Brendon Hong|December 27, 2014|DAILY BEAST
With its vast web of resources and services, including its support groups, the Center has often helped save these people lives.
The LGBT Center That Changed Our Lives|Justin Jones|December 22, 2014|DAILY BEAST
How has your experience been shaped by digital encounters and the social web?
The Unbearable Whiteness of Protesting|Rawiya Kameir, Judnick Mayard|December 10, 2014|DAILY BEAST
The caller mentioned my work, which focused primarily on consumer products, mobile apps, emerging start-ups, and web trends.
A Female Writer’s New Milestone: Her First Death Threat|Annie Gaus|December 1, 2014|DAILY BEAST

That leads us to wonder why galaxies and their black holes somehow “know” where they are in the cosmic web.
The Black Hole Tango|Matthew R. Francis|November 24, 2014|DAILY BEAST
It was no wonder that he felt quite at home in the duck-pond, which was made for web-footed folk.
The Tale of Grandfather Mole|Arthur Scott Bailey
"It is n't distressingly calm now," said the extra-strong frames—they were called web-frames—in the engine-room.
Kipling Stories and Poems Every Child Should Know, Book II|Rudyard Kipling
The touch of the soft fabric reassured him: it was as soft as though woven of spider's web, and strong as fibres of steel.
Astounding Stories, May, 1931|Various
These threads in the web of industry, which had shone that day for the first time, were the lives of two little children.
The Underworld|James C. Welsh
At first girders had solid or plate webs, but for spans over 100 ft. the web always now consists of bracing bars.
Encyclopaedia Britannica, 11th Edition, Volume 4, Part 3|Various