web crawler

or web·crawl·er

[web-kraw-ler]

noun Digital Technology.

(sometimes initial capital letter) a computer program that retrieves data from a website, as in order to index web pages for a search engine.

Compare bot1.

Origin of web crawler

First recorded in 1990–95
Also called spider.
Related formsweb crawling, noun
Dictionary.com Unabridged Based on the Random House Unabridged Dictionary, © Random House, Inc. 2019