Diversity on the World Wide Web: Using Robots to Search the Web

摘要

The growing popularity of the World Wide Web (WWW) has resulted in an almost exponential growth in the number of resources available via this medium. To access these resources, Web users should familiarize themselves with the variety of search options available to them, particularly with search engines developed specifically for the Web. These search engines, often referred to as "spiders" or "robots," offer an advantage over the more common list indexes, which are often manually maintained. Understanding the basic operations and qualities of these robots will aid Web users on their quest for WWW sources.