Spiders, worms and copyright: Using the Robots.txt and Robots Meta tags

摘要

The enormous number of websites on the internet now make searching them all extremely challenging. The software robot (which can also be known as a crawler, spider, wanderer, web agent, webbot, web robot, or worm) was developed as a way of meeting thechallenges, and is now widely used. Search engines such as Google and Yahoo are perhaps the most well-known examples of the use of such robots to expedite searching. While they certainly assist with scanning the ever-growing volume of websites, they also bring problems, some of which are of a legal nature, and therefore pose risks. What effect do these new devices have on the law relating to solicited and unsolicited access, and permissions to copy? Research into legal cases by Chyan Yang and Hsien-Jyh Liao aimed to find out.