Tutorial: Legality and Ethics of Web Scraping

摘要

Researchers and practitioners often use various tools and technologies to automatically retrieve data from the Web (often referred to as Web scraping) when conducting their projects. Unfortunately, they often overlook the legality and ethics of using these tools to collect data. Failure to pay due attention to these aspects of Web Scraping can result in serious ethical controversies and lawsuits. Accordingly, we review legal literature together with the literature on ethics and privacy to identify broad areas of concern together with a list of specific questions that researchers and practitioners engaged in Web scraping need to address. Reflecting on these questions and concerns can potentially help researchers and practitioners decrease the likelihood of ethical and legal controversies in their work.