Don’t Stop at the Top: Using Certificate Transparency Logs to Extend Domain Lists for Web Security Studies

摘要

Comprehensive domain lists are a requirement for many Internet measurement studies. Currently, researchers rely on proprietary lists such as the Alexa top list. Recent research has identified many problems with the existing domain lists. Our work proposes Certificate Transparency (CT) logs as an alternative domain list source for use in internet measurement studies. We describe the process of deriving a domain list from available CT log servers and analyze the gathered domain list. Furthermore, we compare the CT domain list with existing domain top lists by scanning the gathered domains for various web application technologies. Our results indicate a high level of similarity between the lists, but also interesting differences.