Capturing user access patterns in the web for data mining

摘要

Existing methods for knowledge discovery in the Web are mostly server-oriented and approaches taken are affected by the use of proxy servers. As a result, it is difficult to capture individual Web user behavior from the current log mechanism. Asan effort to remedy this problem, we develop in this paper methods for design and implementation of an access pattern collection server to conduct data mining in the Web. We also devise an innovative method, called page conversion, which converts theoriginal Web pages to enciphered ones so that the devised data collection mechanism will not be deliberately bypassed. With the concept of page conversion, the methods we proposed involves a mechanism of software downloading to resolve the difficultyimposed by proxy servers and to effectively capture the Web user behavior. Using the devised mechanism, traversal patterns are generated and compared to those produced by the ordinary Web servers to validate our results. It is shown that the traversalpatterns resulting from the devised system are not only more informative but also more accurate than those generated by ordinary Web servers, showing the importance and the usefulness of the mechanism devised.