The Internet literature frequently refers to the problem of information overload and to difficulties in finding accurate and pertinent information. This has created a demand for information filtering systems that are meant to deliver personalized information by building and applying profiles based on the user's information preferences.; Two issues in profile acquisition were investigated: modes of profile acquisition and profile quality. In relation to profile acquisition, the aim was to test how much the automated processes could be improved by increasing human involvement. Eighteen subjects participated in an experiment conducted using SIFTER (Smart Information Filtering Technology for Electronic Resources), an existing filtering system that ranks incoming information based on profiles. For this study, profiles were based on topical classes in the consumer health domain. The relationship between different modes of user involvement (explicit, implicit feedback-based, and combined) and filtering performance was analyzed. The performance of the system was measured in terms of Normalized Precision, a ranking measure. Results suggested an advantage to providing explicit preferences. Also, the combined mode showed that introducing feedback for the acquisition of user profiles might have a benefit in the long term. Acquiring the profile based only on feedback resulted in the lowest filtering performance. Quality of the profile was understood as the ability to acquire and represent the user's information preferences. User's perception of profile quality was explored by testing (a) how did the profile acquired by the filtering system compare to the profile provided by the user and, (b) how conducive was user's feedback to the acquisition and representation of information preferences. Machine and user profiles were found significantly similar for 60% of the classes. Feedback assessments were more conducive to the representation of user's preferences when these preferences involved specific classes rather than general classes. Characteristics of the user's background, information needs and the document collection that influenced relevance feedback judgements were identified. These attributes could be included in the representation of documents and profiles as well as in the feedback mechanism to improve filtering performance.
展开▼
