Domain knowledge integration in data mining for churn and customer lifetime value modelling: new approaches and applications

摘要

The evaluation of the relationship with the customer and related benefits has become akey point for a company’s competitive advantage. Consequently, interest in keyconcepts, such as customer lifetime value and churn has increased over the years.However, the complexity of building, interpreting and applying customer lifetime valueand churn models, creates obstacles for their implementation by companies. A proposedqualitative study demonstrates how companies implement and evaluate the importanceof these key concepts, including the use of data mining and domain knowledge,emphasising and justifying the need of more interpretable and acceptable models.Supporting the idea of generating acceptable models, one of the main contributions ofthis research is to show how domain knowledge can be integrated as part of the datamining process when predicting churn and customer lifetime value. This is donethrough, firstly, the evaluation of signs in regression models and secondly, the analysisof rules’ monotonicity in decision tables. Decision tables are used for contrastingextracted knowledge, in this case from a decision tree model. An algorithm is presented,which allows verification of whether the knowledge contained in a decision table is inaccordance with domain knowledge. In the case of churn, both approaches are appliedto two telecom data sets, in order to empirically demonstrate how domain knowledgecan facilitate the interpretability of results. In the case of customer lifetime value, bothapproaches are applied to a catalogue company data set, also demonstrating theinterpretability of results provided by the domain knowledge application. Finally, abacktesting framework is proposed for churn evaluation, enabling the validation andmonitoring process for the generated churn models.