An Evaluation Metric for Content Providing Models, Recommendation Systems, and Online Campaigns

摘要

Creating an optimal digital experience for users require providing users desirable content and also delivering these contents in optimal time as user's experience and interaction taking place. There are multiple metrics and variables that may determine the success of a "user digital experience". These metrics may include accuracy, computational cost and other variables. Many of these variables may be contradictory to one another (as explained later in this submission) and their importance may depend on the specific application the digital experience optimization may be pursuing. To deal with this intertwined, possibly contradicting and confusing set of metrics, this work introduces a generalized index entailing all possible metrics and variables - that may be significant in defining a successful "digital experience design model". Besides its generalizability, as it may include any metric the marketers or scientists consider to be important, this new index allows the marketers or the scientists to give different weights to the corresponding metrics as the significance of a specific metric may depends on the specific application. This index is very flexible and could be adjusted as the objective of' user digital experience optimization" may change. Here, we use "recommendation" as equivalent to "content providing" throughout the submission. One well known usage of "recommender systems" is in providing contents such as products, ads, goods, network connections, services, and so on. Recommender systems have other wide and broad applications and - in general - many problems and applications in Al and machine learning could be converted easily to an equivalent "recommender system" one. This feature increases the significance of recommender systems as an important application of AI and machine learning. The introduction of internet has brought a new dimension on the ways businesses sell their products and interact with their customers. Ubiquity of the web and consequently web applications are soaring and as a result much of the commerce and customer experience are taking place on line. Many companies offer their products exclusively or predominantly online. At the same time, many present and potential customers spend much time on line and thus businesses try to use efficient models to interact with online users and engage them in various desired initiatives. This interaction with online users is crucial for businesses that hope to see some desired outcome such as purchase, conversions of any types, simple page views, spending longer time on the business pages and so on. Recommendation system is one of the main tools to achieve these outcomes. The basic idea of recommender systems is to analyze what is the probability of a desires action by a specific user. Then, by knowing this probability, one can make decision of what initiatives to be taken to maximize the desirable outcomes of the online user's actions. The types of initiatives could include, promotional initiatives (sending coupons, cash, ...) or communication with the customer using all available media venues such as mail, email, online ad, etc. the main goal of recommendation or targeting model is to increase some outcomes such as "conversion rate", "length of stay on sites", "number of views" and so on. There are many other direct or indirect metrics influenced by recommender systems. Examples of these could include an increase of the sale of other products which were not the direct goal of the recommendations, an increase the chance of customer coming back at the site, increase in brand awareness and the chance of retargeting the same user at a later time.