PARTIAL INFORMATION AND PARTIAL WEIGHT - TWO NEW INFORMATION THEORETIC METRICS TO HELP SPECIFY A DATA-BASED NATURAL SYSTEM MODEL

摘要

How does one define or specify a system? This is a problem faced routinely in science and engineering, with solutions developed from our understanding of the processes inherent, to assessing the underlying structure based on observational evidence alone. In general, system specification involves identifying a few meaningful predictors (from a large enough set that is plausibly related to the response) and formulating a relation between them and the system response being modeled. For systems where physical relationships are less apparent, and sufficient observational records exist, a range of statistical alternatives have been investigated as a possible way of specifying the underlying form. Here, we present two new contributions that were recently published by Sharma and Mehrotra (2014) as a step towards an assumption free specification of a system using observational information alone. The first of these is the partial information (PI), a new means for specifying the system, its key advantage being the relative lack of major assumptions about the processes being modeled in order to characterize the complete system. The second is the concept of partial weights (PW) which use the identified predictors to formulate a predictive model that acknowledges the relative contributions varied predictor variables make to the prediction of the response. We assess the utility of the PI-PW framework using synthetically generated datasets from known linear, nonlinear and high-dimensional dynamic yet chaotic systems, and demonstrate the efficacy of the procedure in ascertaining the underlying true system with varying extents of observational evidence available. We highlight how this framework can be invaluable in formulating prediction models for natural systems which are modeled using empirical or semi-empirical alternatives, and discuss current limitations that still need to be overcome.