Learning with Hard Constraints

摘要

A learning paradigm is proposed, in which one has both classical supervised examples and constraints that cannot be violated, called here "hard constraints", such as those enforcing the probabilistic normalization of a density function or imposing coherent decisions of the classifiers acting on different views of the same pattern. In contrast, supervised examples can be violated at the cost of some penalization (quantified by the choice of a suitable loss function) and so play the roles of "soft constraints". Constrained variational calculus is exploited to derive a representation theorem which provides a description of the "optimal body of the agent", i.e. the functional structure of the solution to the proposed learning problem. It is shown that the solution can be represented in terms of a set of "support constraints", thus extending the well-known notion of "support vectors".