An extended Fano's inequality for the finite blocklength coding

摘要

Fano's inequality reveals the relation between the conditional entropy and the probability of error. It has been the key tool in proving the converse of coding theorems in the past sixty years. In this paper, an extended Fano's inequality is proposed, which is tighter and more applicable for codings in the finite blocklength regime. Lower bounds on the mutual information and an upper bound on the codebook size are also given, which are shown to be tighter than the original Fano's inequality. Especially, the extended Fano's inequality is tight for some symmetric channels such as the q-ary symmetric channels (QSC).