Relative Entropy
المؤلف:
Cover, T. M. and Thomas, J
المصدر:
Elements of Information Theory. New York: Wiley, 1991.
الجزء والصفحة:
...
14-11-2021
1679
Relative Entropy
Let a discrete distribution have probability function 
, and let a second discrete distribution have probability function 
. Then the relative entropy of 
with respect to 
, also called the Kullback-Leibler distance, is defined by
Although 
, so relative entropy is therefore not a true metric, it satisfies many important mathematical properties. For example, it is a convex function of 
, is always nonnegative, and equals zero only if 
.
Relative entropy is a very important concept in quantum information theory, as well as statistical mechanics (Qian 2000).
REFERENCES:
Cover, T. M. and Thomas, J. A. Elements of Information Theory. New York: Wiley, 1991.
Qian, H. "Relative Entropy: Free Energy Associated with Equilibrium Fluctuations and Nonequilibrium Deviations." 8 Jul 2000. http://arxiv.org/abs/math-ph/0007010.
الاكثر قراءة في الرياضيات التطبيقية
اخر الاخبار
اخبار العتبة العباسية المقدسة
