A Preferred Definition of Conditional Rényi Entropy

Subject Areas : Applied Mathematics

Leila Golshani ^{1 *}

1 - Department of Mathematics, Central Tehran Branch, Islamic Azad University, Tehran, Iran

Received: 2018-02-17 Accepted : 2018-07-12 Published : 2018-12-01

Keywords: Conditional entropy, Tsallis entropy, R&eacute, nyi entropy, Shannon Entropy,

Abstract :

The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced and its properties were shown. But no specific definition has been given for the conditional Rényi entropy. Several authors have used some definitions of the conditional Rényi entropy, to find their properties and relations among them, but there is no general agreement on any specific definition In this paper, we focus on the definitions of the conditional Rényi entropy, and select one of them on the basis of a relation between Rényi and Tsallis entropies, and show that the chain rule holds generally for the case of conditional Rényi entropy. Then, using this definition, we show some of the properties of conditional Rényi entropy. Finally, we show the relations among Rényi, Shannon and Tsallis entropies.