On Defining Expressions for Entropy and Cross-Entropy The Entropic Transreals and Their Fracterm Calculus

Open Access
Authors
Publication date 01-2025
Journal Entropy
Article number 31
Volume | Issue number 27 | 1
Number of pages 13
Organisations
  • Faculty of Science (FNWI) - Informatics Institute (IVI)
Abstract
Classic formulae for entropy and cross-entropy contain operations š‘„/0 and log2š‘„ that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2Ā 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols Ā±āˆž for signed infinite values and a symbol named ⊄ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.
Document type Article
Language English
Published at https://doi.org/10.3390/e27010031
Other links https://www.scopus.com/pages/publications/85215682138
Downloads
Permalink to this page
Back