Let $X \in \text{Random}(\mathcal{X})$ be a D5723: Simple random variable such that
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
| (i) | \begin{equation} \forall \, x \in \mathcal{X} : p(x) : = \mathbb{P}(X = x) \end{equation} |
Then
| (1) | \begin{equation} - \sum_{x \in \mathcal{X}} p(x) \log_a p(x) \leq \log_a |\mathcal{X}| \end{equation} |
| (2) | \begin{equation} - \sum_{x \in \mathcal{X}} p(x) \log_a p(x) = \log_a |\mathcal{X}| \quad \iff \quad \forall \, x \in \mathcal{X} : p(x) = \frac{1}{|\mathcal{X}|} \end{equation} |
