What would be a good example of an infinite entropy ? Let

\[ \begin{aligned} p(x) &= {1 \over Ax\log^2{x}} & \text{where } A = \sum_x {1 \over x\log^2{x}} \\ \end{aligned} \]

Note \(A\) converges as \(\displaystyle \int {1 \over x\log^2{x}}\,dx = -{1 \over \log{x}}\). Now,

\[ \begin{aligned} H(X) &= - \sum_x p(x) \log{p(x)} \\ &= - \sum_x {1 \over Ax\log^2{x}} \log{{1 \over Ax\log^2{x}}} \\ &= \sum_x {\log{(Ax\log^2{x})} \over Ax\log^2{x}} \\ &= \sum_x \left[ {\log{A} \over Ax\log^2{x}} + {\log{x} \over Ax\log^2{x}} + {\log{\log^2{x}} \over Ax\log^2{x}} \right] \\ &= \sum_x \left[ {\log{A} \over Ax\log^2{x}} + \color{blue}{1 \over Ax\log{x}} + {\log{\log^2{x}} \over Ax\log^2{x}} \right] \\ \end{aligned} \]

But the middle term (in blue) diverges, as \(\displaystyle \int {1 \over x\log{x}}\,dx = \log\log x\). Hence \(H(X)\) diverges. \(\Box\)

Source: math.stackexchange.com.

# posted by rot13(Unafba Pune) @ 7:02 AM