What would be a good example of an infinite entropy ? Let

$$\begin{aligned} p(x) &= {1 \over Ax\log^2{x}} & \text{where } A = \sum_x {1 \over x\log^2{x}} \\ \end{aligned}$$

Note $A$ converges as $\displaystyle \int {1 \over x\log^2{x}}\,dx = -{1 \over \log{x}}$. Now,

$$\begin{aligned} H(X) &= - \sum_x p(x) \log{p(x)} \\ &= - \sum_x {1 \over Ax\log^2{x}} \log{{1 \over Ax\log^2{x}}} \\ &= \sum_x {\log{(Ax\log^2{x})} \over Ax\log^2{x}} \\ &= \sum_x \left[ {\log{A} \over Ax\log^2{x}} + {\log{x} \over Ax\log^2{x}} + {\log{\log^2{x}} \over Ax\log^2{x}} \right] \\ &= \sum_x \left[ {\log{A} \over Ax\log^2{x}} + \color{blue}{1 \over Ax\log{x}} + {\log{\log^2{x}} \over Ax\log^2{x}} \right] \\ \end{aligned}$$

But the middle term (in blue) diverges, as $\displaystyle \int {1 \over x\log{x}}\,dx = \log\log x$. Hence $H(X)$ diverges. $\Box$

Source: math.stackexchange.com.

# posted by rot13(Unafba Pune) @ 7:02 AM