Estimate entropy of a finite discrete system using limited samples
Following idea of:
Ma, “Calculation of Entropy from Data of Motion”, Journal of Statistical Physics26, 221–240 (1981) URL. zotero
Consider a system with a state space of the size $\Gamma$ with equal probability. The entropy is given by
$$ H = - \sum_x \frac{1}{\Gamma} \ln\frac{1}{\Gamma} $$