Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
| technical:finite-entropy-estimate [2022/11/26 16:46] – [Distribution] chunchung | technical:finite-entropy-estimate [2022/11/27 05:36] (current) – [Distribution] chunchung | ||
|---|---|---|---|
| Line 19: | Line 19: | ||
| while the length of the sequence is by | while the length of the sequence is by | ||
| \begin{equation} N = \sum_n n k_n . \end{equation} | \begin{equation} N = \sum_n n k_n . \end{equation} | ||
| - | Since each element of the sequence is drawn from the states, we have the total number of possible sequences | + | Here, the value of $k_0$ is not directly known from the observed sequence and is linked to the total number of states $\Gamma$. |
| \begin{equation} N_\mathrm{seq} = \Gamma^N. \end{equation} | \begin{equation} N_\mathrm{seq} = \Gamma^N. \end{equation} | ||
| The number of of the sequences among $N_\mathrm{seq}$ that will give rise to a given combination of $\{k_n\}$ can be calculated from the product of two factors. First is the number of ways to pick the states with $n$ repeats for all $n$ values, | The number of of the sequences among $N_\mathrm{seq}$ that will give rise to a given combination of $\{k_n\}$ can be calculated from the product of two factors. First is the number of ways to pick the states with $n$ repeats for all $n$ values, | ||
| \begin{equation} \frac{\Gamma !}{\prod_n k_n!}. \end{equation} | \begin{equation} \frac{\Gamma !}{\prod_n k_n!}. \end{equation} | ||
| - | Second is the number of ways assigned | + | Second is the number of ways for assigning |
| \begin{equation} \frac{N!}{\prod_n (n!)^{k_n}}. \end{equation} | \begin{equation} \frac{N!}{\prod_n (n!)^{k_n}}. \end{equation} | ||
| - | Overall, the likelihood of getting the combination $\{k_n| n \in \mathbb{N}_0 \}$ ($\mathbb{N}_0$ is the set of non-negative | + | Overall, the likelihood of getting the combination $\{k_n| n \in \mathbb{N} \}$ ($\mathbb{N}$ is the set of positive |
| \begin{equation} \frac{\Gamma! N!}{\Gamma^N \prod_n k_n! (n!)^{k_n}}. \end{equation} | \begin{equation} \frac{\Gamma! N!}{\Gamma^N \prod_n k_n! (n!)^{k_n}}. \end{equation} | ||
| + | In the expression, the number of unobserved states is $k_0 = \Gamma - \sum_{n\in\mathbb{N}}k_n$. | ||
| =====Distribution===== | =====Distribution===== | ||
| Consider a given state, the probability that it gets hit by $n$ points is | Consider a given state, the probability that it gets hit by $n$ points is | ||
| Line 33: | Line 34: | ||
| & = \frac{N!}{n!(N-n)!} \Gamma^{-n}\left(1-\Gamma^{-1}\right)^{N-n} \\ | & = \frac{N!}{n!(N-n)!} \Gamma^{-n}\left(1-\Gamma^{-1}\right)^{N-n} \\ | ||
| & = \frac{1}{n!}\frac{N(N-1)\ldots(N-n+1)}{\Gamma^n}\left(1-\Gamma^{-1}\right)^{N-n} \\ | & = \frac{1}{n!}\frac{N(N-1)\ldots(N-n+1)}{\Gamma^n}\left(1-\Gamma^{-1}\right)^{N-n} \\ | ||
| - | & \approx \frac{\lambda^n}{n!}(1-\frac{\lambda}{N})^N | + | & \approx \frac{\lambda^n}{n!}(1-\frac{\lambda}{N})^N |
| & \approx \frac{\lambda^n}{n!}e^{-\lambda} | & \approx \frac{\lambda^n}{n!}e^{-\lambda} | ||
| \end{align} | \end{align} | ||
| + | approaching the Poisson distribution with $\lambda \equiv N/\Gamma$ at the large $N$, $\Gamma$ limit. | ||