Princeton University Recent Lecture 6. Entropic CLT (3) Lecture 5. Entropic online calculator CLT (2) Lecture 4. Entropic CLT (1) Lecture 3. Sanov’s theorem Lecture 2. Basics online calculator / law of small numbers Categories online calculator Announcement (7) Information theoretic methods (7) Random online calculator graphs (9) Subscribe to Blog via Email
The goal of this lecture is to prove monotonicity of Fisher information in the central limit theorem. Next lecture we will connect Fisher information to entropy, completing the proof of the entropic CLT.
Lemma 2. Let be a random variable online calculator with density and let be a measurable function with . Suppose that for every bounded measurable function , the function is absolutely continuous on with a.e. Then there must exist an absolutely continuous version of the density , and moreover a.s.
Fix . By independence of and , we can apply Lemma 1 to (conditioned on ) to obtain
Remark. The well-known interpretation of conditional expectation as a projection means that under the assumption of finite Fisher information (i.e., score functions in ), the score function of the sum is just the projection of the score of a summand onto the closed subspace . This implies online calculator directly, by the Pythagorean inequality, that convolution decreases Fisher information: . In fact, we can do better, as we will see forthwith.
Remark. Let be an i.i.d. sequence of random variables with mean zero and unit variance. Let . By taking in the theorem and using scaling property of Fisher information, it is easy to obtain . Hence, the above theorem already implies monotonicity of Fisher information along the subsequence of times : that is, is monotone in . However, the theorem is not strong enough to give monotonicity without online calculator passing to a subsequence. For example, if we apply the previous remark repeatedly we only get , which is not very interesting. To prove full monotonicity of the Fisher information, we will need a strengthening of the above Theorem. But it is instructive to first consider the proof of the simpler online calculator case.
To prove monotonicity of the Fisher information in the CLT (without passing to a subsequence) we need a strengthening of the property of Fisher information given in the previous section.
by the above theorem. By the scaling property of Fisher information, this is equivalent to , i.e., monotonicity of the Fisher information. This special case was first proved by Artstein, Ball, Barthe and Naor (2004) with a more complicated proof. The proof we will follow of the more general theorem above is due to Barron and Madiman (2007) .
The online calculator proof of the above theorem is based on an analysis online calculator of variance (ANOVA) type decomposition, which dates back at least to the classic paper of Hoeffding (1948) on U-statistics. To state this decomposition, let be independent random variables, and define the Hilbert online calculator space
The decomposition will be used in the form of the following variance drop lemma whose proof we postpone to next lecture. Here for , .
