Princeton University Recent Lecture 6. Entropic CLT (3) Lecture 5. Entropic CLT (2) Lecture onu 4. E

Princeton University Recent Lecture 6. Entropic CLT (3) Lecture 5. Entropic CLT (2) Lecture onu 4. Entropic CLT (1) Lecture 3. Sanov’s theorem Lecture 2. Basics / law of small numbers Categories Announcement (7) Information theoretic methods (7) Random graphs (9) Subscribe to Blog via Email
In onu the previous lecture, we proved subadditivity of the inverse Fisher information . The key part of the proof was the observation that the score function of the sum could be written as the conditional expectation of a sum of independent random variables, whose variance is trivially computed. This does not suffice, however, to prove monotonicity in the CLT. To do the latter, we need a more refined bound on the Fisher information in terms of overlapping subsets of indices. Following the same proof, the score function of the sum can be written as the conditional expectation of a sum of terms that are now no longer independent. To estimate the variance of this sum, we will use the following “variance drop lemma” whose proof relies on the ANOVA decomposition.
    Of course, if then . Thus Hoeffding’s inequality for the variance of -statistics above and the more general variance drop lemma should be viewed as capturing how much of a drop we get in variance of an additive-type function, when the terms are not independent but have only limited dependencies (overlaps) in their structure. Proof. We may assume without loss of generality that each has mean zero.
With , we can write
where . Denote by the density of . The following facts are readily verified for : . is smooth. . .
where is Brownian motion. This is, like Brownian motion, onu a Markov process, but the drift term (which always pushes trajectories towards 0) ensures that it has a stationary distribution, unlike Brownian motion. The Markov semigroup associated to this Markov process, namely the semigroup of operators defined on an appropriate domain by
    onu has a generator (defined via ) given by . The semigroup generated by governs the evolution of conditional expectations of functions of the process , while the adjoint semigroup generated by governs the evolution of the marginal density of . The above expression for follows from this remark by noting that and are the same in distribution; however, it can also be deduced more simply just by writing down the density of explicitly, and using the smoothness of the Gaussian density to verify each part of the claim.
The differential form follows onu by using the last part of the claim together with integration by parts. The integral form follows from the differential form by the fundamental theorem of calculus, since