A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family

Consider a pair of random vectors (X, Y) and the conditional expectation operator bE[X Y=y]. This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain U ← X ← Y, a compact expression for the Jacobian matrix of bE[U Y=y] is derived. In the second part of the paper, the main identity is specialized to the exponential family. Moreover, via various choices of the random vector U, the new identity is used to recover and generalize several known identities and derive some new ones. As a first example, a connection between the Jacobian of bE[X Y=y] and the conditional variance is established. As a second example, a recursive expression between higher order conditional expectations is found, which is shown to lead to a generalization of the Tweedy's identity. Finally, as a third example, it is shown that the k-th order derivative of the conditional expectation is proportional to the (k+1)-th order conditional cumulant.