Friday, 3 January 2014

Bounding the quantum relative entropy in terms of the classical relative entropy

Theorem:

Let $\X$ be a complex Euclidean space, let $\Sigma$ be an alphabet, let $p,q\in\P(\Sigma)$ be probability vectors, and let $\{\rho_a\,:\,a\in\Sigma\}\subset\Density(\X)$ and $\{\sigma_a\,:\,a\in\Sigma\}\subset\Density(\X)$ be collections of density operators indexed by $\Sigma$. Assume that $\im(\rho_a)\subseteq\im(\sigma_a)$, $p(a)>0$, and $q(a) > 0$ for all $a\in\Sigma$. For two positive definite operators $P$ and $Q$ acting on $\X$, denote the quantum relative entropy as
$S(P || Q )=\tr(P \ \text{log}(P))-\tr(P \ \text{log}(Q))$

Then
$S\Biggl(\sum_{a\in\Sigma} p(a) \rho_a \Bigg\| \sum_{a\in\Sigma} q(a) \sigma_a \Biggr) \leq \sum_{a\in\Sigma} p(a) S(\rho_a \| \sigma_a) + D(p \| q),$
where
$D(p \| q):=\sum_{a\in\Sigma}\Bigl(p(a)\text{log}\Bigl(\frac{p(a)}{q(a)}\Bigr).$
is the classical relative entropy of two probability vectors $p,q\in\P(\Sigma)$.

Proof:

Consider the following fact, which states that for a complex Euclidean space $\X$ and operators $P_0,P_1,Q_0,Q_1\in \Pos(\X)$,
$S(P_0+P_1\| Q_0+Q_1)\leq S(P_0 \|Q_0)+S( P_1 \| Q_1).$
Therefore,
$S\Biggl(\sum_{a\in\Sigma} p(a) \rho_a \Bigg\| \sum_{a\in\Sigma} q(a) \sigma_a \Biggr) \leq \sum_{a\in\Sigma}S\left( p(a) \rho_a \| q(a) \sigma_a \right).$
Now as a consequence, for $P,Q\in\Pos{\X}$ and scalars $\alpha,\beta\in(0,\infty)$
$S(\alpha P \| \beta Q)=\alpha S(P\|Q)+\alpha \text{log}(\alpha/\beta)\tr(P).$
Thus,
\begin{align*} S\Biggl(\sum_{a\in\Sigma} p(a) \rho_a \Bigg\| \sum_{a\in\Sigma} q(a) \sigma_a \Biggr) & \leq \sum_{a\in\Sigma}S\left( p(a) \rho_a \| q(a) \sigma_a \right) \\ &= \sum_{a\in\Sigma} \Bigl(p(a)S(\rho_a \| \sigma_a)+p(a)\text{log}\Bigl(\frac{p(a)}{q(a)}\Bigr)\tr(\rho_a) \Bigr) \\ &=\sum_{a\in\Sigma} \Bigl(p(a)S(\rho_a \| \sigma_a)\Bigr)+\sum_{a\in\Sigma}\Bigl(p(a)\text{log}\Bigl(\frac{p(a)}{q(a)}\Bigr) \Bigr), \end{align*}
since $\rho_a\in\Density(\X)$ implies that $\tr(\rho_a)=1$. Also, by definition of the relative entropy of two probability vectors $p,q\in\P(\Sigma)$,
$D(p \| q):=\sum_{a\in\Sigma}\Bigl(p(a)\text{log}\Bigl(\frac{p(a)}{q(a)}\Bigr).$
Hence,
$\sum_{a\in\Sigma} \Bigl(p(a)S(\rho_a \| \sigma_a)\Bigr)+\sum_{a\in\Sigma}\Bigl(p(a)\text{log}\Bigl(\frac{p(a)}{q(a)}\Bigr) \Bigr)=\sum_{a\in\Sigma} \Bigl(p(a)S(\rho_a \| \sigma_a)\Bigr)+ D(p \| q) \Bigr),$
which implies
$S\Biggl(\sum_{a\in\Sigma} p(a) \rho_a \Bigg\| \sum_{a\in\Sigma} q(a) \sigma_a \Biggr) \leq \sum_{a\in\Sigma} p(a) S(\rho_a \| \sigma_a) + D(p \| q).$