The entropic approach to causal correlations

Let $S=\{{X}_{1},\,\ldots ,\,{X}_{n}\}$ be a set of n random variables taking values ${x}_{1},\,\ldots ,\,{x}_{n}$, whose joint distribution $P({x}_{1},\,\ldots ,\,{x}_{n})$ we wish to characterize entropically. For every nonempty subset $T\subset S$ we shall denote by ${{\boldsymbol{X}}}_{T}={({X}_{i})}_{{X}_{i}\in T}$ the joint random variable that involves all variables in T, taking values ${{\boldsymbol{x}}}_{T}={({x}_{i})}_{{X}_{i}\in T}$. We can then compute the marginal Shannon entropies $H({{\boldsymbol{X}}}_{T})=H(T)$ from the marginal probability distributions $P({{\boldsymbol{X}}}_{T}={{\boldsymbol{x}}}_{T})=P({{\boldsymbol{x}}}_{T})$ as

$$\begin{eqnarray}&&H(T):= -\displaystyle \sum _{{{\boldsymbol{x}}}_{T}}P({{\boldsymbol{x}}}_{T}){\mathrm{log}}_{2}P({{\boldsymbol{x}}}_{T}).\end{eqnarray} \tag{ 3 }$$

Together with $H(\varnothing ):= 0$, every global probability distribution $P({x}_{1},\,\ldots ,\,{x}_{n})$ thus specifies ${2}^{n}$ real numbers in the entropic description, which can be expressed as the components of a $({2}^{n})$-dimensional vector ${\boldsymbol{h}}=(H(\varnothing ),H({X}_{1}),\,\ldots ,\,H({X}_{1}{X}_{2}),\,\ldots ,\,H({X}_{1}\ldots {X}_{n}))={(H(T))}_{T\subset S}$ in ${{\mathbb{R}}}^{{2}^{n}}$.

A fundamental problem in information theory is to decide whether a given vector is an entropy vector, that is, if it is obtainable from some probability distribution. The (closure of the) region of valid entropy vectors

$$\begin{eqnarray}&&{{\rm{\Gamma }}}_{S}^{* }:= \overline{\{{\boldsymbol{h}}\in {{\mathbb{R}}}^{{2}^{n}}\,| \,{\boldsymbol{h}}={(H(T))}_{T\subset S}\}},\end{eqnarray} \tag{ 4 }$$

is known to be a convex cone, called the entropy cone (see [41] for a comprehensive discussion of entropy cones). There is no known explicit description of ${{\rm{\Gamma }}}_{{S}}^{* }$, so one generally has to rely on an approximation of it. A well-known and very useful outer approximation of ${{\rm{\Gamma }}}_{{S}}^{* }$ is the so-called Shannon cone ${{\rm{\Gamma }}}_{{S}}$, defined by the elemental inequalities

$$\begin{eqnarray}H(S\setminus \{{X}_{i}\}) & \leqslant & H(S),\\ H(T)+H(T\cup \{{X}_{i},{X}_{j}\}) & \leqslant & H(T\cup \{{X}_{i}\})+H(T\cup \{{X}_{j}\}),\end{eqnarray} \tag{ 5 }$$

for all $1\leqslant i,j\leqslant n$, $i\ne j$, and $T\subset S\setminus \{{X}_{i},{X}_{j}\}$. That is, the Shannon cone ${{\rm{\Gamma }}}_{{S}}$ is described by a finite system of $m=n+{2}^{n-2}\left(\genfrac{}{}{0em}{}{n}{2}\right)$ linear inequalities, which one can write in the form $I{\boldsymbol{h}}\leqslant {\bf{0}}$, where I is an $m\times {2}^{n}$ real matrix and ${\bf{0}}$ a vector with null entries. The inequalities in equation (5) are the minimal set of inequalities implying the monotonicity of entropy, i.e., $H(U| T):= H({TU})-H(T)\geqslant 0$, and the submodularity (or strong subadditivity), i.e., $I(U:V| T):= H({TU})+H({TV})-H({TUV})-H(T)\geqslant 0$, for any subsets $T,U,V\subset S$. These inequalities and any combination thereof are known as Shannon-type inequalities. It is known that for $n\leqslant 3$ variables every inequality delimiting the entropy cone ${{\rm{\Gamma }}}_{S}^{* }$ is of the Shannon type; however, this is not the case for $n\gt 3$ [41].

The inequalities characterizing the Shannon cone simply arise from demanding that the function $P({{\boldsymbol{x}}}_{T})$ appearing in (3) should be identified with a valid probability distribution (i.e., it should be non-negative and normalized). However, one often wishes to consider (and characterize the entropy vectors for) situations where additional constraints on the random variables are known. For example, X_i and X_j might be known to be independent, which implies that $P({x}_{i},{x}_{j})=P({x}_{i})P({x}_{j})$. Such independence constraints, which are nonlinear in terms of probabilities, define simple linear constraints in terms of entropies, e.g., $P({x}_{i},{x}_{j})=P({x}_{i})P({x}_{j})\to H({X}_{i}{X}_{j})=H({X}_{i})+H({X}_{j})$. These extra constraints can be easily incorporated into the entropic framework since they define a linear subspace, which we denote ${{L}}_{{ \mathcal C }}$, characterized by linear equalities. When combined with the elemental inequalities one obtains a new finite system of inequalities ${I}^{{\prime} }{\boldsymbol{h}}\leqslant {\bf{0}}$ characterizing the 'constrained Shannon cone' ${{\rm{\Gamma }}}_{{S}}\,\bigcap \,{{L}}_{{ \mathcal C }}$.

In some cases, one may also wish to add linear inequality constraints which, in general, may give rise to more general polyhedra described by inhomogeneous systems of linear inequalities ${I}^{{\prime} }{\boldsymbol{h}}\leqslant {\boldsymbol{\beta }}$ [42]. In such cases we will again denote the set of vectors ${\boldsymbol{h}}$ satisfying these additional constraints as ${{L}}_{{ \mathcal C }};$ we will return to this point in more detail in section 3.

Consider again a set of random variables $\{{X}_{1},\,\ldots ,\,{X}_{n}\}$ with a joint probability distribution $P({x}_{1},\,\ldots ,\,{x}_{n})$. We often encounter situations where not all variables, or combinations thereof, are empirically accessible. For example, our system of interest could be composed of three random variables ${X}_{1},{X}_{2},{X}_{3}$ but, for some reason, we can access at most two of them at a time, thus implying that we cannot know their joint entropy $H({X}_{1}{X}_{2}{X}_{3})$. Alternatively, there might be variables that represent latent factors [2] and that, for this reason, are unobservable. In such cases, we face a marginal problem: decide whether some given information on the marginals is compatible with a global description fulfilling certain constraints (for example the elemental entropy inequalities). In the example with three variables, it is easy to see that the elemental inequalities imply that

$$\begin{eqnarray}&&H({X}_{1})+H({X}_{2}{X}_{3})\leqslant H({X}_{1}{X}_{2})+H({X}_{1}{X}_{3}).\end{eqnarray} \tag{ 6 }$$

That is, the global structure of entropy vectors implies nontrivial constraints (which are not elemental inequalities (5)) that should be respected by any marginal information compatible with it.

More formally, given a set of random variables $S=\{{X}_{1},\,\ldots ,\,{X}_{n}\}$, a marginal scenario is a collection of subsets ${ \mathcal M }=\{{M}_{1},\,\ldots ,\,{M}_{| { \mathcal M }| }\}$, ${M}_{j}\subset S$ representing those variables for which we have access to the probability distribution $P({{\boldsymbol{x}}}_{{M}_{j}})$ (and thus to $H({M}_{j})$). Clearly, ${M}_{j}\in { \mathcal M }$ and ${M}_{j}^{{\prime} }\subset {M}_{j}$ implies ${M}_{j}^{{\prime} }\in { \mathcal M }$, that is, given some probability distribution we also have access to any marginal of it. In a slight abuse of notation we will therefore write ${ \mathcal M }$ only in terms of its maximal subsets, since these are sufficient to specify the entire marginal scenario; the complete representation of ${ \mathcal M }$, which explicitly includes all (not necessarily maximal) subsets T for which the marginal distribution $P({{\boldsymbol{x}}}_{T})$ is accessible, will be denoted ${{ \mathcal M }}^{{\rm{c}}}=\{T\,| \,T\subset {M}_{j},{M}_{j}\in { \mathcal M }\}$. In the example above the marginal scenario would then be represented as ${ \mathcal M }=\{\{{X}_{1},{X}_{2}\},\{{X}_{1},{X}_{3}\},\{{X}_{2},{X}_{3}\}\}$, or ${{ \mathcal M }}^{{\rm{c}}}=\{\varnothing ,\{{X}_{1}\},\{{X}_{2}\},\{{X}_{3}\},\{{X}_{1},{X}_{2}\},\{{X}_{1},{X}_{3}\},\{{X}_{2},{X}_{3}\}\}$.

In general we are interested in characterizing the entropy cone ${{\rm{\Gamma }}}_{{ \mathcal M }}^{* }$ associated with a marginal scenario ${ \mathcal M }$, thus obtaining constraints implied by the global entropy cone on the marginal subspace of interest. Geometrically, this corresponds to the projection of the original entropy cone onto the subspace of entropy vectors ${\boldsymbol{h}}={(H(T))}_{T\in {{ \mathcal M }}^{{\rm{c}}}}\in {{\mathbb{R}}}^{| {{ \mathcal M }}^{{\rm{c}}}| }$, corresponding to the variables in ${{ \mathcal M }}^{{\rm{c}}}$. Since, in practice, we work with the Shannon cone ${{\rm{\Gamma }}}_{{S}}$—possibly constrained by some further linear constraints specifying a subset of entropy vectors ${{L}}_{{ \mathcal C }}$, as described previously—which is characterized by a finite system of inequalities, this projection corresponds to a simple variable elimination of all the terms not contained in ${{ \mathcal M }}^{{\rm{c}}}$ [28, 43, 44]. After removing redundant inequalities, the remaining inequalities are facets (i.e., the boundaries) of the Shannon cone, or more generally polyhedron, in the observable marginal subspace. Formally, the marginal Shannon polyhedron ${{\rm{\Gamma }}}_{{ \mathcal M }}$ is defined as

$$\begin{eqnarray}&&{{\rm{\Gamma }}}_{{ \mathcal M }}={{\rm{\Pi }}}_{{ \mathcal M }}({{\rm{\Gamma }}}_{{S}}\,\bigcap \,{{L}}_{{ \mathcal C }}),\end{eqnarray} \tag{ 7 }$$

where ${{\rm{\Pi }}}_{{ \mathcal M }}$ denotes the projection onto the coordinates associated with the marginal scenario ${ \mathcal M }$—i.e., onto the coordinates H(T) with $T\in {{ \mathcal M }}^{{\rm{c}}}$.

The characterization of entropy cones (or polyhedra) and marginal problems outlined above can be easily extended to the case where we no longer assume that there is a well-defined global probability distribution over all the variables in the set S. Instead, we may assume that only certain subsets of variables have such a joint distribution, and that only the marginals of certain subsets of these subsets are empirically accessible. This type of restriction may be imposed by assumptions about the underlying physical theory being described, as will be clear in the example we discuss in section 2.2.5.

We will denote the collection of subsets of S for which we assume joint probability distributions exist by ${ \mathcal S }=\{{S}_{1},\,\ldots ,\,{S}_{| { \mathcal S }| }\}$, with each ${S}_{i}\subset S$ such that ${\cup }_{i}{S}_{i}=S;$ we call ${ \mathcal S }$ the probability structure⁶ . As for the marginal scenario, we will represent ${ \mathcal S }$ by just its maximal subsets in a slight abuse of notation; the complete representation of ${ \mathcal S }$, that explicitly includes all subsets for which a joint probability distribution exists, will similarly be denoted ${{ \mathcal S }}^{{\rm{c}}}$. In such a situation the entropies H(T) cannot be defined for all subsets $T\subset S$, but only for the subsets in ${{ \mathcal S }}^{{\rm{c}}}$. The entropy vectors we shall consider will thus be defined here as ${\boldsymbol{h}}={(H(T))}_{T\in {{ \mathcal S }}^{{\rm{c}}}}\in {{\mathbb{R}}}^{| {{ \mathcal S }}^{{\rm{c}}}| }$. Again, no explicit characterization is known for the set of valid entropy vectors; we will instead rely on its outer approximation characterized via the Shannon constraints, now restricted to each subset ${S}_{i}\in { \mathcal S }$. Namely, the Shannon cone of interest is now

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{ \mathcal S }}=\bigcap _{{{S}}_{i}\in { \mathcal S }}{{\rm{\Gamma }}}_{{{S}}_{i}},\end{eqnarray} \tag{ 8 }$$

where ${{\rm{\Gamma }}}_{{{\rm{S}}}_{i}}\subset {{\mathbb{R}}}^{| {{ \mathcal S }}^{{\rm{c}}}| }$ is the cone defined by the Shannon inequalities on the variables in S_i, which, in particular, leave the other variables in $S\setminus {S}_{i}$ unconstrained. In the extremal case where we do assume a global joint probability distribution for all variables we have ${ \mathcal S }=\{S\}$, ${{ \mathcal S }}^{{\rm{c}}}={2}^{S}$, and we recover ${{\rm{\Gamma }}}^{{ \mathcal S }}={{\rm{\Gamma }}}_{{S}}$.

One can similarly consider marginal scenarios under a given probability structure ${ \mathcal S }$, with the constraint that marginals must arise from existing probability distributions, i.e., for all ${M}_{j}\in { \mathcal M }$ there must exist an ${S}_{i}\in { \mathcal S }$ such that ${M}_{j}\subset {S}_{i}$. One can also add linear constraints to the entropy vectors under consideration, as before, represented by some subset of entropy vectors ${{L}}_{{ \mathcal C }}$. We can thus define the marginal Shannon polyhedron associated with ${ \mathcal S },{ \mathcal M }$, and ${{L}}_{{ \mathcal C }}$ as

$$\begin{eqnarray}&&{{\rm{\Gamma }}}_{{ \mathcal M }}^{{ \mathcal S }}={{\rm{\Pi }}}_{{ \mathcal M }}({{\rm{\Gamma }}}^{{ \mathcal S }}\bigcap {{L}}_{{ \mathcal C }}).\end{eqnarray} \tag{ 9 }$$

The choice of probability structure can generally be considered on a case-by-case basis depending on the scenario being modeled. Unless otherwise stated we will take ${ \mathcal S }=\{S\}$ but, as we will discuss, this will not always be the most pertinent choice.

In order to describe the causal relations between random variables, we will first use the framework of causal Bayesian networks⁷ [2]. Such networks can be conveniently represented as directed acyclic graphs (DAGs), in which each node represents a variable and directed edges (arrows) encode the causal relations between them. A set of variables $S=\{{X}_{1},\,\ldots ,\,{X}_{n}\}$ forms a Bayesian network with respect to a given DAG if and only if the variables admit a global probability distribution $P({x}_{1},\,\ldots ,\,{x}_{n})$, i.e., ${ \mathcal S }=\{S\}$, that factorizes according to

$$\begin{eqnarray}&&P({x}_{1},\,\ldots ,\,{x}_{n})=\displaystyle \prod _{i=1}^{n}P({x}_{i}| {\mathrm{Pa}}_{i}),\end{eqnarray} \tag{ 10 }$$

where ${\mathrm{Pa}}_{i}$ stands for the graph-theoretical parents of variable X_i, that is, all those variables X_j that have an outgoing edge pointing to X_i in the DAG under consideration. The decomposition above implies a set of CIs, which are either independence relations of the type $P({x}_{i},{x}_{j})=P({x}_{i})P({x}_{j})$ (in which case we write ${X}_{i}\perp \perp {X}_{j}$) or conditional independence relations such as $P({x}_{i},{x}_{j}| {x}_{k})=P({x}_{i}| {x}_{k})P({x}_{j}| {x}_{k})$ (denoted ${X}_{i}\perp \perp {X}_{j}\,| \,{X}_{k}$)⁸ . Given a DAG, a complete list of CIs can be obtained via the d-separation criterion [2]. If the arrows in the DAG representation of a Bayesian network describe the direct causal relations between the variables in question, then we call it a causal Bayesian network.

Entropically, these CIs correspond to simple linear relations: ${X}_{i}\perp \perp {X}_{j}\to H({X}_{i}{X}_{j})=H({X}_{i})+H({X}_{j})$ and ${X}_{i}\perp \perp {X}_{k}| {X}_{k}\to H({X}_{i}{X}_{j}| {X}_{k})=H({X}_{i}| {X}_{k})+H({X}_{j}| {X}_{k})$. As a result, the set of entropy vectors compatible with a given DAG is the intersection of the entropy cone ${{\rm{\Gamma }}}_{{S}}^{* }$ with the linear subspace ${{L}}_{\mathrm{CI}}$ defined by the set of linear constraints that characterize the CIs associated with the DAG [29, 35]. In practice, we again rely on the outer approximation given by the intersection of the Shannon cone ${{\rm{\Gamma }}}_{{S}}$ with ${{L}}_{\mathrm{CI}}$.

If all the variables in a DAG are observable, in order to check the compatibility of a given entropy vector with the DAG it suffices to check whether all the entropic CIs are satisfied. However, we are often interested in DAGs containing latent, nonobservable, variables. Splitting the n variables making up the DAG into j observable variables ${O}_{1},\,\ldots ,\,{O}_{j}$ and n − j latent variables ${{\rm{\Lambda }}}_{1},\,\ldots ,\,{{\rm{\Lambda }}}_{n-j}$ we thus need to compute the marginal Shannon cone ${{\rm{\Pi }}}_{{ \mathcal M }}({{\rm{\Gamma }}}_{{S}}\bigcap {{L}}_{\mathrm{CI}})$ where ${ \mathcal M }=\{\{{O}_{1},\,\ldots ,\,{O}_{j}\}\}$.

As an illustration, consider the paradigmatic causal Bayesian network for a local hidden variable model satisfying Bell's assumption of local causality [5, 7]. The relevant DAG, shown in figure 1, has five variables, four of which are observable while the hidden variable Λ is not: in the context of Bell's Theorem the 'hidden variables' indeed refer to the latent factors introduced above. This DAG represents the physical scenario where two distant observers receive physical systems produced by a common source (the hidden variable Λ) and make different measurements (choices of which are labeled by X and Y), obtaining measurement outcomes (represented by the variables A and B). That is, the probability structure is ${ \mathcal S }=\{S\}$ with $S=\{X,Y,A,B,{\rm{\Lambda }}\}$, and the marginal scenario is ${ \mathcal M }=\{\{X,Y,A,B\}\}$. Some of the CIs implied by this DAG are given by $P({xy}\lambda )=P(x)P(y)P(\lambda )$ (the measurement independence assumption), $P(a| {xyb}\lambda )=P(a| x\lambda )$ and $P(b| {xya}\lambda )=P(b| y\lambda )$ (the locality assumption) that in turn imply (after eliminating the hidden variable Λ) Bell inequalities for the observed variables [5, 7]. These constraints also imply the no-signaling constraints $P(a| {xy})=P(a| x)$ and $P(b| {xy})=P(b| y)$.

Zoom In Zoom Out Reset image size

This example shows that, in general, DAGs with latent variables imply CIs both on the level of observable and unobservable variables. The CIs involving latent variables are not directly testable but imply further constraints (Bell inequalities, in the example above) that can be tested to check whether the observable behavior is compatible with the proposed underlying DAG.

If, instead of characterizing the allowed probability distributions, we consider the entropic description of the Bell scenario, i.e., the Shannon cone together with the linear constraints arising from the DAG's CIs, then after eliminating the latent variable Λ one obtains no further constraints other than the elemental inequalities (which are trivial since they are respected by all probability distributions) and the observable CIs implied by the DAG: $H({XY})=H(X)+H(Y)$, $H(A| {XY})=H(A| X)$ and $H(B| {XY})=H(B| Y)$ [45]. The first CI relation represents the independence of the two measurement choices, while the two latter ones are no-signaling conditions. Thus, for this particular causal Bayesian network, when the entropic approach is applied to the variables making up the DAG, one does not obtain any nontrivial constraints (i.e., entropic Bell inequalities) [45]. However, there are many examples of Bayesian networks for which one does obtain such nontrivial constraints [9, 29, 35]. In fact, as we will see in section 3, a slight modification of this method also leads to nontrivial constraints on causal correlations.

While the DAG method fails to provide nontrivial constraints for the Bell scenario (a result that can be extended to a larger class of 'line-like' Bayesian networks [45]), it has been known for some time that entropic Bell inequalities can be derived using different methods [26]. Interestingly, these inequalities can even be turned into necessary and sufficient conditions for a given probability distribution to satisfy Bell's local causality assumption [31].

The method that allows such inequalities to be derived is motivated by the realization that the entropic approach can be applied to any marginal scenario for a relevant set of random variables [28], and not only those arising from causal Bayesian networks. In particular, when we are interested in constraints on conditional distributions of the form $P({ab}| {xy})$, where we have distinct sets of input and output variables, we may consider the output variables conditioned on certain relevant input variables (e.g. A_xy and B_xy, where the notation A_xy denotes the random variable $A| (X=x,Y=y)$)⁹ . The choice of relevant input variables to condition on, as well as the appropriate probability structure, will depend on the physical situation being considered. In general, a global probability distribution may not exist on such 'counterfactual' variables even if one does exist on the unconditioned variables.

Let us illustrate how this method may be applied by considering again its application to the Bell scenario. Instead of considering all the input and output variables as in the DAG approach (e.g. $X,Y,A,B$), one can consider copies of the output variables conditioned on the corresponding party's input, i.e., ${A}_{x},{B}_{y}$, where A_x denotes the random variable $A| (X=x)$. Indeed, due to the no-signaling constraints, the output variables can only depend on the corresponding local input. Furthermore, from Fine's Theorem [46] we know that Bell's local causality assumption is equivalent to the existence of a well defined (although empirically inaccessible) joint probability distribution $P({a}_{1},\,\ldots ,\,{a}_{| { \mathcal X }| },{b}_{1},\,\ldots ,\,{b}_{| { \mathcal Y }| })$ (where ${ \mathcal X }=\{1,\,\ldots ,\,| { \mathcal X }| \}$ and ${ \mathcal Y }=\{1,\,\ldots ,\,| { \mathcal Y }| \}$ denote the alphabets of Alice and Bob's inputs) on these variables¹⁰ that marginalizes to the observable one given by $P({ab}| {xy})=P({a}_{x},{b}_{y})$. Hence, the appropriate probability structure for local correlations in the Bell scenario is ${ \mathcal S }=\{S\}$ with $S=\{{A}_{1},\ldots ,{A}_{| { \mathcal X }| },{B}_{1},\,\ldots ,\,{B}_{| { \mathcal Y }| }\}$, and we consider the Shannon cone ${{\rm{\Gamma }}}_{S}={{\rm{\Gamma }}}^{{ \mathcal S }}$ that contains all ${2}^{| { \mathcal X }| +| { \mathcal Y }| }$-dimensional entropy vectors ${\boldsymbol{h}}=(H(\varnothing ),H({A}_{1}),\,\ldots ,\,H({B}_{1}),\,\ldots ,\,H({A}_{1}\ldots {A}_{| { \mathcal X }| }{B}_{1}\ldots {B}_{| { \mathcal Y }| }))$. The marginal scenario in this case is simply ${ \mathcal M }={\{\{{A}_{x},{B}_{y}\}\}}_{x,y}$ and local correlations are then characterized by the cone ${{\rm{\Pi }}}_{{ \mathcal M }}({{\rm{\Gamma }}}_{S})$.

In contrast to the characterization based directly on the DAG variables, this approach leads to nontrivial entropic inequalities (i.e., not obtainable from the elemental inequalities in equation (5)) in the Bell scenario. For example, for two measurement settings per party, which we label in this case x, y = 0, 1, one obtains the Braunstein–Caves inequality [26] together with its symmetries obtained by relabeling the inputs, namely,

$$\begin{eqnarray}&&I({A}_{0}:{B}_{0})+I({A}_{0}:{B}_{1})+I({A}_{1}:{B}_{0})-I({A}_{1}:{B}_{1})-H({A}_{0})-H({B}_{0})\leqslant 0,\\ &&\,\end{eqnarray} \tag{ 11 }$$

where $I({A}_{x}:{B}_{y}):= H({A}_{x})+H({B}_{y})-H({A}_{x}{B}_{y})$ is the mutual information between the variables A_x and B_y. This inequality can be understood as the entropic counterpart of the paradigmatic CHSH inequality [47].

Although the choice of probability structure above corresponds, via Fine's theorem, to the assumption of a local hidden variable theory, one can also consider other possibilities. For instance, taking ${ \mathcal S }={ \mathcal M }$ amounts to assuming a nonsignaling theory [40]. In this case, the entropy cone is characterized only by the Shannon inequalities and one can obtain a characterization of the extremal rays of the cone, corresponding to the entropic analog of Popescu–Rohrlich boxes [37].

In general (i.e., beyond the simplest Bell scenario), both methods based on the variables in a causal Bayesian network and on counterfactual variables can lead to nontrivial constraints [9, 29, 35, 37, 48, 49]. To conclude this section, let us nonetheless highlight an important difference between the two methods: while the former is valid for arbitrary input alphabets, the latter fixes the number of inputs to which the inequalities apply.

The ability to apply the entropic approach to DAGs, as outlined in section 2.2.4, is a powerful tool for characterizing the correlations obtainable within arbitrary causal networks. However, the notion of causal correlations defined in equation (1) is somewhat more general and cannot be directly expressed within the framework of causal Bayesian networks. In order to see why this is the case, let us first note that the random variables of interest are $X,Y,A,B$, representing the inputs $X,Y$ and outputs $A,B$ for Alice and Bob. Note that since we consider signaling scenarios here, unlike in the Bell scenario, we do not need to include any latent variable Λ in our description to account for shared randomness, since this can be established via local randomness and communication.

If Alice and Bob share a correlation compatible with a fixed causal order (i.e. either ${\rm{A}}\prec {\rm{B}}$ or ${\rm{B}}\prec {\rm{A}}$), then the functional dependences between $X,Y,A,B$ can indeed be expressed as a DAG—specifically, the two DAGs containing these variables in figure 2. However, a causal correlation may in general not be compatible with any fixed causal order, but may require a mixture thereof. This has some similarities with the situation in the Svetlichny definition of genuine multipartite nonlocality [50, 51] where a convex mixture of different DAGs has to be considered.

Zoom In Zoom Out Reset image size

To tackle this problem it is necessary to find a way to take into account the constraints arising separately from each of the two fixed causal orders, and then to combine them to obtain those satisfied by causal correlations. In order to do this, we exploit the fact that any mixture of fixed-order causal correlations can be seen as arising from a latent variable that determines the causal order for each individual experiment [23]. We thus introduce a new random variable Q which we call a 'switch', and which determines univocally the appropriate causal Bayesian network for each trial. The resulting causal model is shown in figure 2, where the DAG with ${\rm{A}}\prec {\rm{B}}$ is used for Q = 0, and the one with ${\rm{B}}\prec {\rm{A}}$ for Q = 1. By identifying ${q}_{0},{q}_{1}$ in equation (1) as ${q}_{0}=P(Q=0)$, and ${q}_{1}=P(Q=1)$, one can readily see that this description is equivalent to the definition of causal correlations in equation (1).

Both DAGs imply the independence of the inputs, $X\perp \perp Y$. The DAG for Q = 0 (i.e., for ${\rm{A}}\prec {\rm{B}}$) also implies the CI relation $A\perp \perp Y| X$ (i.e. that there is no signaling from ${\rm{B}}$ to ${\rm{A}}$), while the DAG for Q = 1 implies $B\perp \perp X| Y$ instead. In addition, the switch variable Q should be independent of Alice and Bob's inputs X and Y, so that we have ${XY}\perp \perp Q$, which, together with $X\perp \perp Y$, implies that $X\perp \perp Y\perp \perp Q$.

In order to use the 'conditional' causal Bayesian network in figure 2 to characterize the set of entropy vectors obtainable from causal correlations, we first note that we can directly use the techniques of section 2.2.4 to construct the Shannon cones for each of the two DAGs appearing in the figure conditioned on Q (i.e., for fixed-order correlations with ${\rm{A}}\prec {\rm{B}}$ or ${\rm{B}}\prec {\rm{A}}$). Denoting these cones ${{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}$ and ${{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}$, we have

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}={{\rm{\Gamma }}}_{{S}}\cap {{L}}_{{ \mathcal C }}^{{\rm{A}}\prec {\rm{B}}}\end{eqnarray} \tag{ 12 }$$

and

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}={{\rm{\Gamma }}}_{{S}}\cap {{L}}_{{ \mathcal C }}^{{\rm{B}}\prec {\rm{A}}},\end{eqnarray} \tag{ 13 }$$

where ${{\rm{\Gamma }}}_{{S}}$ is the Shannon cone for the four variables in $S=\{X,Y,A,B\}$, the probability structure is simply ${ \mathcal S }=\{S\}$, and ${{L}}_{{ \mathcal C }}^{{\rm{A}}\prec {\rm{B}}}$ denotes the linear subspace defined by the CI constraints for the case ${\rm{A}}\prec {\rm{B}}$, namely, the equations $H({XY})=H(X)+H(Y)$ and $H({YA}| X)=H(Y| X)+H(A| X)$, and similarly for ${{L}}_{{ \mathcal C }}^{{\rm{B}}\prec {\rm{A}}}$. These cones are characterized by the systems of inequalities ${I}_{0}{\boldsymbol{h}}\leqslant {\bf{0}}$ and ${I}_{1}{\boldsymbol{h}}\leqslant {\bf{0}}$, where ${\boldsymbol{h}}={(H(T))}_{T\subset S}$.

Recall that in the probabilistic case the polytope of causal correlations is simply the convex hull of the polytopes of correlations for ${\rm{A}}\prec {\rm{B}}$ and ${\rm{B}}\prec {\rm{A}}$ [24], and with the new variable Q the definition in equation (1) can be rewritten as

$$\begin{eqnarray}&&\begin{array}{r}P({ab}| {xy})\,=\,P(Q=0){P}^{{\rm{A}}\prec {\rm{B}}}({ab}| {xy},Q=0)+P(Q=1){P}^{{\rm{B}}\prec {\rm{A}}}({ab}| {xy},Q=1).\\ & & \end{array}\end{eqnarray} \tag{ 14 }$$

In contrast, the convex hull of the cones ${{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}$ and ${{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}$ does not contain all entropy vectors of causal correlations due to the concavity of the Shannon entropy. Indeed, in appendix A we provide an explicit example of a causal correlation whose entropy vector is not contained in the convex hull $\mathrm{conv}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$.

To see more precisely why this is the case, and how to give a correct entropic characterization of causal correlations, observe that, when taking a convex mixture of two causal correlations with different causal orders, the 'conditional entropy vectors' ${{\boldsymbol{h}}}_{0}={(H(T| Q=0))}_{T\subset S}$ and ${{\boldsymbol{h}}}_{1}={(H(T| Q=1))}_{T\subset S}$ must be contained in ${{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}$ and ${{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}$, respectively, and thus satisfy ${I}_{0}{{\boldsymbol{h}}}_{0}\leqslant {\bf{0}}$ and ${I}_{1}{{\boldsymbol{h}}}_{1}\leqslant {\bf{0}}$. For any causal correlation, the convex mixture

$$\begin{eqnarray}&&{{\boldsymbol{h}}}_{\mathrm{conv}}=P(Q=0){{\boldsymbol{h}}}_{0}+P(Q=1){{\boldsymbol{h}}}_{1}\end{eqnarray} \tag{ 15 }$$

is thus contained in $\mathrm{conv}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$. Observe now that, in contrast to the convex sum (14) defining causal correlations, ${{\boldsymbol{h}}}_{\mathrm{conv}}$ thus defined is equal to ${(H(T| Q))}_{T\subset S}$, rather than just ${(H(T))}_{T\subset S}$, and hence the convex hull of the fixed-order cones characterizes the conditional entropies (conditioned on the switch variable Q) obtainable with causal correlations, rather than the entropy vectors of causal correlations directly.

With the appropriate transformation, the inequalities $I{\boldsymbol{h}}\leqslant {\bf{0}}$ characterizing¹¹ $\mathrm{conv}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$ can be transformed into inequalities satisfied by the standard (i.e., nonconditional) entropy vector $\tilde{{\boldsymbol{h}}}={(H(T))}_{T\subset \tilde{S}}$ for the variables now in $\tilde{S}=S\cup \{Q\}$ (and the probability structure is consequently extended to $\widetilde{{ \mathcal S }}=\{\tilde{S}\}$). Specifically, each row ${\boldsymbol{I}}$ of the matrix I (defining each individual inequality ${\boldsymbol{I}}\cdot {\boldsymbol{h}}\leqslant 0$) must undergo the linear transformation ${{ \mathcal T }}_{Q}:{{\mathbb{R}}}^{{2}^{| S| }}\to {{\mathbb{R}}}^{{2}^{| S| +1}}$ mapping ${\boldsymbol{I}}\mapsto \tilde{{\boldsymbol{I}}}:= {{ \mathcal T }}_{Q}({\boldsymbol{I}})$ with the components of $\tilde{{\boldsymbol{I}}}$ given by¹²

$$\begin{eqnarray}&&{(\tilde{{\boldsymbol{I}}})}_{T\cup \{Q\}}={({\boldsymbol{I}})}_{T},{(\tilde{{\boldsymbol{I}}})}_{\{Q\}}=-\displaystyle \sum _{T\ne \varnothing }{({\boldsymbol{I}})}_{T},\ \mathrm{and}\ {(\tilde{{\boldsymbol{I}}})}_{T}=0\end{eqnarray} \tag{ 16 }$$

for all nonempty subsets $T\subset S$. We will denote by ${\mathrm{conv}}_{Q}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$ the cone of vectors $\tilde{{\boldsymbol{h}}}$ satisfying the resulting inequalities $\tilde{I}\tilde{{\boldsymbol{h}}}\leqslant {\bf{0}}$.

To complete the characterization of entropy vectors for causal correlations, we recall that, in addition to the fact that any distribution on $\tilde{{S}}$ must give an entropy vector in the Shannon cone ${{\rm{\Gamma }}}_{\tilde{{S}}}$, the conditional DAG in figure 2 gives us the CI constraints $X\perp \perp Y\perp \perp Q$. Moreover, since Q is a binary variable (as there are only two orders to switch between) we have $H(Q)\leqslant 1$. A consequence of this final inequality constraint is that the set of entropy vectors under consideration will be characterized by an inhomogeneous system of inequalities of the form $\tilde{I}\tilde{{\boldsymbol{h}}}\leqslant \tilde{{\boldsymbol{\beta }}}$ for some $\tilde{{\boldsymbol{\beta }}}\in {{\mathbb{R}}}^{{2}^{| S| +1}}$ and is thus no longer a cone but a polyhedron. The polyhedron characterizing entropy vectors associated with the conditional DAG (when still including Q) is thus given by

$$\begin{eqnarray}&&\begin{array}{r}{\widetilde{{\rm{\Gamma }}}}_{\mathrm{AB}}^{\mathrm{causal}}\,=\,{{\rm{\Gamma }}}_{\tilde{{\rm{S}}}}\cap {\mathrm{conv}}_{Q}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})\,\cap \,{{L}}_{{ \mathcal C }}(\{(X\perp \perp Y\perp \perp Q),H(Q)\leqslant 1\}),\\ & & \end{array}\end{eqnarray} \tag{ 17 }$$

where the notation ${{L}}_{{ \mathcal C }}(\cdot )$ denotes the subset (here, a polyhedron) in the entropy vector space defined by the corresponding linear constraints.

Finally, following the general approach presented in section 2.2, it remains just to eliminate the terms containing the (unobservable) switch variable Q in order to obtain the inequalities characterizing bipartite causal correlations. This is done by projecting ${\widetilde{{\rm{\Gamma }}}}_{\mathrm{AB}}^{\mathrm{causal}}$ onto the marginal scenario ${ \mathcal M }=\{S\}=\{\{X,Y,A,B\}\}$. We thus finally obtain the polyhedron

$$\begin{eqnarray}&&{{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}={{\rm{\Pi }}}_{{ \mathcal M }}({\widetilde{{\rm{\Gamma }}}}_{\mathrm{AB}}^{\mathrm{causal}}),\end{eqnarray} \tag{ 18 }$$

which we shall refer to as the causal Shannon polyhedron or simply the causal polyhedron and is again characterized by an inhomogeneous system of inequalities $I^{\prime} {\boldsymbol{h}}\leqslant {\boldsymbol{\beta }}$ for some ${\boldsymbol{\beta }}\in {{\mathbb{R}}}^{{2}^{| S| }}$.

We emphasize that the construction given above is in fact not at all restricted to the description of causal correlations, and can be used to characterize arbitrary convex mixtures of different Bayesian networks. Furthermore, as we will see in section 4, this method can be generalized to convex combinations of more distributions, in our case corresponding to more than two causal orders in multipartite scenarios (and even correlations with 'dynamical causal order' [23, 25, 53]).

The constructive description of the causal polyhedron ${{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}$ from equations (17) and (18) also makes it clear how we can characterize it, in practice, as a system of linear inequalities. A description of ${\widetilde{{\rm{\Gamma }}}}_{\mathrm{AB}}^{\mathrm{causal}}$ in terms of its facets is straightforwardly obtained by taking the union of the inequalities describing the individual cones and polyhedron appearing in equation (17) and eliminating redundant ones. The inequalities characterizing ${{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}$ can then be found by eliminating the terms not contained in the marginal scenario ${ \mathcal M }=\{S\}$, either by Fourier–Motzkin elimination [43] or by finding its extremal rays and projecting out the unwanted coordinates.

The resulting system of inequalities is thus satisfied by any bipartite causal correlation. However, many of these inequalities are either elemental inequalities (as in equation (5)) or can be obtained from these by using the independence constraint $X\perp \perp Y$, and thus represent trivial constraints. After characterizing the polyhedron in equation (18) and eliminating all trivial inequalities, i.e., those satisfied by any distribution P(xyab) with $X\perp \perp Y$, we find 35 novel entropic causal inequalities. Several of these inequalities are equivalent under the exchange of parties (i.e., exchanging $(X,A)\leftrightarrow (Y,B)$), and under this symmetry there are in fact 20 equivalence classes of entropic causal inequalities, the full list of which is given in appendix B. Of these, 10 have bounds of 0 (i.e., are of the form ${\boldsymbol{I}}\cdot {\boldsymbol{h}}\leqslant 0$), while the remaining 10 have nonzero bounds (resulting from a nontrivial dependence on H(Q) before this variable was eliminated; see appendix B). Simple interpretations of the entropic causal inequalities seem to be less forthcoming than for the bipartite causal inequalities in terms of probabilities [24] (for binary inputs and outputs—recall that the entropic inequalities given here are, in contrast, valid for any number of possible inputs and outputs). One of the simpler examples, which is symmetric under the exchange of parties, is

$$\begin{eqnarray}&&I(X:{YA})+I(Y:{XB})-H({AB})\leqslant 0.\end{eqnarray} \tag{ 19 }$$

Note that the fact that we find nontrivial inequalities is in stark contrast to the situation for Bell-type inequalities (and line-like causal Bayesian networks), where the DAG-based entropic method only leads to trivial inequalities obtainable from the elemental inequalities and no-signaling conditions [45].

While these entropic inequalities are obeyed by any bipartite causal correlation, we note that a priori they need not be tight. Indeed, recall that the Shannon cone is only an outer approximation to the true entropy cone, so it is thus interesting to study the tightness and violation of these inequalities more carefully.

Although one generally would not expect every point on the boundary of ${{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}$ to be obtainable by a causal correlation, it is nonetheless desirable to be able to saturate each inequality by some causal probability distribution for appropriate distributions for X and Y. By looking at deterministic causal distributions with binary inputs and outputs, which can easily be enumerated, we readily verified that all 10 families of inequalities that are bounded by 0 (given in equation (B1)) can indeed be saturated when taking uniformly distributed inputs. However, we were unable to find causal distributions, either by mixing binary ones or by considering more outputs, that saturate the remaining inequalities, and their tightness remains an open question.

To understand now the violation by noncausal distributions of the entropic inequalities, we consider the extremal rays of the constrained Shannon cone

$$\begin{eqnarray}&&{{\rm{\Gamma }}}_{S}\cap {{\rm{L}}}_{{ \mathcal C }}(\{X\perp \perp Y\})\end{eqnarray} \tag{ 20 }$$

which violate the inequalities¹³ . A crucial question is whether or not these extremal rays actually correspond to valid probability distributions (i.e., whether they support entropy vectors), and if not, whether the inequalities can nonetheless be violated.

In order to look at this, it is instructive to first restrict our attention to distributions satisfying $H(X)\leqslant 1$, $H(Y)\leqslant 1$, $H(A)\leqslant 1$ and $H(B)\leqslant 1$. These constraints are satisfied by all distributions with binary inputs and outputs, and this therefore also allows us to compare the violation of the entropic causal inequalities to the violation of standard causal inequalities that are understood well in this scenario [24]. Imposing these constraints on the cone in equation (20), one obtains a polytope with extremal points corresponding to the extremal rays of the cone scaled to satisfy these constraints (together with the null vertex ${\bf{0}}$). Under these we found that the 10 inequalities in equation (B1) and the two inequalities in equation (B2) could be violated, although the latter are (in that case) weaker than, and implied by, the former and are thus redundant. The remaining 8 inequalities in equations (B3) and (B4) cannot be violated. All in all, the set of binary causal correlations is entropically characterized by the 10 inequalities in equation (B1) that are bounded by 0.

Amongst the extremal points violating each of these inequalities, those that give the maximal violation all satisfy $H(X)=H(Y)=1$ and $H({XY})=H({XYAB})$ and thus, if realizable, correspond to deterministic conditional distributions taken with uniformly distributed inputs X and Y¹⁴ . In fact, all but one of these 10 inequalities are maximally violated (by which we henceforth mean with respect to the Shannon cone augmented with the independence constraint $X\perp \perp Y$) by one of the three following deterministic distributions taken with uniform inputs:

$$\begin{eqnarray}P({ab}| {xy}) & = & {\delta }_{a,y}\,{\delta }_{b\oplus y,x}\\ P({ab}| {xy}) & = & {\delta }_{a\oplus x,y}\,{\delta }_{b,x}\\ P({ab}| {xy}) & = & {\delta }_{a\oplus x,y}\,{\delta }_{b\oplus y,x},\end{eqnarray} \tag{ 21 }$$

where $x,y,a,b$ take the binary values $0,1$, and $\oplus$ denotes addition modulo 2. For example, equation (19) is violated by the third distribution above with a value for the left-hand side of 1. The one exception not violated by the distributions in equation (21) is the second inequality in (B1),

$$\begin{eqnarray}&&I(A:B)-I(A:B| X)-I(A:B| Y)-2H({AB}| {XY})\leqslant 0,\end{eqnarray} \tag{ 22 }$$

which, in turn, is violated by the deterministic distribution (again taken with uniform inputs)

$$\begin{eqnarray}&&P({ab}| {xy})={\delta }_{a\oplus x,{xy}}\,{\delta }_{b\oplus y,{xy}}.\end{eqnarray} \tag{ 23 }$$

However, unlike for the other inequalities, this distribution does not give the maximal possible violation of inequality (22) (which is 1/2), as the corresponding extremal point ${{\boldsymbol{h}}}_{\mathrm{ext}}$ that does maximally violate it is not reachable by a valid probability distribution with binary inputs and outputs. This is easily verified by making use of the previous observation that this extremal point must correspond to a deterministic distribution taken with uniform inputs, the set of which can easily be enumerated for binary inputs and outputs. Amongst such distributions, the one in equation (23) gives the best violation of $1-\tfrac{3}{2}{\mathrm{log}}_{2}\tfrac{3}{2}\approx 0.123\gt 0$.

The distributions in equation (21) are particularly interesting, as they all violate maximally some symmetries of the GYNI inequality (2) (under relabeling of the inputs and outputs), but not equation (2) itself. Interestingly, it turns out that all binary deterministic noncausal distributions, when taken with uniform inputs, violate at least one of our entropic inequalities except the distribution ${P}^{\mathrm{GYNI}}({ab}| {xy})={\delta }_{a,y}{\delta }_{b,x}$ (which violates maximally equation (2)) and its four symmetries under input-independent relabeling of outputs only. Note, however, that if Alice and Bob have a noncausal resource producing the distribution ${P}^{\mathrm{GYNI}}$, they can produce any of the distributions in equation (21) by appropriately XORing their input with their output, and thus still obtain an operational violation of an entropic causal inequality¹⁵ . It is interesting to observe that distributions maximally violating GYNI-type inequalities have such a crucial role in violating the entropic causal inequalities given that the entropic inequalities superficially bear little resemblance to these, and are valid for arbitrary numbers of inputs and outputs.

Returning to the more general situation with no upper bound imposed on $H(X),H(Y),H(A)$ and H(B), we see that all the remaining entropic causal inequalities can be violated by entropy vectors that are parallel to the realizable entropy vectors giving violations in the restricted scenario—more precisely, those obtained from the distributions in equation (21) (for all but one of the remaining inequalities) and equation (23) (for the remaining one). This shows that, given large enough alphabets for the input and output variables, all the entropic causal inequalities we obtained can indeed be violated by noncausal probability distributions, since if the distribution P(xyab) has entropy vector ${\boldsymbol{h}}$ then the distribution

$$\begin{eqnarray}&&P({\boldsymbol{x}}{\boldsymbol{y}}{\boldsymbol{a}}{\boldsymbol{b}})=P({x}_{1}{y}_{1}{a}_{1}{b}_{1})\,\times \cdots \times \,P({x}_{n}{y}_{n}{a}_{n}{b}_{n}),\end{eqnarray} \tag{ 24 }$$

where ${\boldsymbol{x}}=({x}_{1},\,\ldots ,\,{x}_{n})$ and similarly for ${\boldsymbol{y}}$, ${\boldsymbol{a}}$ and ${\boldsymbol{b}}$, has entropy vector $n\cdot {\boldsymbol{h}}$ ¹⁶ . One should be careful, however, to note that the operation of sharing multiple independent correlations among the same parties is not a free operation either in the framework of causal correlations (since, for example, two independent copies of a causal distribution may give rise to a noncausal one), or in the process matrix framework (where two independent copies of a process matrix does not, in general, produce a valid process matrix). Nevertheless, $P({\boldsymbol{a}}{\boldsymbol{b}}| {\boldsymbol{x}}{\boldsymbol{y}})=P({\boldsymbol{x}}{\boldsymbol{y}}{\boldsymbol{a}}{\boldsymbol{b}})/P({\boldsymbol{x}}{\boldsymbol{y}})$ obtained from equation (24) still represents a valid (possibly noncausal) distribution.

It is interesting also to ask how sensitive the entropic causal inequalities are for detecting noncausality. Since it does not appear possible to saturate the inequalities (B2)–(B4) with nonzero bounds using causal distributions, these inequalities are not tight and, consequentially, unable to detect noncausal correlations that are very close to being causal. For the other inequalities in equation (B1) this is nonetheless a pertinent question. More precisely, one may ask whether there exists a distribution ${P}^{\varepsilon }$ of the form

$$\begin{eqnarray}&&{P}^{\varepsilon }({ab}| {xy})=\varepsilon {P}^{\mathrm{NC}}({ab}| {xy})+(1-\varepsilon ){P}^{{\rm{C}}}({ab}| {xy}),\end{eqnarray} \tag{ 25 }$$

where ${P}^{\mathrm{NC}}$ is a noncausal distribution and ${P}^{{\rm{C}}}$ is causal, that violates any of these entropic inequalities for arbitrarily small $\varepsilon \gt 0$.

We looked in detail at this question for the case of binary inputs and outputs, where the inequalities in equation (B1) can all both be saturated by causal distributions, and violated by noncausal ones. By trying exhaustively all deterministic distributions ${P}^{\mathrm{NC}}$ and ${P}^{{\rm{C}}}$, we found that such behavior was exhibited (for such distributions) only by the two inequalities

$$\begin{eqnarray}&&I(A:B| X)-I(Y:B)-2H(B| {XY})\leqslant 0\end{eqnarray} \tag{ 26 }$$

and

$$\begin{eqnarray}&&I({XA}:Y)+I({YB}:X)-H(X| {YA})-H(A)\leqslant 0.\end{eqnarray} \tag{ 27 }$$

Equation (26), for example, is violated by ${P}^{\varepsilon }$ for all $\varepsilon \gt 0$ when taking ${P}^{\mathrm{NC}}({ab}| {xy})={\delta }_{a\oplus x,y}\,{\delta }_{b\oplus y,x}$ and ${P}^{{\rm{C}}}({ab}| {xy})={\delta }_{a,0}\,{\delta }_{b\oplus y,x}$ along with uniformly distributed inputs X and Y, which also gives a violation of the GYNI-type causal inequality

$$\begin{eqnarray}&&\displaystyle \frac{1}{4}\displaystyle \sum _{x,y,a,b}{\delta }_{a\oplus x,y}\,{\delta }_{b\oplus y,x}\,P({ab}| {xy})\leqslant \displaystyle \frac{1}{2}\end{eqnarray} \tag{ 28 }$$

with a left-hand side value of $\tfrac{1+\varepsilon }{2}\gt \tfrac{1}{2}$.

For the remaining inequalities, such mixtures that violate a standard causal inequality for arbitrarily small ε only violate an entropic causal inequality when $\varepsilon \gt {\varepsilon }_{0}$ for some ${\varepsilon }_{0}$ bounded away from 0. We observed identical behavior when we extended our consideration also to various nondeterministic distributions ${P}^{\mathrm{NC}}$ and ${P}^{{\rm{C}}}$, and it thus seems that only equations (26) and (27) exhibit this ability to detect the noncausality of distributions that are arbitrarily close to being causal.

A final point worth discussing relates to the physical interpretation of the distributions violating entropic causal inequalities. One of the motivations in introducing the notion of causal correlations was whether nature permits more general causal structures that might allow such correlations to be realized, for example in quantum gravity. In particular, the authors of [14] introduced the so-called process matrix formalism, in which quantum mechanics is assumed to hold locally for each party, while no global order is assumed between the parties. They showed that causal inequalities can be violated within this framework, and this helped motivate further studies of causal and noncausal correlations, where it has been shown that the violation of causal inequalities is ubiquitous within this framework [21, 22, 24, 25, 54, 55]. It is thus interesting to see whether entropic causal inequalities share this property and can also be violated within the process matrix framework.

To look for such violations, we used the optimization techniques of [24, 25] with qubit systems to try and optimize the violation of the GYNI-type inequalities that the distributions in equation (21) violate maximally. We also tried minimizing the distance to other deterministic noncausal correlations such as equation (23), as well as optimizing in random directions in probability space. Unfortunately, we were unable to find any process matrices operating on qubits that violate our entropic causal inequalities with such techniques. We additionally attempted to reproduce (as closely as possible) distributions of the form (25) for small ε in order to violate inequalities (26) and (27), but similarly found no violation. Finally, we looked at correlations obtained by mixing noncausal correlations realizable by process matrices with causal correlations. An analogous mixing procedure was shown to enable all nonlocal distributions to violate the entropic Bell inequalities described in section 2.2.5 [31], but we were unable to find violations of any entropic causal inequalities with this approach.

This lack of violation is perhaps unsurprising given the general lack of sensitivity of the entropic inequalities to nearly causal distributions, and the fact that the best-known violations of causal inequalities for this scenario with process matrices are relatively small [24]. Nonetheless, it remains possible that violations can be found with higher-dimensional systems or more inputs and outputs; we leave this as an open question.

To keep the discussion simple, we will consider only the case of binary inputs, but the generalization to arbitrary inputs is straightforward. We thus consider the variables in

$$\begin{eqnarray}&&S=\{{A}_{00},{A}_{01},{A}_{10},{A}_{11},{B}_{00},{B}_{01},{B}_{10},{B}_{11}\}.\end{eqnarray} \tag{ 29 }$$

Note that, in contrast to the example of Bell inequalities discussed in section 2.2.5, we need to consider copies of each variable for each input pair (x, y). This is a consequence of the fact that the correlations which we want to characterize may be signaling, e.g., for the causal order ${\rm{A}}\prec {\rm{B}}$, B₀₀ and B₁₀ will in general be different.

Since A_xy and ${B}_{x^{\prime} y^{\prime} }$ are jointly observable only if $x=x^{\prime}$ and $y=y^{\prime}$, the marginal scenario in this case is

$$\begin{eqnarray}&&{ \mathcal M }=\{\{{A}_{00},{B}_{00}\},\{{A}_{01},{B}_{01}\},\{{A}_{10},{B}_{10}\},\{{A}_{11},{B}_{11}\}\}.\end{eqnarray} \tag{ 30 }$$

In contrast to the DAG-based method, several choices of probability structure ${ \mathcal S }$ compatible with ${ \mathcal M }$ are possible, and the particular choice must be motivated on the basis of physical assumptions. One natural possibility would be to take ${ \mathcal S }={ \mathcal M }$, as one may have no a priori reason to think that the variables A_xy and ${A}_{x^{\prime} y^{\prime} }$ have simultaneous physical meaning for $(x,y)\ne (x^{\prime} ,y^{\prime} )$, and hence may not have a well-defined joint probability distribution. On the other hand, in some cases one may imagine that such inputs correspond to the choice of measurements of some physical properties that are simultaneously well-defined, as in a classical theory; hence, one may alternatively take ${ \mathcal S }=\{{\cup }_{{M}_{j}\in { \mathcal M }}{M}_{j}\}=\{S\}$. In the following, we will adopt the former approach and take ${ \mathcal S }={ \mathcal M }$, since this constitutes the minimum assumptions compatible with the marginal scenario. The Shannon cone for ${ \mathcal S }$ is thus

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{ \mathcal S }}={{\rm{\Gamma }}}_{\{{A}_{00},{B}_{00}\}}\cap {{\rm{\Gamma }}}_{\{{A}_{01}{B}_{01}\}}\cap {{\rm{\Gamma }}}_{\{{A}_{10}{B}_{10}\}}\cap {{\rm{\Gamma }}}_{\{{A}_{11}{B}_{11}\}},\end{eqnarray} \tag{ 31 }$$

as in equation (8). We note however that this physically motivated choice for ${ \mathcal S }$ implies, for this particular scenario, that a global probability distribution does in fact exist¹⁷ . Taking ${ \mathcal S }=\{S\}$ would thus provide an equivalent entropic characterization, and moreover, this equivalence also holds at the level of Shannon (rather than entropy) cones (see appendix C for a more detailed discussion).

We follow a method analogous to that used in section 3.1. First, we characterize the cones ${{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}$ and ${{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}$ of entropy vectors for fixed-order causal correlations, then, we characterize the convex mixtures of such correlations.

To do this, we note that the no-signaling conditions obeyed by fixed-order correlations (see section 2.1) impose constraints on the counterfactual variables. For example, correlations consistent with the order ${\rm{A}}\prec {\rm{B}}$ obey $P(a| {xy})=P(a| {xy}^{\prime} )$ for all $x,y,y^{\prime} ,a$, which implies ${A}_{{xy}}={A}_{{xy}^{\prime} }$ and thus $H({A}_{{xy}})=H({A}_{{xy}^{\prime} })$ also. Similarly, for ${\rm{B}}\prec {\rm{A}}$, we have $H({B}_{{xy}})=H({B}_{x^{\prime} y})$ for all $x,x^{\prime} ,y$. The cones ${{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}$ and ${{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}$ are thus given by

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}}={{\rm{\Gamma }}}^{{ \mathcal S }}\cap {{L}}_{{ \mathcal C }}(\{{A}_{00}={A}_{01},\ {A}_{10}={A}_{11}\})\end{eqnarray} \tag{ 32 }$$

and

$$\begin{eqnarray}&&{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}}={{\rm{\Gamma }}}^{{ \mathcal S }}\cap {{L}}_{{ \mathcal C }}(\{{B}_{00}={B}_{10},\ {B}_{01}={B}_{11}\}),\end{eqnarray} \tag{ 33 }$$

where ${{L}}_{{ \mathcal C }}(\cdot )$ again denotes the linear subspace defined by the corresponding constraints.

As in section 3.1, we introduce the latent switch variable Q, denote the augmented set of random variables $\tilde{S}=S\cup \{Q\}$, and extend the probability structure as

$$\begin{eqnarray}&&\widetilde{{ \mathcal S }}=\{\{{A}_{{xy}},{B}_{{xy}},Q\}| \,x,y\in \{0,1\}\}\end{eqnarray} \tag{ 34 }$$

(in appendix C we discuss further the implications of different choices of probability structures). With this extra variable we note again that the convex hull $\mathrm{conv}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$ contains the vectors ${{\boldsymbol{h}}}_{\mathrm{conv}}={(H(T| Q))}_{T\in {{ \mathcal S }}^{{\rm{c}}}}$ for causal correlations. The system of inequalities $I{\boldsymbol{h}}\leqslant {\bf{0}}$ characterizing $\mathrm{conv}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$ can then again be transformed in a similar way to equation (16) into a new system $\tilde{I}\tilde{{\boldsymbol{h}}}\leqslant {\bf{0}}$ defining the cone of corresponding entropy vectors $\tilde{{\boldsymbol{h}}}={(H(T))}_{T\in {\widetilde{{ \mathcal S }}}^{{\rm{c}}}}$, which we again denote by ${\mathrm{conv}}_{Q}({{\rm{\Gamma }}}^{{\rm{A}}\prec {\rm{B}}},{{\rm{\Gamma }}}^{{\rm{B}}\prec {\rm{A}}})$. In contrast to the DAG-based method, the only constraint on Q is, now, $H(Q)\leqslant 1$, since Q need not be independent of the counterfactual output variables ${A}_{{xy}},{B}_{{xy}}$. Finally, we need to project onto the marginal scenario ${ \mathcal M }$ in equation (30). The causal polyhedron is thus given, in analogy to equations (17) and (18), by

$$\begin{eqnarray}&&\begin{array}{r}{{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}\,=\,{{\rm{\Pi }}}_{{ \mathcal M }}[{{\rm{\Gamma }}}^{\widetilde{{ \mathcal S }}}\cap {\mathrm{conv}}_{Q}({{\rm{\Gamma }}}^{A\prec B},{{\rm{\Gamma }}}^{B\prec A})\cap {L}_{{ \mathcal C }}(\{H(Q)\leqslant 1\})],\\ & & \end{array}\end{eqnarray} \tag{ 35 }$$

where we have ${{\rm{\Gamma }}}^{\widetilde{{ \mathcal S }}}={\bigcap }_{x,y\in \{\mathrm{0,1}\}}{{\rm{\Gamma }}}_{\{{A}_{{xy}},{B}_{{xy}},Q\}}$.

As in section 3.1, the construction above allows one to obtain the full list of entropic inequalities characterizing ${{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}$. After removing the trivial inequalities directly implied by Shannon constraints on ${ \mathcal M }$, we find that there are 6 nontrivial entropic causal inequalities, which can be grouped into two equivalence classes of inequalities under the relabeling of inputs:

$$\begin{eqnarray}&&I({A}_{00}:{B}_{00})-H({A}_{01})-H({B}_{10})\leqslant 1\end{eqnarray} \tag{ 36 }$$

and

$$\begin{eqnarray}&&\begin{array}{l}I({A}_{00}:{B}_{00})+I({A}_{11}:{B}_{11})-H({A}_{01}{B}_{01})-H({A}_{10}{B}_{10})\leqslant 2.\\ \end{array}\end{eqnarray} \tag{ 37 }$$

The fact that these inequalities have nontrivial bounds is, as for the DAG-based method, a result of the constraint $H(Q)\leqslant 1$ which means ${{\rm{\Gamma }}}_{\mathrm{AB}}^{\mathrm{causal}}$ is a polyhedron characterized by a set of inhomogeneous inequalities. Indeed, if one chooses not to eliminate Q from the entropic description, one obtains a convex cone characterized by the above equations, except that the right-hand sides are multiplied by H(Q) (see the discussion in appendix B).

In contrast to the case for the DAG-based approach, where violation of the causal inequalities we obtained was possible even with deterministic distributions, it is clear that such distributions provide no interesting behavior in the counterfactual approach since any such distribution will have a null entropy vector. By looking at equal mixtures of deterministic causal distributions ${P}^{{\rm{A}}\prec {\rm{B}}}$ and ${P}^{{\rm{B}}\prec {\rm{A}}}$, however, we were able to verify that the inequalities in equations (36) and (37) can indeed be saturated by such (causal) distributions and are thus tight.

In order to study the potential violation of these entropic inequalities, we again need to look at nondeterministic distributions. One can easily see, however, that equations (36) and (37) cannot be violated when restricted to distributions satisfying $H({A}_{{xy}})\leqslant 1$ and $H({B}_{{xy}})\leqslant 1$ for all $x,y\in \{0,1\}$, as this also implies that $I({A}_{{xy}}:{B}_{{xy}})\leqslant 1$. This means that the inequalities for counterfactual variables are unable to detect noncausality when both parties are restricted to binary outputs.

To find violations we again look at the extremal rays of the Shannon cone ${{\rm{\Gamma }}}^{{ \mathcal S }}$ of equation (31) which violate one of the inequalities, and examine whether these rays can be reached by any probability distribution. Considering bounds on $H({A}_{{xy}})$ and $H({B}_{{xy}})$ strictly larger than 1, we find that violations are possible for any such bound. Moreover, the entropy vectors giving maximal violation of equations (36) and (37) are generally realizable with equal mixtures of causal and noncausal distributions. For example, given the constraints $H({A}_{{xy}})\leqslant {\mathrm{log}}_{2}k$ and $H({B}_{{xy}})\leqslant {\mathrm{log}}_{2}k$ for some integer $k\geqslant 2$, the distribution

$$\begin{eqnarray}&&{P}_{k}({ab}| {xy})={\delta }_{x,y}\ \displaystyle \frac{1}{k}{\delta }_{a,b}+(1-{\delta }_{x,y})\ {\delta }_{a,0}{\delta }_{b,0},\end{eqnarray} \tag{ 38 }$$

where $a,b\in \{0,\,\ldots ,\,k-1\}$, realizes such an extremal point for all $k\geqslant 2$, and provides a violation of both equations (36) and (37) for $k\gt 2$. For k = 2 (binary outputs), this distribution can be written as the convex combination

$$\begin{eqnarray}&&{P}_{2}({ab}| {xy})=\displaystyle \frac{1}{2}{P}^{\mathrm{NC}}({ab}| {xy})+\displaystyle \frac{1}{2}{P}^{{\rm{C}}}({ab}| {xy}),\end{eqnarray} \tag{ 39 }$$

where ${P}^{\mathrm{NC}}({ab}| {xy})={\delta }_{a\oplus x\oplus 1,y}{\delta }_{b\oplus y\oplus 1,x}$ maximally violates a GYNI-type inequality (it is simply a symmetry of the third distribution in equation (21), obtained by flipping all outputs), and ${P}^{{\rm{C}}}({ab}| {xy})={\delta }_{a,0}{\delta }_{b,0}$ is causal. Even though it does not violate equation (36) or (37), P₂ is noncausal. The distribution P_k can be seen as a possible generalization of a GYNI-violating distribution.

This link to the GYNI-type inequalities and correlations can be made more explicit by considering the related distribution

$$\begin{eqnarray}&&{P}_{k}^{{\prime} }({ab}| {xy})={\delta }_{x,y}\ \displaystyle \frac{1}{k-1}{\delta }_{a,b}(1-{\delta }_{a,0}{\delta }_{b,0})+(1-{\delta }_{x,y})\ {\delta }_{a,0}{\delta }_{b,0},\end{eqnarray} \tag{ 40 }$$

with again $a,b\in \{0,\,\ldots ,\,k-1\}$. We have ${P}_{2}^{{\prime} }={P}^{\mathrm{NC}}$, and, for $k\geqslant 3$, ${P}_{k}^{{\prime} }$ has the same entropy vector as ${P}_{k-1}$. ${P}_{k}^{{\prime} }$ can be clearly simulated from ${P}_{2}^{{\prime} }={P}^{\mathrm{NC}}$ by making use of shared randomness and by letting both parties replace the output 1 obtained from ${P}_{2}^{{\prime} }$ by a shared random value $a=b\in \{1,\,\ldots ,\,k-1\}$. It is interesting to see, then, that the GYNI-maximally violating distributions also provide the best behavior entropically when augmented with shared randomness, even though they fail to violate the inequalities when the parties have only binary outputs.

As for the DAG-based method, it is also interesting to look at the sensitivity of the inequalities with respect to the detection of noncausality. To do so, we again looked at distributions of the form given in equation (25), but where ${P}^{\mathrm{NC}}$ and ${P}^{{\rm{C}}}$ are now equal mixtures of 3-outcome deterministic noncausal and causal distributions, respectively. The number of such distributions makes an exhaustive search difficult, but by sampling randomly we were nonetheless able to verify the existence of distributions ${P}^{\varepsilon }({ab}| {xy})$ which violate the entropic inequalities (36) and (37) for arbitrary small ε. Although the examples we found are not particularly informative or simple (and thus we refrain from giving them explicitly), they nevertheless show that the entropic inequalities (36) and (37) exhibit the desired sensitivity.

Finally, one may again ask whether one can violate any of the entropic inequalities for counterfactuals within the process matrix formalism, or whether any noncausal correlation can be mixed with a causal one to violate an entropic inequality, as is the case for entropic Bell inequalities obtained from the counterfactual approach [31]. We leave this as an open question, but note only that we were not able to find a way to do so: for example, we were unable to find a violation (with or without the use of shared randomness) for noncausal distributions realizable within the process matrix framework.