One- and two-axis squeezing of atomic ensembles in optical cavities

Spin squeezed states of atomic ensembles have many applications as resources for quantum enhanced metrology [1–5], continuous variable quantum information processing [6], and multipartite entanglement [7–9]. Various methods for generating spin squeezed states in atomic ensembles have been proposed [10–15] and realized experimentally [4, 16–21]. In particular, cavity-based schemes where the light–matter interaction is enhanced by placing the atoms in an optical cavity have have shown impressive results and have realized highly squeezed states [5, 13, 21]. To take full advantage of these experimental advances and to ensure a continued increase in their capabilities, it is important to determine the ideal operation conditions and the squeezing attainable with such cavity based approaches.

A commonly used measure for the degree of squeezing in an ensemble is the possible gain in precision by using the squeezed state for interferometry. Wineland et al [22] showed that this can be quantified by

$$\begin{eqnarray}&&{\xi }^{2}=\mathop{\min }\limits_{\theta }\left(\displaystyle \frac{N(\langle {\hat{J}}_{\theta }^{2}\rangle -\langle {\hat{J}}_{\theta }{\rangle }^{2})}{\langle {\hat{J}}_{z}{\rangle }^{2}}\right),\end{eqnarray} \tag{ 1 }$$

where $\langle {\hat{J}}_{z}\rangle \approx N/2$ is the mean spin and ${\hat{J}}_{\theta }=\cos (\theta ){\hat{J}}_{x}+\sin (\theta ){\hat{J}}_{y}$. Here, ${\hat{J}}_{x,y,z}$ are the collective spin operators defined in the usual manner [22]. For ${\xi }^{2}\lt 1$ a gain in interferometric precision is possible compared to using a coherent spin state.

In general, cavity based schemes are known to exhibit a $1/\sqrt{{NC}}$ scaling of ${\xi }^{2}$ when limited by dissipation. Here, C is the single atom cooperativity (defined below) and N is the total number of atoms. This scaling is obtained as a tradeoff between the competing processes of the coherent evolution causing squeezing and the dissipative processes of spontaneous emission and cavity decay [11, 14, 15].

The squeezing parameter ${\xi }^{2}$ is, however, not a complete characterization of the dynamics. The precise figure of merit will depend on the application for which the squeezing operation is used, and so may the optimal method of squeezing. For example, if the objective is to prepare a specific squeezed state for metrology, dissipative schemes [12, 14] where the system is driven into a squeezed dark state may be beneficial. However, in continuous variable quantum information processing applications [6] where the objective is to implement a squeezing operation on a generic input state, coherent schemes [11, 13, 23–25] may be advantageous.

A demonstrated approach to coherent spin squeezing is to implement a one-axis twisting Hamiltonian [26]:

$$\begin{eqnarray}&&{\hat{H}}_{1-\mathrm{axis}}=\alpha {\hat{J}}_{\theta }^{2}.\end{eqnarray} \tag{ 2 }$$

This nonlinear Hamiltonian has already been realized for atoms in optical cavities [11, 15, 26, 27], and in several other physical systems [28–30]. Theoretically, squeezing can also be induced by the two-axis countertwisting Hamiltonian

$$\begin{eqnarray}&&{\hat{H}}_{2-\mathrm{axis}}=\alpha ({\hat{J}}_{\theta }^{2}-{\hat{J}}_{\theta +\tfrac{\pi }{2}}^{2}),\end{eqnarray} \tag{ 3 }$$

which may offer advantages over one-axis twisting. In the absence of decoherence, ${\hat{H}}_{2-\mathrm{axis}}$ leads to Heisenberg limited squeezing, ${\xi }^{2}\sim 1/N$, which is the fundamental limit [26]. This is in contrast to the one-axis twisting Hamiltonian (2), which has a theoretical limit of ${\xi }^{2}\sim 1/{N}^{\tfrac{2}{3}}$ arising from the curvature of the Bloch sphere [26, 31]. Furthermore, the two-axis Hamiltonian squeezes exponentially in time while the one-axis Hamiltonian squeezes only polynomially [26, 32]. This has motivated efforts to realize two-axis Hamiltonians in various settings [32–34].

In this article, we extend the cavity-based one-axis twisting scheme of [11] to show how an effective two-axis twisting Hamiltonian can be engineered. For atoms strongly coupled to the cavity such that dissipation can be neglected, the two-axis scheme creates stronger squeezing than the one axis scheme. However, for weakly coupled atoms the situation is different. We find that when limited by decoherence, ${\xi }^{2}$ scales as $1/\sqrt{{NC}}$ for both the one- and two-axis schemes and the two schemes exhibit similar amounts of squeezing. We find that this is because the collective decay adds more noise to the squeezed quadrature in the two-axis scheme than the one-axis scheme, as shown qualitatively in figure 1. For quantum information processing, not only the amount of squeezing but also the purity of the squeezing operation matters [6]. We therefore also compare the performance of both schemes to an ideal squeezing operation. We find that also in this case, the one-axis scheme performs similar to or better than the two-axis scheme when limited by decoherence.

Zoom In Zoom Out Reset image size

In the one-axis twisting scheme of [11], a collection of atoms is placed in a cavity such that two ground states are both coupled off-resonantly through the cavity field to an excited state (figure 2). By illuminating the atoms with bichromatic light, pairwise exchange between the ground states can be realized, resulting in the quadratic Hamiltonian ${\hat{H}}_{1-\mathrm{axis}}=\alpha {\hat{J}}_{\theta }^{2}$. Below, we first show that by adding a second bichromatic laser to the setup of [11], the effective dynamics can be described by a two-axis twisting Hamiltonian of the form in equation (3). We then proceed by analyzing and comparing the squeezing properties of both the original one-axis scheme and the modified two-axis scheme, including the effects of dissipation. Finally, we elaborate on the requirements for the validity of the effective dynamics considered.

Zoom In Zoom Out Reset image size

We assume that the atoms have two stable ground states $| a\rangle$ and $| b\rangle$ and an excited level $| e\rangle$. The ground states are coupled to the excited level through four laser couplings and two cavity couplings with coupling constants ${g}_{{a}}$ and ${g}_{{b}}$ as shown in figure 2. In a suitable rotating frame, the Hamiltonian describing the system is

$$\begin{eqnarray}&&\hat{H}=\displaystyle \sum _{k=1}^{N}\left(\displaystyle \frac{{{\rm{\Omega }}}_{1}}{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}+\displaystyle \frac{{{\rm{\Omega }}}_{3}}{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{3}t}+{g}_{{a}}{{\rm{e}}}^{{\rm{i}}({{\rm{\Delta }}}_{2}+\delta )t}\hat{c}\right)| {\rm{e}}{\rangle }_{k}\langle {a}| +\left(\displaystyle \frac{{{\rm{\Omega }}}_{2}}{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}+\displaystyle \frac{{{\rm{\Omega }}}_{4}}{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{4}t}+{g}_{{b}}{{\rm{e}}}^{{\rm{i}}({{\rm{\Delta }}}_{1}+\delta )t}\hat{c}\right)| {\rm{e}}{\rangle }_{k}\langle {b}| +{\rm{H}}.{\rm{c}}.,\end{eqnarray} \tag{ 4 }$$

where H.c. is the Hermitian conjugate. The N atoms are labeled by the subscript k and we have defined the detunings ${{\rm{\Delta }}}_{1}={\omega }_{{\rm{e}}}-{\omega }_{{\rm{L}}1}$, ${{\rm{\Delta }}}_{2}={{\rm{\Delta }}}_{1}-{\omega }_{{b}}$, ${{\rm{\Delta }}}_{3}={{\rm{\Delta }}}_{1}+2\delta$, ${{\rm{\Delta }}}_{4}={{\rm{\Delta }}}_{2}+2\delta$, and $\delta ={\omega }_{{\rm{L}}1}-{\omega }_{{\rm{b}}}-{\omega }_{\mathrm{cav}}$. Here, ${\omega }_{{e}}$ (${\omega }_{{b}}$) is the transition frequency between level $| a\rangle$ and $| e\rangle$ ($| b\rangle$), ${\omega }_{{\rm{L}}x}$ is the frequency of laser x, and ${\omega }_{\mathrm{cav}}$ is the cavity resonance frequency. The four laser couplings are denoted ${{\rm{\Omega }}}_{1-4}$ and ${g}_{{a}}$ (${g}_{{b}}$) is the cavity coupling of level $| a\rangle$ ($| b\rangle$). We have assumed the frequencies of the upper of the two lasers addressing different transitions to differ by twice the ground state splitting ${\omega }_{{\rm{L}}1}-{\omega }_{{\rm{L}}2}=2{\omega }_{{b}}$ and similarly the lower two fields differ by the same amount ${\omega }_{{\rm{L}}3}-{\omega }_{{\rm{L}}4}=2{\omega }_{{b}}$. Furthermore, we have assumed that the laser fields addressing the same transitions differ in frequency by $2\delta$ so that ${\omega }_{{\rm{L}}1}-{\omega }_{{\rm{L}}3}={\omega }_{{\rm{L}}2}-{\omega }_{{\rm{L}}4}=2\delta$. The decay of state $| e{\rangle }_{k}$ is assumed to be described by the Lindblad operators ${\hat{L}}_{x}^{(k)}=\sqrt{{\gamma }_{x}}| x{\rangle }_{k}\langle e|$, where ${\gamma }_{x}$ is the decay rate into state $| x\rangle$ and $x\in \{a,b,o\}$. The state $| o\rangle$ represents all other ground states than $| a\rangle$ and $| b\rangle$. The total decay rate of the excited state is ${\rm{\Gamma }}={\gamma }_{a}+{\gamma }_{b}+{\gamma }_{o}$. The decay of the cavity field is assumed to be described by the Lindblad operator ${\hat{L}}_{c}=\sqrt{\kappa }\hat{c}$, where κ is the intensity decay rate of the cavity and $\hat{c}$ is the annihilation operator of the cavity field. We assume that both atomic ground states are coupled to the excited state through the same cavity field.

The basic mechanism behind the scheme can be understood from considering the various transitions mediated by the laser and cavity fields. Assuming large detunings, the couplings from laser 1 and 2 allows a two-photon resonant transitions of the form $| {aa}\rangle \to | {bb}\rangle$ ($| {bb}\rangle \to | {aa}\rangle$). Here, an atom in state $| a\rangle$ ($| b\rangle$) absorbs a photon from laser 1 (2) and emits a cavity photon that is absorbed by another atom in state $| a\rangle$ ($| b\rangle$), which then emits into laser 2 (1) resulting in the simultaneous transfer of two atoms from $| a\rangle$ to $| b\rangle$ ($| b\rangle$ to $| a\rangle$). Since laser 1 is detuned by δ and laser 2 by $-\delta$, processes involving only a single atom are off resonant and will be suppressed. In the two atom process, however, the two detunings cancel, making the total two atom process $| {aa}\rangle \to | {bb}\rangle$ ($| {bb}\rangle \to | {aa}\rangle$) resonant. The resulting dynamics can thus be described by a term ${\hat{J}}_{+}^{2}$ (${\hat{J}}_{-}^{2}$) in an effective Hamiltonian for the ground states where ${\hat{J}}_{+}={\sum }_{k}| a{\rangle }_{k}\langle b|$ and ${\hat{J}}_{-}={\hat{J}}_{+}^{\dagger }$. Other resonant processes are transitions of the form $| {ab}\rangle \to | {ba}\rangle$ ($| {ba}\rangle \to | {ab}\rangle$) where an atom in state $| a\rangle$ ($| b\rangle$) absorbs a photon from laser 1 (2) and emits a cavity photon that is absorbed by an atom in state $| b\rangle$ ($| a\rangle$), which then emits into laser 1 (2). These processes are described by a term ${\hat{J}}_{-}{\hat{J}}_{+}$ (${\hat{J}}_{+}{\hat{J}}_{-}$) in the Hamiltonian. As a consequence, the effective Hamiltonian describing the evolution due to laser 1 and 2 is

$$\begin{eqnarray}&&{\hat{H}}_{\mathrm{eff}}\sim \displaystyle \frac{{| {{\rm{\Omega }}}_{1}| }^{2}{| {g}_{{b}}| }^{2}}{4{{\rm{\Delta }}}_{1}^{2}\delta }{\hat{J}}_{+}{\hat{J}}_{-}+\displaystyle \frac{{| {{\rm{\Omega }}}_{2}| }^{2}{| {g}_{{a}}| }^{2}}{4{{\rm{\Delta }}}_{2}^{2}\delta }{\hat{J}}_{-}{\hat{J}}_{+}+\displaystyle \frac{{{\rm{\Omega }}}_{1}^{* }{g}_{{b}}{g}_{{a}}^{* }{{\rm{\Omega }}}_{2}}{4{{\rm{\Delta }}}_{1}{{\rm{\Delta }}}_{2}\delta }{\hat{J}}_{+}^{2}+\displaystyle \frac{{{\rm{\Omega }}}_{2}^{* }{g}_{{a}}{g}_{{b}}^{* }{{\rm{\Omega }}}_{1}}{4{{\rm{\Delta }}}_{1}{{\rm{\Delta }}}_{2}\delta }{\hat{J}}_{-}^{2},\end{eqnarray} \tag{ 5 }$$

as shown in [11]. Tuning the strength of the laser couplings such that $| {{\rm{\Omega }}}_{1}{g}_{{b}}^{* }| /{{\rm{\Delta }}}_{1}=| {{\rm{\Omega }}}_{2}{g}_{{a}}^{* }| /{{\rm{\Delta }}}_{2}=| {\rm{\Omega }}{g}^{* }| /{\rm{\Delta }}$, ${\hat{H}}_{\mathrm{eff}}$ reduces to the one-axis Hamiltonian ${\hat{H}}_{1-\mathrm{axis}}=\alpha {\hat{J}}_{\theta }^{2}$ with $\alpha ={| {\rm{\Omega }}| }^{2}{| g| }^{2}/{{\rm{\Delta }}}^{2}\delta$ and ${e}^{-2{\rm{i}}\theta }=\tfrac{{{\rm{\Omega }}}_{1}^{* }{g}_{{\rm{b}}}{g}_{{\rm{a}}}^{* }{{\rm{\Omega }}}_{2}}{| {{\rm{\Omega }}}_{1}^{* }{g}_{{\rm{b}}}{g}_{{\rm{a}}}^{* }{{\rm{\Omega }}}_{2}| }$ [11]. By adding lasers 3 and 4, we basically add the same effective terms to the Hamiltonian as with laser 1 and 2, except they are now proportional to ${{\rm{\Omega }}}_{3}$, ${{\rm{\Omega }}}_{4}$, and $-1/\delta$ instead of ${{\rm{\Omega }}}_{1},{{\rm{\Omega }}}_{2}$, and $1/\delta$ (see figure 2). Matching the strengths of the lasers results in destructive interference of the ${\hat{J}}_{-}{\hat{J}}_{+}$ and ${\hat{J}}_{+}{\hat{J}}_{-}$ terms. In addition, a relative phase of π between laser 1 and 3 while laser 2 and 4 are in phase with each other ensures constructive interference of the ${\hat{J}}_{+}^{2}$ and ${\hat{J}}_{-}^{2}$ terms resulting in an effective two-axis Hamiltonian of the form in equation (3).

We now proceed by deriving the effective Hamiltonian describing the system. Motivated by the above considerations, we assume that we are in the far detuned limit where ${\rm{\Delta }}\gg {\rm{\Omega }},\delta ,g$. Consequently, we can adiabatically eliminate the excited states of the atoms using the effective operator formalism introduced in [35]. We neglect fast oscillating terms ($\sim {{\rm{e}}}^{2{\rm{i}}{\omega }_{b}t}$) in the Hamiltonian and assume $1/{{\rm{\Delta }}}_{1}\approx 1/({{\rm{\Delta }}}_{1}+2\delta )$ and $1/{{\rm{\Delta }}}_{2}\approx 1/({{\rm{\Delta }}}_{2}+2\delta )$ since we are considering the limit ${\rm{\Delta }}\gg \delta$. After some algebra, we end up with an effective Hamiltonian

$$\begin{eqnarray}{\hat{H}}_{\mathrm{eff}1} & = & -\displaystyle \sum _{k=1}^{N}\left(\displaystyle \frac{({| {{\rm{\Omega }}}_{1}| }^{2}+{| {{\rm{\Omega }}}_{3}| }^{2}){{\rm{\Delta }}}_{1}}{4{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}}+{\mathfrak{R}}\left\{\displaystyle \frac{{{\rm{\Omega }}}_{1}^{* }{{\rm{\Omega }}}_{3}{{\rm{e}}}^{2{\rm{i}}\delta t}}{4{{\rm{\Delta }}}_{1}-2{\rm{i}}{\rm{\Gamma }}}\right\}\right)| a{\rangle }_{k}\langle a| \\ & & +\,\left(\displaystyle \frac{({| {{\rm{\Omega }}}_{2}| }^{2}+{| {{\rm{\Omega }}}_{4}| }^{2}){{\rm{\Delta }}}_{2}}{4{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}}+{\mathfrak{R}}\left\{\displaystyle \frac{{{\rm{\Omega }}}_{2}^{* }{{\rm{\Omega }}}_{4}{{\rm{e}}}^{2{\rm{i}}\delta t}}{4{{\rm{\Delta }}}_{2}-2{\rm{i}}{\rm{\Gamma }}}\right\}\right)| b{\rangle }_{k}\langle b| \\ & & +\displaystyle \frac{4{| {g}_{{\rm{a}}}| }^{2}{{\rm{\Delta }}}_{2}}{4{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}}{\hat{c}}^{\dagger }\hat{c}| a{\rangle }_{k}\langle a| +\displaystyle \frac{4{| {g}_{{\rm{b}}}| }^{2}{{\rm{\Delta }}}_{1}}{4{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}}{\hat{c}}^{\dagger }\hat{c}| b{\rangle }_{k}\langle b| \\ & & +2\left[\displaystyle \frac{{g}_{{\rm{b}}}^{* }{{\rm{\Delta }}}_{1}}{{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}}({{\rm{\Omega }}}_{1}{{\rm{e}}}^{-{\rm{i}}\delta t}+{{\rm{\Omega }}}_{3}{{\rm{e}}}^{{\rm{i}}\delta t}){\hat{c}}^{\dagger }| b{\rangle }_{k}\langle a| \right.\\ & & +\left.\displaystyle \frac{{g}_{{\rm{a}}}^{* }{{\rm{\Delta }}}_{2}}{{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}}({{\rm{\Omega }}}_{2}{{\rm{e}}}^{-{\rm{i}}\delta t}+{{\rm{\Omega }}}_{4}{{\rm{e}}}^{{\rm{i}}\delta t}){\hat{c}}^{\dagger }| a{\rangle }_{k}\langle b| +{\rm{H}}.{\rm{c}}.\right].\end{eqnarray} \tag{ 6 }$$

The effective Lindblad operators describing the atomic decay are

$$\begin{eqnarray}{\hat{L}}_{x,\mathrm{eff}1}^{(k)} & = & \sqrt{{\gamma }_{x}}\left[\left(\displaystyle \frac{{{\rm{\Omega }}}_{1}+{{\rm{\Omega }}}_{3}{{\rm{e}}}^{2{\rm{i}}\delta t}}{2{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}+\displaystyle \frac{2{g}_{{\rm{a}}}{{\rm{e}}}^{{\rm{i}}({{\rm{\Delta }}}_{2}+\delta )t}}{2{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}}\right)| x{\rangle }_{k}\langle a| \right.\\ & & +\left.\left(\displaystyle \frac{{{\rm{\Omega }}}_{2}+{{\rm{\Omega }}}_{4}{{\rm{e}}}^{2{\rm{i}}\delta t}}{2{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}+\displaystyle \frac{2{g}_{{\rm{b}}}{{\rm{e}}}^{{\rm{i}}({{\rm{\Delta }}}_{1}+\delta )t}}{2{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}}\right)| x{\rangle }_{k}\langle b| \right].\end{eqnarray} \tag{ 7 }$$

The first four terms in equation (6) are the AC Stark shifts from the laser fields while the next two terms are the cavity induced shifts of the ground states. The terms $\propto {{\rm{e}}}^{2{\rm{i}}\delta t}$ in the AC Stark shifts are fast oscillating for large δ and can therefore be neglected in this limit. Furthermore, the constant terms can be compensated by properly adjusting the frequency of the laser fields. We will therefore neglect the AC Stark shifts in what follows⁵ . In addition, we also neglect the cavity induced shifts since under the right conditions, these give negligible phase shifts to the ground states as we will discuss later. The remaining terms in equation (6) describe Raman transitions from $| a\rangle \to | b\rangle$ ($| b\rangle \to | a\rangle$) through laser 1 or 3 (2 or 4) and the cavity field.

Assuming that the cavity field is weakly populated, we now proceed by adiabatic eliminating the cavity field (see appendix A). For laser fields tuned such that

$$\begin{eqnarray}&&\displaystyle \frac{2{{\rm{\Omega }}}_{1}{g}_{{b}}^{* }{{\rm{\Delta }}}_{1}}{4{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}}=\displaystyle \frac{2{\rm{i}}{{\rm{\Omega }}}_{2}{g}_{{a}}^{* }{{\rm{\Delta }}}_{2}}{4{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}}=\chi \end{eqnarray} \tag{ 8 }$$

and ${{\rm{\Omega }}}_{1}=-{{\rm{\Omega }}}_{3}$, ${{\rm{\Omega }}}_{2}={{\rm{\Omega }}}_{4}$, we find an effective two-axis twisting Hamiltonian,

$$\begin{eqnarray}&&{\hat{H}}_{\mathrm{eff}2}\approx \displaystyle \frac{8{\rm{i}}{| \chi | }^{2}\delta }{4{\delta }^{2}+{\tilde{\kappa }}^{2}}[{\hat{J}}_{+}^{2}-{\hat{J}}_{-}^{2}],\end{eqnarray} \tag{ 9 }$$

where

$$\begin{eqnarray}&&\tilde{\kappa }=\kappa +{\rm{\Gamma }}\left(\displaystyle \frac{4\langle {\hat{N}}_{a}\rangle {| {g}_{{\rm{a}}}| }^{2}}{4{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}}+\displaystyle \frac{4\langle {\hat{N}}_{b}\rangle {| {g}_{{\rm{b}}}| }^{2}}{4{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}}\right),\end{eqnarray} \tag{ 10 }$$

is the modified decay rate of the cavity due to the atom–cavity coupling. Here ${\hat{N}}_{a}={\sum }_{k}| a{\rangle }_{k}\langle a|$ and ${\hat{N}}_{b}={\sum }_{k}| b{\rangle }_{k}\langle b|$ are the atomic number operators. These have been replaced with their average values in deriving the effective dynamics assuming that we can neglect fluctuations around the mean for the calculation of $\tilde{\kappa }$. Note that the effective Hamiltonian in equation (9) corresponds to setting $\alpha =16{| \chi | }^{2}\delta /(4{\delta }^{2}+{\tilde{\kappa }}^{2})$ and $\theta =-\pi /4$ in equation (3)⁶ . The effective Lindblad operators are

$$\begin{eqnarray}{\hat{L}}_{x,\mathrm{eff}2}^{(k)} & = & \sqrt{{\gamma }_{x}}\left[\displaystyle \frac{{{\rm{\Omega }}}_{1}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}}{2{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}}(1-{{\rm{e}}}^{2{\rm{i}}\delta t})| x{\rangle }_{k}\langle a| +\displaystyle \frac{{{\rm{\Omega }}}_{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}}{2{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}}(1+{{\rm{e}}}^{2{\rm{i}}\delta t})| x{\rangle }_{k}\langle b| \right.\\ & & -\left.\left(\displaystyle \frac{2{{ig}}_{{\rm{a}}}\chi {{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}}{2{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}}| x{\rangle }_{k}\langle a| +\displaystyle \frac{2{g}_{{\rm{b}}}\chi {{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}}{2{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}}| x{\rangle }_{k}\langle b| \right)\times \left(\displaystyle \frac{{\rm{i}}{\hat{J}}_{+}-{\hat{J}}_{-}}{\delta +{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{\rm{i}}{\hat{J}}_{+}+{\hat{J}}_{-}}{\delta -{\rm{i}}\tilde{\kappa }/2}{{\rm{e}}}^{2{\rm{i}}\delta t}\right)\right]\end{eqnarray} \tag{ 11 }$$

$$\begin{eqnarray}&&{\hat{L}}_{c,\mathrm{eff}2}=-\sqrt{\kappa }\chi {{\rm{e}}}^{-{\rm{i}}\delta t}\left[\displaystyle \frac{{\rm{i}}{\hat{J}}_{+}-{\hat{J}}_{-}}{\delta +{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{\rm{i}}{\hat{J}}_{+}+{\hat{J}}_{-}}{\delta -{\rm{i}}\tilde{\kappa }/2}{{\rm{e}}}^{2{\rm{i}}\delta t}\right].\end{eqnarray} \tag{ 12 }$$

We now proceed by deriving the evolution of the collective spin state predicted by the effective operators.

2.1. Equations of motion (EOM)

EOM for the mean of an atomic operator $\langle \hat{O}\rangle$ can be found from the Heisenberg–Langevin equation

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}t}\langle \hat{O}\rangle & = & {\rm{i}}\langle [{\hat{H}}_{\mathrm{eff}2},\hat{O}]\rangle +\displaystyle \sum _{x}\displaystyle \sum _{k}\langle {({\hat{L}}_{x,\mathrm{eff}2}^{(k)})}^{\dagger }\hat{O}{\hat{L}}_{x,\mathrm{eff}2}^{(k)}\rangle -\displaystyle \frac{1}{2}\langle \hat{O}{({\hat{L}}_{x,\mathrm{eff}2}^{(k)})}^{\dagger }{\hat{L}}_{x,\mathrm{eff}2}^{(k)}\rangle \\ & & -\displaystyle \frac{1}{2}\langle {({\hat{L}}_{x,\mathrm{eff}2}^{(k)})}^{\dagger }{\hat{L}}_{x,\mathrm{eff}2}^{(k)}\hat{O}\rangle .\end{eqnarray} \tag{ 13 }$$

To obtain a closed set of EOM, we linearize the noise of the atomic operators in the limit of $N\gg 1$ similar to what was done in [11]. The linearization of the noise can be described as making the transformation

$$\begin{eqnarray}{\hat{J}}_{+} & \to & \langle {\hat{J}}_{+}\rangle +\lambda \delta {\hat{J}}_{+},\qquad \quad {\hat{N}}_{a}\to \langle {N}_{a}\rangle \\ {\hat{J}}_{-} & \to & \langle {\hat{J}}_{-}\rangle +\lambda \delta {\hat{J}}_{-},\qquad \quad {\hat{N}}_{b}\to \langle {N}_{b}\rangle \\ {\hat{J}}_{z} & \to & \langle {\hat{J}}_{z}\rangle ,\end{eqnarray} \tag{ 14 }$$

in the EOM and only keeping terms to second order in λ. Here $\delta \hat{O}=\hat{O}-\langle \hat{O}\rangle$ describe the fluctuations around the mean. The result of this is a closed set of EOM that can be solved numerically (see appendix B).

In the absence of decoherence, it is also possible to numerically solve the Schrödinger equation for a given initial state without performing any linearization of the noise. In order to investigate the accuracy of the linearization performed above, we have therefore evaluated the evolution dictated by a two-axis twisting Hamiltonian of the form ${\hat{H}}_{2-\mathrm{axis}}=\alpha ({\hat{J}}_{-\pi /4}^{2}-{\hat{J}}_{\pi /4}^{2})$ both by directly solving the Schrödinger equation numerically and by performing the linearization of the noise. The squeezing parameter, ${\xi }^{2}$ calculated from both methods are shown in figure 3. We have assumed N = 1000 atoms and that all atoms start out in state $| a\rangle$. Near the minimum of ${\xi }^{2}$, ${\hat{J}}_{z}$ begins to decrease rapidly and as a result, our linearization begins to break down. For smaller times, the linearization however captures the dynamics quite accurately. Since dissipation will result in a different minimum of ${\xi }^{2}$ at an earlier time than for the bare Hamiltonian evolution, we expect our linearization to be valid also when the squeezing is limited by dissipation.

Zoom In Zoom Out Reset image size

For comparison, we have also plotted the squeezing parameter for a one-axis twisting Hamiltonian of the form in equation (2) using both the direct method and the linearization of the noise. As with the two-axis Hamiltonian, the linearization breaks down near the minimum of ${\xi }^{2}$ though the effect is more severe. We believe the reason for this is that the atomic spin is not only squeezed but also twisted into a non-gaussian state for a high amount of squeezing [26, 31, 36]. This is not captured by the linearization. Hence the results of the linearization of the one axis Hamiltonian should not be trusted when the squeezing is close to the minimum obtained from the Hamiltonian. Figure 3 also shows how the two-axis Hamiltonian results in higher squeezing and squeezes faster than the one-axis Hamiltonian in the absence of decoherence.

In order to include the effect of decoherence, we numerically solve the EOM from the effective operators. First, however, we make some analytical estimates of what to expect in order to better understand the numerical results. We approximately solve the EOM for the evolution of the squeezing parameter ${\xi }^{2}$ under ${\hat{H}}_{\mathrm{eff}2}$, starting from a coherent spin state polarized along z. We assume that dissipation is sufficiently weak that $J=N/2$ is preserved, and we consider the planar limit where ${J}_{z}\approx N/2$ throughout the squeezing. From equations (8) and (9), we find the effective interaction strength $\alpha \approx 4{| \chi | }^{2}/\delta \approx {{\rm{\Omega }}}^{2}{g}^{2}/({{\rm{\Delta }}}^{2}\delta )$, where we have defined a generic laser coupling Ω, cavity coupling g and detuning Δ to characterize the system. We assume the limit of large detuning Δ where the two first terms of the Lindblad operators ${\hat{L}}_{x,\mathrm{eff}2}^{(k)}$ describing spontaneous emission in equation (11) are dominant (see discussion below).

From the Heisenberg–Langevin equation, we find that

$$\begin{eqnarray}&&\displaystyle \frac{{\rm{d}}\langle {J}_{x}^{2}\rangle }{{\rm{d}}{t}}\approx -2N\alpha \langle {J}_{x}^{2}\rangle +\displaystyle \frac{{N}^{2}\kappa {{\rm{\Omega }}}^{2}{g}^{2}}{4{{\rm{\Delta }}}^{2}{\delta }^{2}}+\displaystyle \frac{{\rm{\Gamma }}{{\rm{\Omega }}}^{2}}{8{{\rm{\Delta }}}^{2}}N,\end{eqnarray} \tag{ 15 }$$

where we have assumed that $N\alpha \gg {\rm{\Gamma }}{{\rm{\Omega }}}^{2}/{{\rm{\Delta }}}^{2}$. The resulting evolution of the squeezing parameter is

$$\begin{eqnarray}&&\displaystyle \frac{{\rm{d}}{\xi }^{2}}{{\rm{d}}{t}}\approx \alpha \left(-2N{\xi }^{2}+N\displaystyle \frac{\kappa }{\delta }+\displaystyle \frac{1}{2C}\displaystyle \frac{\delta }{\kappa }\right),\end{eqnarray} \tag{ 16 }$$

where $C={g}^{2}/(\kappa {\rm{\Gamma }})$ is the single-atom cooperativity. The first term in equation (16) is the unitary evolution from the two-axis Hamiltonian while the second and third terms describe noise added by cavity decay and spontaneous emission from the atoms, respectively.

In the limit of large single-atom cooperativity C, where spontaneous emission becomes negligible, dissipation via the cavity can be suppressed by operating at large detuning δ from cavity resonance. At finite cooperativity, however, the squeezing is optimized at a detuning $\delta \sim {\delta }_{s}=\sqrt{{NC}}\kappa$ that minimizes the combined effect of the two forms of dissipation. Squeezing at all requires ${\rm{d}}{\xi }^{2}/{\rm{d}}{t}\lt 0$ at $\xi =1$, corresponding to a large collective cooperativity $\sqrt{{NC}}\gt 1$. The squeezing parameter then initially decays until reaching a minimum value of ${\xi }^{2}\sim 1/\sqrt{{NC}}$ after a time ${t}_{s}\sim \mathrm{ln}(\sqrt{{NC}})/(\alpha N)$, where the rate of squeezing can no longer compete with the rate of adding noise.

The scaling of the squeezing parameter obtained above is the same as the scaling for the one axis scheme derived in [11]. One might have expected a more favorable scaling of ${\xi }^{2}$ for the two-axis Hamiltonian than for the one-axis Hamiltonian since, in the absence of noise, the former gives an exponential decrease of ${\xi }^{2}$ while the latter only leads to a decrease of the form $1/(\alpha {Nt})$. However, for the coupling configuration leading to the one-axis Hamiltonian ${\hat{H}}_{1-\mathrm{axis}}=\alpha {\hat{J}}_{x}^{2}$, the effective Lindblad operator describing the cavity decay is $\propto {\hat{J}}_{x}$ (see appendix A). For large $\alpha t$, the squeezed component from the Hamiltonian is almost entirely described by ${\hat{J}}_{x}$ with only a small admixture of ${\hat{J}}_{z}$. The cavity decay thus nearly conserves the value of the squeezed component and primarily adds noise to the anti-squeezed component $\sim {\hat{J}}_{y}$ (figure 1(a)). Consequently, the one-axis scheme is more stable against cavity decay than one would naively expect. For the two-axis Hamiltonian scheme, cavity decay adds noise to both the squeezed and anti-squeezed components (see appendix C and figure 1(b)), which counteracts the faster squeezing such that the scaling of ${\xi }^{2}$ becomes the same for the two schemes.

Without the assumptions of ${J}_{z}=N/2$ and constant $J={N}_{a}+{N}_{b}$, we can numerically solve the EOM given in appendix B to evaluate ${\xi }^{2}$ in the limit $N\gg {NC}$, where the scheme is limited by dissipation. From the Lindblad operators in equation (11), we can estimate the effect of spontaneous decay of the atoms on the collective atomic state. To determine the ideal operating conditions we note that the effect of the two first terms in equation (11) will not decrease with increasing ${{\rm{\Delta }}}_{1},{{\rm{\Delta }}}_{2}$, since we expect $t\propto {{\rm{\Delta }}}^{2}/({{\rm{\Omega }}}^{2}{\rm{\Gamma }})$. The other terms will, however, be suppressed for large detunings (Δ). In the numerical simulations, we find that these terms have a detrimental effect on the squeezing and the detunings should therefore be chosen large enough for these terms to be negligible compared to the two first terms We include these terms in our numerical optimizations, but choose ${{\rm{\Delta }}}_{1},{{\rm{\Delta }}}_{2}$ sufficiently large that they are negligible. The result are then almost independent of ${{\rm{\Delta }}}_{1},{{\rm{\Delta }}}_{2}$ and we do not optimize over these parameters.

We numerically minimize ${\xi }^{2}$ for the two-axis scheme while requiring the laser fields to be tuned such that ${{\rm{\Omega }}}_{2}={{\rm{\Omega }}}_{4}$ and ${{\rm{\Omega }}}_{1}=-{{\rm{\Omega }}}_{3}$ (this assumption forces the Hamiltonian to remain of the two-axis form and not cross into the one-axis Hamiltonian in the numerical optimization). We minimize in the interaction time t_s, the two-photon detuning δ, and the ratio ${{\rm{\Omega }}}_{2}/{{\rm{\Omega }}}_{1}$ which can be an imaginary number reflecting a phase difference between the two laser fields. Note that we keep ${{\rm{\Omega }}}_{\mathrm{1,2}}/{{\rm{\Delta }}}_{\mathrm{1,2}}\lesssim 1/50$ to ensure the validity of the adiabatic elimination (see below). The result of the optimization is shown in figure 4(a), which also shows the optimal squeezing for the one-axis scheme considered in [11]. The effective operators for the one-axis scheme can be obtained from the effective operators of the two-axis scheme by simply setting ${{\rm{\Omega }}}_{3}={{\rm{\Omega }}}_{4}=0$ (see appendix A). The numerical simulations confirm the $1/\sqrt{{NC}}$ scaling of ${\xi }^{2}$ for both schemes and it is seen that the one-axis scheme reaches slightly higher squeezing than the two-axis scheme.

Zoom In Zoom Out Reset image size

The numerically optimized detuning and squeezing time for the two-axis scheme are shown in figure 4(b) and are in good agreement with the analytical estimates of $\delta \sim \sqrt{{NC}}\kappa$ and $t\sim \tfrac{\mathrm{ln}(\sqrt{{NC}}){{\rm{\Delta }}}^{2}}{\sqrt{{NC}}{{\rm{\Omega }}}^{2}}\tfrac{1}{{\rm{\Gamma }}}$. In contrast to this, the maximum squeezing for the one-axis scheme is found for $\delta =0$ where the effective Hamiltonian is vanishing. This was already noted in [11] but the origin of this result was not clear at the time. From our analysis, we observe that the optimum corresponds to a dissipative scheme, very similar to [14]. In both schemes, the effective Lindblad operator associated with the cavity decay drives the system into a squeezed state for non-balanced laser couplings (${{\rm{\Omega }}}_{2}\ne {{\rm{\Omega }}}_{1}$). The main difference between [14] and the dissipative scheme considered here is that while we consider a three level system with one excited state and two different detunings (${{\rm{\Delta }}}_{1},{{\rm{\Delta }}}_{2}$), the scheme in [14] considers a four level system with two excited states and equal detunings.

3.1. Squeezing fidelity

Even though the degree of squeezing obtainable with the two-axis and one-axis scheme are similar, the squeezing operations are very different. In particular, we found that the one-axis scheme leads to maximum squeezing when operated in a dissipative fashion. To further compare the performance of the two schemes we consider the fidelity of the squeezing operation for both schemes when compared to a perfect squeezing operation on a coherent spin state. We define canonical position and momentum operators [37]

$$\begin{eqnarray}&&\hat{x}={\hat{J}}_{x}/\sqrt{\langle {\hat{J}}_{z}\rangle },\end{eqnarray} \tag{ 17 }$$

$$\begin{eqnarray}&&\hat{p}={\hat{J}}_{y}/\sqrt{\langle {\hat{J}}_{z}\rangle },\end{eqnarray} \tag{ 18 }$$

to describe the spin ensemble. We assume that the ensemble is initially prepared in a coherent spin state and that $\langle \hat{{J}_{z}}\rangle \approx N/2\gg 1$. In this regime, the canonical operators have the usual canonical commutation relation, $[\hat{x},\hat{p}]\approx i$ and the spin ensemble is described by a Gaussian state characterized by $\hat{x}$ and $\hat{p}$. The perfect squeezing operation amounts to performing the transformation $\{\hat{x},\hat{p}\}\to \{s\hat{x},\hat{p}/s\}$, where $0\lt s\lt 1$ is squeezing in the $\hat{x}$ quadrature ($s\gt 1$ is squeezing in $\hat{p}$). For a given amount of squeezing, we perform a numerical optimization of the fidelity between the perfectly squeezed Gaussian state and the state produced by either the one-axis or two-axis squeezing scheme. The output states of the squeezing schemes are approximately Gaussian since the initial state is a coherent spin state and we are considering squeezing well above the Heisenberg limit. Since all operations and states are Gaussian they are completely characterized by the first and second moments of $\hat{x}$ and $\hat{p}$. We find these from numerical integration of the EOM as before and calculate the fidelity between the perfectly squeezed state and the output state of the squeezing schemes as described in [38].

We optimize in the interaction time t_s, the two-photon detuning δ, and the ratio ${{\rm{\Omega }}}_{2}/{{\rm{\Omega }}}_{1}$ assuming that ${{\rm{\Omega }}}_{3}=-{{\rm{\Omega }}}_{1}$ and ${{\rm{\Omega }}}_{4}={{\rm{\Omega }}}_{2}$ in the two-axis scheme. We also allow for initial and final rotations of the spin state and optimize in the rotation angles. The unitary evolution from the one-axis and two-axis schemes will, in general, squeeze a linear combination of $\hat{x}$ and $\hat{p}$ depending on the initial state and the amount of squeezing. Initial and final rotations ensure that the output state is squeezed in $\hat{x}$. In the previous squeezing analysis, we simply considered an initial state at the origin ($\langle \hat{x}\rangle =\langle \hat{p}\rangle =0$) in our numerical simulations since we were only interested in the maximal amount of squeezing obtainable with the two schemes. However, if the squeezing operation is to be performed as part of a continuous variable quantum information protocol [6] or to enhance the measured signal in a quantum metrology protocol [27, 31, 39], the initial state will in general not be at the origin. We therefore consider a distribution of initial states around the origin to ensure a fair comparison. We consider rotation symmetric distributions of initial states with equal distance to the origin, $r=\sqrt{\langle \hat{x}{\rangle }^{2}+\langle \hat{p}{\rangle }^{2}}$ and calculate the average squeezing fidelity for a given distance.

In figures 5(a) and (b) we show the resulting infidelity $\epsilon =1-F$ for the one- and two-axis Hamiltonians. In figure 5(c), we compare the ratio of the infidelities. For both approaches, the infidelity vanishes for weak squeezing $\mathrm{log}(s)\to 0$, where the system is almost unperturbed, and increases as the squeezing is increased. For the two-axis scheme, the error is almost independent of the displacement, whereas the one axis scheme display a cross over between two modes of operation. For small displacements, the dissipative approach yields a much better performance, whereas for large displacements, unitary operation is desirable resulting in a fidelity almost independent of the size of the displacement.

Zoom In Zoom Out Reset image size

The reason for this is that the dissipative scheme drives the system towards a squeezed dark state at the origin i.e. it decreases both $\langle \hat{x}\rangle$ and $\langle \hat{p}\rangle$ and as a result, the fidelity with the perfectly squeezed state decreases as we move away from the origin. This is not the case for the unitary one-axis scheme, which therefore performs better away from the origin and leads to similar but slightly better fidelities than the two-axis scheme. The relative performance of the one-axis and two-axis scheme might change if one moves out of the planar limit considered here where $\langle {\hat{J}}_{z}\rangle \approx N/2$. Since the one-axis Hamiltonian distorts the state quite severely for high squeezing [31], it might perform worse than the two-axis Hamiltonian in this limit when compared to an ideal squeezing operation. As the initial state moves away from the pole of the Bloch sphere, this effect might also become more severe. It is, however, beyond the scope of this work to consider the non-planar limit.

3.2. Validity of effective dynamics

In our analysis so far, we have neglected the cavity shifts in the effective Hamiltonian ${\hat{H}}_{\mathrm{eff}1}$ in equation (6). Furthermore, we have neglected terms oscillating as ${{\rm{e}}}^{2{\rm{i}}\delta t}$ or faster in the EOM of the two-axis scheme. In order to investigate these assumptions, we have performed numerical simulations using ${\hat{H}}_{\mathrm{eff}1}$ and ${\hat{L}}_{x,\mathrm{eff}1}^{(k)}$ without adiabatically eliminating the cavity field and only neglecting terms oscillating faster than ${{\rm{e}}}^{2{\rm{i}}\delta t}$. We still neglect the AC Stark shifts $\propto {| {\rm{\Omega }}| }^{2}$ in ${\hat{H}}_{\mathrm{eff}1}$ since the constant part of these can be compensated by adjusting the frequencies of the lasers, and it is clear that the fast oscillating terms are negligible for weak driving. We do, however, keep the cavity induced shifts where the requirements for negligible shifts are more subtle. We perform a linearization of the noise as before, but now include the cavity field operator in the transformation. This allows us to obtain a closed set of EOM for the same mean values as before, and also for $\langle {\hat{J}}_{+}\hat{c}\rangle ,\langle {\hat{J}}_{+}{\hat{c}}^{\dagger }\rangle ,\langle {\hat{c}}^{2}\rangle$,$\langle {\hat{c}}^{\dagger }\hat{c}\rangle$ and their Hermitian conjugates. These expressions thus replace the EOM presented in appendix B but for brevity, we do not reproduce them here. From these equations, we can investigate under which conditions the adiabatic elimination of the cavity field is valid. We find that a sufficient condition to neglect the cavity shifts is that ${{\rm{\Delta }}}_{\mathrm{1,2}}\gg {NC}{\rm{\Gamma }}$ and furthermore, we need $\kappa \gg 8N{| \chi | }^{2}\delta /(4{\delta }^{2}+{\kappa }^{2})$ in order for the adiabatic elimination of the cavity to be valid. For $\delta ={\delta }_{s}$ the latter criterion translates into $({\rm{\Gamma }}/\kappa ){| {{\rm{\Omega }}}_{\mathrm{1,2}}| }^{2}/{{\rm{\Delta }}}_{1,2}^{2}\ll 1/\sqrt{{NC}}$. Thus, we can always ensure the validity of the adiabatic elimination if we keep the dynamics slow enough using sufficiently weak laser fields and large detunings ${{\rm{\Delta }}}_{\mathrm{1,2}}$. Figure 6 shows how the model where the cavity field has been adiabatically eliminated compares to the one without the adiabatic elimination confirming the above conclusion.

Zoom In Zoom Out Reset image size

We have shown how an effective two-axis twisting Hamiltonian can be realized with a collection of atoms inside an optical cavity. The resulting dynamics of this Hamiltonian leads to spin squeezing of the atoms and, in the absence of dissipation, reaches the ideal Heisenberg limit for metrology. However, the maximum squeezing obtainable in the presence of dissipation is similar to what was found for the one-axis twisting Hamiltonian in [11] and scales as $1/\sqrt{{NC}}$, where NC is the collective cooperativity. The reason why the two-axis scheme does not squeeze more strongly than the one-axis scheme—despite squeezing more quickly—is that collective decay through the cavity mode adds significantly more noise to the squeezed quadrature in the two-axis scheme. It is therefore expected that if the collective decay can be suppressed, the two-axis scheme would outperform the one-axis scheme. This motivates schemes without strong collective decay, such as squeezing through the Rydberg blockade [40].

We have furthermore compared the fidelities of both the one-axis scheme and the two-axis scheme with respect to a perfect squeezing operation. We found that a dissipative operation of the one-axis scheme performed significantly better than the two-axis scheme for squeezing of a state near the origin in phase space. Away from the origin, however, the unitary versions of both the one-axis and two-axis schemes outperform the dissipative scheme and lead to similar fidelities. Unitary versions may therefore be desirable for continuous variable quantum information processing [6] or surpassing detection noise in quantum metrology [27, 31, 39]. Approximately unitary dynamics might be realized by quantum erasure schemes [13, 25] or with small ensembles in the ultra-strong coupling regime $C\sim N$.

The twisting schemes can be implemented with ⁸⁷Rb atoms trapped in an optical lattice, as demonstrated in [13, 19, 21, 41]. One option is to realize the ground states on the hyperfine clock transition $| F=1,{m}_{F}=0\rangle \,\leftrightarrow | F=2,{m}_{F}=0\rangle$ with a quantization axis along the cavity, and the excited state in the $5{P}_{3/2}$ manifold. The drive fields should then have a linear polarization orthogonal to the cavity axis, and the cavity mode of the opposite linear polarization will mediate the four-photon processes giving rise to the twisting Hamiltonians.

In our analysis, we have assumed that all atoms are equally coupled to the cavity field. For uneven couplings, if the atoms are subject to a uniform driving Ω and do not move, the dynamics are expected to resemble the homogeneous case and we thus expect similar results also for inhomogeneous coupling [37, 41, 42]. An extension of this work would be to include fluctuating couplings of the atoms, which e.g. would be the case for systems where a large atomic ensemble is trapped inside a glass cell [20]. Such systems can contain many millions of atoms, which could compensate a smaller coupling to the cavity field since the relevant parameter is the collective cooperativity NC. Furthermore, by allowing the atoms to transverse the beam sufficiently many times during the interaction, one can obtain a motional averaging such that the interaction is effectively with the symmetric collective mode despite the random positions of the atoms [43]. As a result, large atomic squeezed states could be realized.

During the preparation of this manuscript, we became aware of related work on realizing a two-axis twisting Hamiltonian in a cavity setup [44, 45]. The setup described in [44] is very similar to ours and they also find Heisenberg limited squeezing when not limited by dissipation. In contrast to [44], however, we also analyze the performance of the scheme in the presence of strong dissipation and find a similar performance as for the one-axis scheme. Furthermore, [44] operates with a more advanced level structure requiring four atomic levels compared to the three-level structure considered here. The scheme in [45] is related in the sense that the squeezing operation can be described by an effective two-axis squeezing operation but the mechanism is quite different from what is described here and the squeezing is assumed to happen on an optical transition. Nevertheless, this work obtains the same scaling of the squeezing parameter as $1/\sqrt{{NC}}$. We also note that [32] suggests an alternative approach to engineering a two-axis twisting Hamiltonian, namely, converting the one-axis scheme considered here into two-axis twisting in the xz-plane by tuning the relative strength of the two lasers. The performance of such a scheme could be investigated in future work using the techniques demonstrated here.

We gratefully acknowledge the support from the Carlsberg Foundation, the European Research Council under the European Union's Seventh Framework Programme (FP/2007-2013) through ERC Grant QIOS (Grant No. 306576) and ERC Grant Agreement no 337603, the Danish Council for Independent Research (Sapere Aude), Qubiz — Quantum Innovation Center, and VILLUM FONDEN via the QMATH Centre of Excellence (Grant No. 10059). GB and ED acknowledge support from the National Science Foundation. ED acknowledges support from the Hertz Foundation. MS-S acknowledges support from the Alfred P. Sloan Foundation and AFOSR.

Here we give the effective operators describing the dynamics after adiabatically eliminating the excited states of the atoms and the cavity field. We have neglected the AC Stark shifts and cavity shifts of the atomic ground states as described in the main text. Furthermore, we have not assumed that the laser fields are tuned according to equation (8). The effective Hamiltonian is

$$\begin{eqnarray}{\hat{H}}_{\mathrm{eff}2} & = & -\displaystyle \frac{1}{2}\left(\displaystyle \frac{{| {g}_{{b}}| }^{2}{{\rm{\Delta }}}_{1}^{2}}{{(2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2)}^{2}}\left(\displaystyle \frac{{| {{\rm{\Omega }}}_{3}| }^{2}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{| {{\rm{\Omega }}}_{1}| }^{2}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){\hat{J}}_{+}{\hat{J}}_{-}\right.\\ & & +\displaystyle \frac{{| {g}_{{\rm{a}}}| }^{2}{{\rm{\Delta }}}_{2}^{2}}{{(2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2)}^{2}}\left(\displaystyle \frac{{| {{\rm{\Omega }}}_{4}| }^{2}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{| {{\rm{\Omega }}}_{2}| }^{2}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){\hat{J}}_{-}{\hat{J}}_{+}\\ & & +\displaystyle \frac{{g}_{{\rm{a}}}{g}_{{b}}^{* }{{\rm{\Delta }}}_{1}{{\rm{\Delta }}}_{2}}{(2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2)(2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2)}\left(\displaystyle \frac{{{\rm{\Omega }}}_{3}{{\rm{\Omega }}}_{4}^{* }}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{1}{{\rm{\Omega }}}_{2}^{* }}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){\hat{J}}_{-}{\hat{J}}_{-}\\ & & +\left.\displaystyle \frac{{g}_{a}^{* }{g}_{{b}}{{\rm{\Delta }}}_{1}{{\rm{\Delta }}}_{2}}{(2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2)(2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2)}\left(\displaystyle \frac{{{\rm{\Omega }}}_{3}^{* }{{\rm{\Omega }}}_{4}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{1}^{* }{{\rm{\Omega }}}_{2}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){\hat{J}}_{+}{\hat{J}}_{+}\right)+{\rm{H}}.{\rm{c}}.\end{eqnarray} \tag{ A1 }$$

The effective Lindblad operators describing atomic decay are

$$\begin{eqnarray}{\hat{L}}_{x2,\mathrm{eff}2}^{(k)} & = & \sqrt{{\gamma }_{x}}\left(\displaystyle \frac{{{\rm{\Omega }}}_{1}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}+{{\rm{\Omega }}}_{3}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{3}t}}{2{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}}| x{\rangle }_{k}\langle a| +\displaystyle \frac{{{\rm{\Omega }}}_{2}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}+{{\rm{\Omega }}}_{4}{{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{4}t}}{2{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}}| x{\rangle }_{k}\langle b| \right.\\ & & -\displaystyle \frac{{| {g}_{{\rm{a}}}| }^{2}}{2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2}\displaystyle \frac{{{\rm{\Delta }}}_{2}}{{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{4}{{\rm{e}}}^{2{\rm{i}}\delta t}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{2}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}| x{\rangle }_{k}\langle a| {\hat{J}}_{+}\\ & & -\displaystyle \frac{{| {g}_{{\rm{b}}}| }^{2}}{2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2}\displaystyle \frac{{{\rm{\Delta }}}_{1}}{{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{3}{{\rm{e}}}^{2{\rm{i}}\delta t}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{1}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}| x{\rangle }_{k}\langle b| {\hat{J}}_{-}\\ & & -\displaystyle \frac{{g}_{{\rm{a}}}{g}_{{\rm{b}}}^{* }}{2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2}\displaystyle \frac{{{\rm{\Delta }}}_{1}}{{{\rm{\Delta }}}_{2}-{\rm{i}}{\rm{\Gamma }}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{3}{{\rm{e}}}^{2{\rm{i}}\delta t}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{1}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{2}t}| x{\rangle }_{k}\langle a| {\hat{J}}_{-}\\ & & -\left.\displaystyle \frac{{g}_{{\rm{a}}}^{* }{g}_{{\rm{b}}}}{2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2}\displaystyle \frac{{{\rm{\Delta }}}_{2}}{{{\rm{\Delta }}}_{1}-{\rm{i}}{\rm{\Gamma }}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{4}{{\rm{e}}}^{2{\rm{i}}\delta t}}{\delta -{\rm{i}}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{2}}{\delta +{\rm{i}}\tilde{\kappa }/2}\right){{\rm{e}}}^{{\rm{i}}{{\rm{\Delta }}}_{1}t}| x{\rangle }_{k}\langle b| {\hat{J}}_{+}\right),\end{eqnarray} \tag{ A2 }$$

while the effective Lindblad operator describing the cavity decay is

$$\begin{eqnarray}&&{\hat{L}}_{c,\mathrm{eff}2}=\sqrt{\kappa }\left(\displaystyle \frac{{g}_{{b}}^{* }{{\rm{\Delta }}}_{1}}{2{{\rm{\Delta }}}_{1}^{2}+{{\rm{\Gamma }}}^{2}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{1}{{\rm{e}}}^{-{\rm{i}}\delta t}}{\delta +{i}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{3}{{\rm{e}}}^{{\rm{i}}\delta t}}{\delta -{i}\tilde{\kappa }/2}\right){\hat{J}}_{-}+\displaystyle \frac{{g}_{{a}}^{* }{{\rm{\Delta }}}_{2}}{2{{\rm{\Delta }}}_{2}^{2}+{{\rm{\Gamma }}}^{2}/2}\left(\displaystyle \frac{{{\rm{\Omega }}}_{2}{{\rm{e}}}^{-{\rm{i}}\delta t}}{\delta +{i}\tilde{\kappa }/2}-\displaystyle \frac{{{\rm{\Omega }}}_{4}{{\rm{e}}}^{{\rm{i}}\delta t}}{\delta -{i}\tilde{\kappa }/2}\right){\hat{J}}_{+}\right)\end{eqnarray} \tag{ A3 }$$

Here we give the expression for the linearized EOM for the model where the cavity field has been eliminated. Note that we have neglected all terms oscillating as ${{\rm{e}}}^{2{\rm{i}}\delta t}$ or faster in the two-axis scheme. The validity of this is discussed in the main text. In order to simplify the expressions, we write the effective operators defined in appendix A as

$$\begin{eqnarray}&&{\hat{H}}_{\mathrm{eff}2}=-\displaystyle \frac{1}{2}({H}_{+-}{\hat{J}}_{+}{\hat{J}}_{-}+{H}_{++}{\hat{J}}_{+}{\hat{J}}_{+}+{\rm{H}}.{\rm{c}}.),\end{eqnarray} \tag{ B1 }$$

$$\begin{eqnarray}&&{\hat{L}}_{x,\mathrm{eff}2}^{(k)}=\sqrt{{\gamma }_{x}}({\chi }_{1}| x{\rangle }_{k}\langle a| +{\chi }_{2}| x{\rangle }_{k}\langle b| +{\chi }_{3}| x{\rangle }_{k}\langle a| {\hat{J}}_{+}+{\chi }_{4}| x{\rangle }_{k}\langle b| {\hat{J}}_{-}+{\chi }_{5}| x{\rangle }_{k}\langle a| {\hat{J}}_{-}+{\chi }_{6}| x{\rangle }_{k}\langle b| {\hat{J}}_{+}),\end{eqnarray} \tag{ B2 }$$

$$\begin{eqnarray}&&{\hat{L}}_{c,\mathrm{eff}2}={\kappa }_{1}{\hat{J}}_{-}+{\kappa }_{2}{\hat{J}}_{+}.\end{eqnarray} \tag{ B3 }$$

With these definitions, the EOMs are

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{+}^{2}\rangle & = & 2\langle {\hat{J}}_{z}\rangle (({{\rm{i}}{H}}_{++}^{* }-{\kappa }_{1}{\kappa }_{2}^{* })\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +({{\rm{i}}{H}}_{++}^{* }+{\kappa }_{1}{\kappa }_{2}^{* })\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +(2{\rm{i}}{\mathfrak{R}}\{{H}_{+-}\}+{| {\kappa }_{1}| }^{2}-{| {\kappa }_{2}| }^{2})\langle {\hat{J}}_{+}^{2}\rangle )\\ & & -{\rm{\Gamma }}(({| {\chi }_{1}| }^{2}+{| {\chi }_{2}| }^{2})\langle {\hat{J}}_{+}^{2}\rangle +({| {\chi }_{3}| }^{2}+{| {\chi }_{6}| }^{2})(3\langle {\hat{J}}_{+}^{2}\rangle | \langle {\hat{J}}_{+}\rangle {| }^{2}+3\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle \langle {\hat{J}}_{+}{\rangle }^{2}-5| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}{\rangle }^{2})\\ & & +({| {\chi }_{4}| }^{2}+{| {\chi }_{5}| }^{2})(3| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}^{2}\rangle +3\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -5\langle {\hat{J}}_{+}{\rangle }^{2}| \langle {\hat{J}}_{+}\rangle {| }^{2})\\ & & +2\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}^{2}\rangle (\langle {\hat{N}}_{a}\rangle ({| {\chi }_{3}| }^{2}-{| {\chi }_{5}| }^{2})+\langle {\hat{N}}_{b}\rangle ({| {\chi }_{6}| }^{2}-{| {\chi }_{4}| }^{2}))\\ & & +{\chi }_{1}{\chi }_{6}^{* }((\langle {\hat{N}}_{a}\rangle +\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\\ & & +(2({\chi }_{2}{\chi }_{3}^{* }-{\chi }_{1}^{* }{\chi }_{4})\langle {\hat{J}}_{z}\rangle +({\chi }_{1}{\chi }_{4}^{* }+{\chi }_{2}^{* }{\chi }_{3})(\langle {\hat{N}}_{b}\rangle +\langle {\hat{N}}_{a}\rangle ))\langle {\hat{J}}_{+}^{2}\rangle \\ & & +{\chi }_{2}^{* }{\chi }_{5}((\langle {\hat{N}}_{a}\rangle +\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -2\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )+({\chi }_{3}{\chi }_{5}^{* }+{\chi }_{4}^{* }{\chi }_{6})(6\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle -5\langle {\hat{J}}_{+}{\rangle }^{4})\\ & & +({\chi }_{3}^{* }{\chi }_{5}+{\chi }_{4}{\chi }_{6}^{* })(\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{-}^{2}\rangle +\langle {\hat{J}}_{-}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle +2| \langle {\hat{J}}_{+}\rangle {| }^{2}(\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )-5| \langle {\hat{J}}_{+}\rangle {| }^{4}))\end{eqnarray} \tag{ B4 }$$

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle & = & 4\langle {\hat{J}}_{z}\rangle {\mathfrak{I}}\{{H}_{++}\langle {\hat{J}}_{+}^{2}\rangle \}+2({| {\kappa }_{1}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\kappa }_{2}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{J}}_{z}\rangle -{\rm{\Gamma }}(({| {\chi }_{1}| }^{2}+{| {\chi }_{2}| }^{2})\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle \\ & & -{| {\chi }_{2}| }^{2}{\hat{N}}_{a}+({| {\chi }_{5}| }^{2}+{| {\chi }_{4}| }^{2})(\langle {\hat{J}}_{-}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle +\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{-}^{2}\rangle +4| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -5| \langle {\hat{J}}_{+}\rangle {| }^{4})\\ & & -2({| {\chi }_{5}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{4}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\chi }_{4}| }^{2}\langle {\hat{N}}_{a}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{a}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle \\ & & +({| {\chi }_{6}| }^{2}+{| {\chi }_{3}| }^{2})(\langle {\hat{J}}_{-}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle +\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{-}^{2}\rangle +| \langle {\hat{J}}_{+}\rangle {| }^{2}(\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +3\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )-5| \langle {\hat{J}}_{+}\rangle {| }^{4})\\ & & +2({| {\chi }_{3}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}(\langle {N}_{a}\rangle -1)\\ & & +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}(\langle {\hat{N}}_{b}\rangle -1)\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}(\langle {\hat{N}}_{a}\rangle -1)\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{2}{\chi }_{5}^{* }\langle {\hat{J}}_{-}^{2}\rangle \}(\langle {\hat{N}}_{b}\rangle -1)\\ & & -2{\mathfrak{R}}\{({\chi }_{4}^{* }{\chi }_{6}+{\chi }_{3}{\chi }_{5}^{* })(\langle {\hat{J}}_{+}{\rangle }^{2}(\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\\ & & +3| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}^{2}\rangle -5\langle {\hat{J}}_{+}{\rangle }^{2}| \langle {\hat{J}}_{+}\rangle {| }^{2})-{\chi }_{4}^{* }{\chi }_{6}\langle {\hat{N}}_{a}\rangle \langle {\hat{J}}_{+}^{2}\rangle \})\\ & & +{\gamma }_{a}(({| {\chi }_{1}| }^{2}+{| {\chi }_{3}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{N}}_{a}\rangle +({| {\chi }_{2}| }^{2}+{| {\chi }_{6}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +{| {\chi }_{4}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{N}}_{b}\rangle \\ & & +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle \\ & & +2{\mathfrak{R}}\{({\chi }_{2}{\chi }_{5}^{* }+{\chi }_{3}{\chi }_{5}^{* }\langle {\hat{N}}_{a}\rangle +{\chi }_{4}^{* }{\chi }_{6}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{+}^{2}\rangle \})\end{eqnarray} \tag{ B5 }$$

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle & = & 4\langle {\hat{J}}_{z}\rangle {\mathfrak{I}}\{{H}_{++}\langle {\hat{J}}_{+}^{2}\rangle \}+2({| {\kappa }_{1}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\kappa }_{2}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{J}}_{z}\rangle -{\rm{\Gamma }}(({| {\chi }_{1}| }^{2}+{| {\chi }_{2}| }^{2})\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle \\ & & -{| {\chi }_{1}| }^{2}{\hat{N}}_{b}+({| {\chi }_{3}| }^{2}+{| {\chi }_{6}| }^{2})(\langle {\hat{J}}_{-}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle +\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{-}^{2}\rangle +4| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -5| \langle {\hat{J}}_{+}\rangle {| }^{4})\\ & & +2({| {\chi }_{3}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -{| {\chi }_{3}| }^{2}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -{| {\chi }_{5}| }^{2}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle \\ & & +({| {\chi }_{5}| }^{2}+{| {\chi }_{4}| }^{2})(\langle {\hat{J}}_{-}{\rangle }^{2}\langle {\hat{J}}_{+}^{2}\rangle +\langle {\hat{J}}_{+}{\rangle }^{2}\langle {\hat{J}}_{-}^{2}\rangle +| \langle {\hat{J}}_{+}\rangle {| }^{2}(3\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )-5| \langle {\hat{J}}_{+}\rangle {| }^{4})\\ & & -2({| {\chi }_{5}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{4}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}(\langle {N}_{a}\rangle -1)\\ & & +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}(\langle {\hat{N}}_{b}\rangle -1)\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}(\langle {\hat{N}}_{a}\rangle -1)\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{2}{\chi }_{5}^{* }\langle {\hat{J}}_{-}^{2}\rangle \}(\langle {\hat{N}}_{b}\rangle -1)\\ & & -2{\mathfrak{R}}\{({\chi }_{4}{\chi }_{6}^{* }+{\chi }_{3}^{* }{\chi }_{5})(\langle {\hat{J}}_{+}{\rangle }^{2}(2\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\\ & & +3| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{-}^{2}\rangle -5\langle {\hat{J}}_{-}{\rangle }^{2}| \langle {\hat{J}}_{+}\rangle {| }^{2})-{\chi }_{3}^{* }{\chi }_{5}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{-}^{2}\rangle \})\\ & & +{\gamma }_{b}(({| {\chi }_{1}| }^{2}+{| {\chi }_{3}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{a}\rangle +({| {\chi }_{2}| }^{2}+{| {\chi }_{6}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +{| {\chi }_{4}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{b}\rangle \\ & & +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle \\ & & +2{\mathfrak{R}}\{({\chi }_{2}{\chi }_{5}^{* }+{\chi }_{3}{\chi }_{5}^{* }\langle {\hat{N}}_{a}\rangle +{\chi }_{4}^{* }{\chi }_{6}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{+}^{2}\rangle \})\end{eqnarray} \tag{ B6 }$$

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{N}}_{a}\rangle & = & -2{\mathfrak{I}}\{{H}_{++}\langle {\hat{J}}_{+}^{2}\rangle \}-| {\kappa }_{1}^{2}| \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +| {\kappa }_{2}^{2}| \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +{\gamma }_{a}({| {\chi }_{2}| }^{2}\langle {\hat{N}}_{b}\rangle +({| {\chi }_{3}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{a}\rangle \\ & & +2{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{4}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}\langle {\hat{N}}_{b}\rangle )\\ & & -({\gamma }_{b}+{\gamma }_{o})(({| {\chi }_{1}| }^{2}+2{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{a}\rangle +({| {\chi }_{4}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\chi }_{6}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{N}}_{b}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle \\ & & +2{\mathfrak{R}}\{{\chi }_{2}{\chi }_{5}^{* }\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{3}{\chi }_{5}^{* }\langle {\hat{J}}_{+}^{2}\rangle \}\langle {\hat{N}}_{a}\rangle )\end{eqnarray} \tag{ B7 }$$

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{N}}_{b}\rangle & = & 2{\mathfrak{I}}\{{H}_{++}\langle {\hat{J}}_{+}^{2}\rangle \}+| {\kappa }_{1}^{2}| \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -| {\kappa }_{2}^{2}| \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -({\gamma }_{a}+{\gamma }_{o})(({| {\chi }_{3}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{a}\rangle \\ & & +{| {\chi }_{2}| }^{2}\langle {\hat{N}}_{b}\rangle +2{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{2}^{* }{\chi }_{3}\}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle +2{\mathfrak{R}}\{{\chi }_{4}^{* }{\chi }_{6}\langle {\hat{J}}_{+}^{2}\rangle \}\langle {\hat{N}}_{b}\rangle )\\ & & +{\gamma }_{b}(({| {\chi }_{1}| }^{2}+2{| {\chi }_{5}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle )\langle {\hat{N}}_{a}\rangle +({| {\chi }_{4}| }^{2}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -{| {\chi }_{6}| }^{2}\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{N}}_{b}\rangle +2{\mathfrak{R}}\{{\chi }_{1}^{* }{\chi }_{4}\}\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle \\ & & +2{\mathfrak{R}}\{{\chi }_{2}{\chi }_{5}^{* }\langle {\hat{J}}_{+}^{2}\rangle \}+2{\mathfrak{R}}\{{\chi }_{3}{\chi }_{5}^{* }\langle {\hat{J}}_{+}^{2}\rangle \}\langle {\hat{N}}_{a}\rangle )\end{eqnarray} \tag{ B8 }$$

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{+}\rangle & = & 2{\rm{i}}\langle {\hat{J}}_{z}\rangle ({\mathfrak{R}}\{{H}_{+-}\}\langle {\hat{J}}_{+}\rangle +{H}_{++}^{* }\langle {\hat{J}}_{-}\rangle )+({| {\kappa }_{1}| }^{2}(\langle {\hat{J}}_{z}\rangle -1)-{| {\kappa }_{2}| }^{2}\langle {\hat{J}}_{z}\rangle )\langle {\hat{J}}_{+}\rangle +{\kappa }_{1}{\kappa }_{2}^{* }\langle {\hat{J}}_{-}\rangle \\ & & -{\rm{\Gamma }}\left(\displaystyle \frac{{| {\chi }_{1}| }^{2}+{| {\chi }_{2}| }^{2}}{2}\langle {\hat{J}}_{+}\rangle +\displaystyle \frac{{| {\chi }_{3}| }^{2}+{| {\chi }_{6}| }^{2}}{2}(\langle {\hat{J}}_{-}\rangle \langle {\hat{J}}_{+}^{2}\rangle +2\langle {\hat{J}}_{+}\rangle \langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle -2| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}\rangle )\right.\\ & & +({| {\chi }_{3}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{6}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}\rangle -({| {\chi }_{5}| }^{2}\langle {\hat{N}}_{a}\rangle +{| {\chi }_{4}| }^{2}\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}\rangle \\ & & +\displaystyle \frac{{| {\chi }_{4}| }^{2}+{| {\chi }_{5}| }^{2}}{2}(\langle {\hat{J}}_{-}\rangle \langle {\hat{J}}_{+}^{2}\rangle +2\langle {\hat{J}}_{+}\rangle \langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle -2| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{+}\rangle )\\ & & -{\chi }_{1}^{* }{\chi }_{4}\langle {\hat{J}}_{z}\rangle \langle {\hat{J}}_{+}\rangle +{\chi }_{1}{\chi }_{4}^{* }(\langle {\hat{N}}_{a}\rangle +\langle {\hat{N}}_{b}\rangle )\langle {\hat{J}}_{+}\rangle \\ & & +\left({\chi }_{2}^{* }{\chi }_{3}\displaystyle \frac{\langle {\hat{N}}_{a}\rangle +\langle {\hat{N}}_{b}\rangle }{2}+{\chi }_{2}{\chi }_{3}^{* }\langle {\hat{J}}_{z}\rangle \right)\langle {\hat{J}}_{+}\rangle +{\chi }_{2}^{* }{\chi }_{5}\langle {\hat{N}}_{b}\rangle \langle {\hat{J}}_{-}\rangle +\displaystyle \frac{{\chi }_{4}^{* }{\chi }_{6}+{\chi }_{3}{\chi }_{5}^{* }}{2}(3\langle {\hat{J}}_{+}^{2}\rangle -2\langle {\hat{J}}_{+}{\rangle }^{2})\langle {\hat{J}}_{+}\rangle \\ & & +\left.\displaystyle \frac{{\chi }_{4}{\chi }_{6}^{* }+{\chi }_{3}^{* }{\chi }_{5}}{2}((\langle {\hat{J}}_{+}{\hat{J}}_{-}\rangle +\langle {\hat{J}}_{-}{\hat{J}}_{+}\rangle )\langle {\hat{J}}_{-}\rangle +\langle {\hat{J}}_{+}\rangle \langle {\hat{J}}_{-}^{2}\rangle -2| \langle {\hat{J}}_{+}\rangle {| }^{2}\langle {\hat{J}}_{-}\rangle )+{\chi }_{1}{\chi }_{6}^{* }\langle {\hat{N}}_{a}\rangle \langle {\hat{J}}_{-}\rangle \right)\end{eqnarray} \tag{ B9 }$$

Furthermore, we have that $\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{z}\rangle =\tfrac{1}{2}\left(\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{N}}_{a}\rangle -\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{N}}_{b}\rangle \right),\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{-}^{2}\rangle =$ ${\left(\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{+}^{2}\rangle \right)}^{\dagger }$, and $\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{-}\rangle ={\left(\tfrac{{\rm{d}}}{{\rm{d}}{t}}\langle {\hat{J}}_{+}\rangle \right)}^{\dagger }$.

In this appendix, we provide some intuition for the effects of cavity dissipation in the two-axis twisting scheme. With appropriate laser detunings, we can engineer the following effective Hamiltonian and collective Lindblad operator (which appear as equations (9) and (12) in the main text):

$$\begin{eqnarray}&&{\hat{H}}_{\mathrm{eff}2}=\displaystyle \frac{{\rm{i}}\alpha }{2}[{\hat{J}}_{+}^{2}-{\hat{J}}_{-}^{2}]=\displaystyle \frac{\alpha }{2}[{({\hat{J}}_{x}-{\hat{J}}_{y})}^{2}-{({\hat{J}}_{x}+{\hat{J}}_{y})}^{2}],\end{eqnarray} \tag{ C1 }$$

$$\begin{eqnarray}{\hat{L}}_{c,\mathrm{eff}2} & = & \sqrt{\displaystyle \frac{{\gamma }_{c}}{4}}[({\rm{i}}{\hat{J}}_{+}+{\hat{J}}_{-}){{\rm{e}}}^{{\rm{i}}\delta t+{\rm{i}}\phi }-({\rm{i}}{\hat{J}}_{+}-{\hat{J}}_{-}){{\rm{e}}}^{-{\rm{i}}\delta t-{\rm{i}}\phi }]\\ & = & \sqrt{\displaystyle \frac{{\gamma }_{c}}{2}}[({\hat{J}}_{x}-{\hat{J}}_{y}){{\rm{e}}}^{{\rm{i}}\delta t+{\rm{i}}\phi +{\rm{i}}\pi /4}\\ & & +({\hat{J}}_{x}+{\hat{J}}_{y}){{\rm{e}}}^{-{\rm{i}}\delta t-{\rm{i}}\phi -{\rm{i}}\pi /4}]\end{eqnarray} \tag{ C2 }$$

where $\alpha =16{| \chi | }^{2}\delta /(4{\delta }^{2}+{\tilde{\kappa }}^{2})$, ${\gamma }_{c}=16{| \chi | }^{2}\kappa /(4{\delta }^{2}+\tilde{\kappa })$, and $\phi =\mathrm{Arg}(\delta +i\tilde{\kappa }/2)$. The dynamics of the quantum state under ${\hat{H}}_{\mathrm{eff}2}$ and ${\hat{L}}_{c,\mathrm{eff}2}$ are described by the quantum master equation:

$$\begin{eqnarray}\displaystyle \frac{{\rm{d}}}{{\rm{d}}{t}}\hat{\rho } & = & -{\rm{i}}[{\hat{H}}_{\mathrm{eff}2},\hat{\rho }]+{\hat{L}}_{c,\mathrm{eff}2}\ \hat{\rho }{({\hat{L}}_{c,\mathrm{eff}2})}^{\dagger }-\displaystyle \frac{1}{2}\hat{\rho }{({\hat{L}}_{c,\mathrm{eff}2})}^{\dagger }{\hat{L}}_{c,\mathrm{eff}2}\\ & & -\displaystyle \frac{1}{2}{({\hat{L}}_{c,\mathrm{eff}2})}^{\dagger }{\hat{L}}_{c,\mathrm{eff}2}\ \hat{\rho }.\end{eqnarray} \tag{ C3 }$$

The Lindblad operator ${\hat{L}}_{c,\mathrm{eff}2}$ consists of two terms that accumulate opposite phases ${{\rm{e}}}^{\pm {\rm{i}}\delta t}$. If we expand each term in the master equation, we get some terms whose phase factors cancel, and others with phases at ${{\rm{e}}}^{\pm 2{\rm{i}}\delta t}$, e.g.:

$$\begin{eqnarray*}{({\hat{L}}_{c,\mathrm{eff}2})}^{\dagger }{\hat{L}}_{c,\mathrm{eff}2} & = & \displaystyle \frac{{\gamma }_{c}}{2}[{({\hat{J}}_{x}-{\hat{J}}_{y})}^{2}+{({\hat{J}}_{x}+{\hat{J}}_{y})}^{2}+({\hat{J}}_{x}+{\hat{J}}_{y})({\hat{J}}_{x}-{\hat{J}}_{y}){{\rm{e}}}^{{\rm{i}}2\delta t+{\rm{i}}2\phi +{\rm{i}}\pi /2}\\ & & +({\hat{J}}_{x}-{\hat{J}}_{y})({\hat{J}}_{x}+{\hat{J}}_{y}){{\rm{e}}}^{-{\rm{i}}2\delta t-{\rm{i}}2\phi -{\rm{i}}\pi /2}].\end{eqnarray*}$$

Appealing to an argument similar to the rotating wave approximation, we can neglect the pair of rapidly oscillating terms, so long as we are interested in timescales that are long compared to the detuning δ. This approximation decouples the two terms in the original Lindblad operator, and gives a pair of two new Lindblad operators instead:

$$\begin{eqnarray}{\hat{L}}_{c,\mathrm{eff}2}^{(1)} & = & \sqrt{\displaystyle \frac{{\gamma }_{c}}{2}}[{\hat{J}}_{x}+{\hat{J}}_{y}]\\ {\hat{L}}_{c,\mathrm{eff}2}^{(2)} & = & \sqrt{\displaystyle \frac{{\gamma }_{c}}{2}}[{\hat{J}}_{x}-{\hat{J}}_{y}].\end{eqnarray} \tag{ C4 }$$

This pair of operators generates isotropic spreading of the Wigner function in the J_x–J_y plane, as sketched in figure (1) in the main text. In fact, one can show that the action of this pair of operators is equivalent to any pair of Lindblad operators of the form:

$$\begin{eqnarray}{\hat{L}}_{c,\mathrm{eff}2}^{(1)} & = & \sqrt{{\gamma }_{c}}[{\hat{J}}_{x}\cos \theta +{\hat{J}}_{y}\sin \theta ]\\ {\hat{L}}_{c,\mathrm{eff}2}^{(2)} & = & \sqrt{{\gamma }_{c}}[-{\hat{J}}_{x}\sin \theta +{\hat{J}}_{y}\cos \theta ],\end{eqnarray} \tag{ C5 }$$

which implies that there is no preferred axis for this dissipation.