We study the system of attractively interacting fermions hopping in a square lattice with any linear combinations of Rashba or Dresselhaus spin-orbit coupling in a normal Zeeman field. By imposing self-consistence equations at half filling with zero chemical potential, we find that there are three phases: band insulator (BI), superfluid (SF), and topological superfluid (TSF) with a Chern number $C=2$. The $C=2$ TSF happens in small Zeeman fields and weak interactions which is in the experimentally most easily accessible regime. The transition from the BI to the SF is a first-order one due to the multiminima structure of the ground state energy landscape. There is a class of topological phase transitions (TPTs) from the SF to the $C=2$ TSF at the low critical field $h_{c1}$, then another one from the $C=2$ TSF to the BI at the upper critical field $h_{c2}$. We derive effective actions to describe these two classes of topological phase transitions, then use them to study the Majorana edge modes and the zero modes inside the vortex core of the $C=2$ TSF near both $h_{c1}$ and $h_{c2}$, especially exploring their spatial and spin structures. We find that the edge modes decay into the bulk with oscillating behaviors and determine both the decay and oscillating lengths. We compute the bulk spectra and map out the Berry curvature distribution in momentum space near both $h_{c1}$ and $h_{c2}$. We elaborate some intriguing bulk-Berry curvature-edge-vortex correspondences. We also discuss the competitions between SFs and charge density wave states in more general cases. A possible classification scheme of all the TPTs in a square lattice is outlined. Comparisons with previous works on related systems are discussed. Possible experimental implications in cold atoms in an optical lattice are given.

• Received 2 February 2017

DOI:https://doi.org/10.1103/PhysRevB.98.174506