By means of finite-size exact diagonalization we theoretically study the electronic many-body effects on the nearly flat-band structure with time-reversal symmetry in a checkerboard lattice model and identify the topological nature of two quantum phases, with ninefold and threefold degeneracy, that appear, respectively, at small and large values $\lambda$ of a nearest-neighbor spin-dependent interaction. Numerical evidence from the evolution of low-lying energy spectra and Berry phases with both spin-independent and spin-dependent twisted boundary conditions reveal that these two different ground states share the same topological spin Chern number. Quantum phase transition between these two states by tuning $\lambda$ is confirmed by evaluating the closing of energy and quasispin excitation spectra. Finally, the counting rules of spin excitation spectra are demonstrated as the fingerprints of the fractionalized quantum spin Hall states.

• Received 25 February 2014
• Revised 22 July 2014

DOI:https://doi.org/10.1103/PhysRevB.90.081102