We investigate experimentally the backscattering properties of an array of atoms that is evanescently coupled to an optical nanofiber in the strongly nonparaxial regime. We observe that the power and the polarization of the backscattered light depend on the nanofiber-guided excitation field in a way that significantly deviates from the predictions of a simple model based on two-level atoms and a scalar waveguide. Even though it has been widely used in previous experimental and theoretical studies of waveguide-coupled quantum emitters, this simple model is thus in general not adequate even for a qualitative description of such systems. We develop an ab initio model that includes the multilevel structure of the atoms and the full vectorial properties of the guided field and find very good agreement with our data.

• Received 13 February 2014

DOI:https://doi.org/10.1103/PhysRevA.89.031804