Sex-biased genes are genes with a preferential or specific expression in one sex and tend to show an accelerated rate of evolution in animals. Various hypotheses—which are not mutually exclusive—have been put forth to explain observed patterns of rapid evolution. One possible explanation is positive selection, but this has been shown only in few animal species and mostly for male-specific genes. Here, we present a large-scale study that investigates evolutionary patterns of sex-biased genes in the predominantly self-fertilizing plant Arabidopsis thaliana . Unlike most animal species, A. thaliana does not possess sex chromosomes, its flowers develop both male and female sexual organs, and it is characterized by low outcrossing rates. Using cell-specific gene expression data, we identified genes whose expression is enriched in comparison with all other tissues in the male and female gametes (sperm, egg, and central cell), as well as in synergids, pollen, and pollen tubes, which also play an important role in reproduction. Genes specifically expressed in gametes and synergids show higher rates of protein evolution compared with the genome-wide average and no evidence for positive selection. In contrast, pollen- and pollen tube-specific genes not only have lower rates of protein evolution but also exhibit a higher proportion of adaptive amino acid substitutions. We show that this is the result of increased levels of purifying and positive selection among genes with pollen- and pollen tube-specific expression. The increased proportion of adaptive substitutions cannot be explained by the fact that pollen- and pollen tube-expressed genes are enriched in segmental duplications, are on average older, or have a larger effective population size. Our observations are consistent with prezygotic sexual selection as a result of interactions during pollination and pollen tube growth such as pollen tube competition.