Highly structured electron distribution functions in the electron diffusion region (EDR) during magnetic reconnection are studied by means of fully kinetic simulations. Four types of structures (striations, arcs, swirls, and rings) in momentum space are analyzed to understand their formation mechanisms. Discrete striations are formed by particles undergoing different numbers of meandering bounces in the EDR, and are a result of oscillations in the out-of-plane force on meandering electrons. Predictions for the separation between striations and the triangular shape of the distribution are obtained analytically. Arcs and swirls are due to partial re-magnetization of accelerated electrons. Near the end of the outflow jet, electron re-magnetization gives rise to the ring structure. Understanding the distribution structures is critical to unraveling the kinetic processes occurring in the EDR, and will guide the identification of EDRs based on satellite measurements.