In experimental plant communities, relationships between biodiversity and ecosystem functioning have been found to strengthen over time1, 2, a fact often attributed to increased resource complementarity between species in mixtures3 and negative plant–soil feedbacks in monocultures4. Here we show that selection for niche differentiation between species can drive this increasing biodiversity effect. Growing 12 grassland species in test monocultures and mixtures, we found character displacement between species and increased biodiversity effects when plants had been selected over 8 years in species mixtures rather than in monocultures. When grown in mixtures, relative differences in height and specific leaf area between plant species selected in mixtures (mixture types) were greater than between species selected in monocultures (monoculture types). Furthermore, net biodiversity and complementarity effects1, 2 were greater in mixtures of mixture types than in mixtures of monoculture types. Our study demonstrates a novel mechanism for the increase in biodiversity effects: selection for increased niche differentiation through character displacement. Selection in diverse mixtures may therefore increase species coexistence and ecosystem functioning in natural communities and may also allow increased mixture yields in agriculture or forestry. However, loss of biodiversity and prolonged selection of crops in monoculture may compromise this potential for selection in the longer term.