Many soils of agricultural land in affluent countries have been enriched with phosphorus (P), because P application via fertilisers and manures was larger than P withdrawal via harvested biomass. This practice threatens the long-term availability of P fertilisers derived from finite rock phosphates, as well as surface water quality because of P leaching and run-off losses. In response, restrictions on P fertilisation have been implemented in some countries. The objective of this study is to examine the effects of balanced P fertilisation in comparison to a surplus P fertilisation on dry matter (DM) grass yield, grass quality, and soil P status. A 15 years’ lasting field experiment was conducted on four permanent grassland sites, on sand (two sites), peat and young marine clay in the Netherlands. Fertilisation levels, including cattle slurry, were aimed to implement P surpluses of 0, 9, and 18 kg P ha−1 year−1, and N surpluses of 180 and 300 kg ha−1 year−1. Grasslands were alternately grazed and mown, and grass yields and soil P levels were measured. Annual DM yield, P content, and P yield of grazed grassland were lower at balanced P fertilisation than at a surplus of 9 or 18 kg P ha−1 year−1 on sand and peat. Differences between P treatments remained constant over time. On the recently reclaimed marine clay, DM yield did not differ between P treatments, but P content and P yield did respond to different P surpluses. Differences between sites in the response to P surpluses were related to differences in soil P status, according to P-AL (capacity indicator) and P-CaCl2 (intensity indicator). At balanced P fertilisation, P-AL tended to decrease, while P-CaCl2 tended to remain constant. At surplus P, P-AL tended to increase and P-CaCl2 tended to remain constant. Herbage yield and P uptake also strongly responded to N treatments. In conclusion, there is a risk that balanced P fertilisation reduces herbage yield and P content relative to surplus P fertilisation, even at relatively high soil P status. The risk of yield reduction seems to be related to the ratio between the P intensity indicator and P capacity indicator.