Metabolic demands of modern hybrid sows have increased over the years, which increases the chance that sows enter a substantial negative energy balance (NEB) during lactation. This NEB can influence the development of follicles and oocytes that will give rise to the next litter. To study effects of a lactational NEB on follicular development, we used 36 primiparous sows of which 18 were subjected to feed restriction (3.25 kg/day) and 18 were full-fed (6.5 kg/day) during the last 2 weeks of a 24.1 ± 0.3 day lactation. Feed restriction resulted in a 70% larger lactational body weight loss and 76% higher longissimus dorsi depth loss, but similar amounts of backfat loss compared to the full fed sows. These changes were accompanied by lower plasma insulin-like growth factor 1 (IGF1) and higher plasma creatinine levels in the restricted sows from the last week of lactation onward. Ovaries were collected 48 h after weaning. Restricted sows had a lower average size of the 15 largest follicles (−26%) and cumulus–oocyte complexes showed less expansion after 22 h in vitro maturation (−26%). Less zygotes of restricted sows reached the metaphase stage 24 h after in vitro fertilization and showed a higher incidence of polyspermy (+89%). This shows that feed restriction had severe consequences on oocyte developmental competence. Follicular fluid of restricted sows had lower IGF1 (−56%) and steroid levels (e.g., β-estradiol, progestins, and androgens), which indicated that follicles of restricted sows were less competent to produce steroids and growth factors needed for oocytes to obtain full developmental competence.