Bumble bees (Bombus) are a group of eusocial bees with a strongly generalised feeding pattern, collecting pollen from many different botanical families. Though predominantly generalists, some bumble bee species seem to have restricted dietary choices. It is unclear whether restricted diets in bumble bees are inherent or a function of local conditions due to a lack of data for many species across different regions. The objective of this study was to determine whether bumble bee species displayed specific patterns of pollen collection, and whether patterns were influenced by phylogenetic relatedness or tongue length, a trait known to be associated with structuring floral visitation. Bumble bee pollen collection patterns were quantified from 4,132 pollen loads taken from 58 bumble bee species, representing 24% of the pollen-collecting diversity of this genus. Phylogenetic trait mapping showed a conserved pattern of dietary dissimilarity across species, but not for dietary breadth. Dietary dissimilarity was driven by collection of Fabaceae, with the most similar species collecting around 50%–60% of their diet from this botanical family. The proportion of the diet collected from Fabaceae also showed a conserved phylogenetic signal. Greater collection of Fabaceae was associated with longer tongue lengths, with shorter tongued species focusing on alternative botanical families. However, this result was largely driven by phylogenetic relatedness, not tongue length per se. These results demonstrate that, though generalists, bumble bees are still subject to dietary restrictions that constrain their foraging choices. These dietary constraints have implications for their persistence should their core resources decline in abundance.