Inter-shield differences in the composition of lavas from Hawaiian volcanoes are generally thought to result from the melting of a heterogeneous mantle source containing variable amounts or types of oceanic crust (sediment, basalt, and/or gabbro) that was recycled into the mantle at an ancient subduction zone. Here we investigate the origin of chemical heterogeneity in the Hawaiian mantle plume by comparing the incompatible trace element abundances of tholeiitic basalts from (1) the three active Hawaiian volcanoes (Kilauea, Mauna Loa, and Loihi) and (2) the extinct Koolau shield (a compositional end member for Hawaiian volcanoes). New model calculations suggest that the mantle sources of Hawaiian volcanoes contain a significant amount of recycled oceanic crust with a factor of ∼2 increase from ∼8–16% at Loihi and Kilauea to ∼15–21% at Mauna Loa and Koolau. We propose that the Hawaiian plume contains a package of recycled oceanic crust (basalt and gabbro, with little or no marine sediment) that was altered by interaction with seawater or hydrothermal fluids prior to being variably dehydrated during subduction. The recycled oceanic crust in the mantle source of Loihi and Kilauea lavas is dominated by the uppermost portion of the residual slab (gabbro-free and strongly dehydrated), whereas the recycled oceanic crust in the mantle source of Mauna Loa and Koolau lavas is dominated by the lowermost portion of the residual slab (gabbro-rich and weakly dehydrated). The present-day distribution of compositional heterogeneities in the Hawaiian plume cannot be described by either a large-scale bilateral asymmetry or radial zonation. Instead, the mantle source of the active Hawaiian volcanoes is probably heterogeneous on a small scale with a NW–SE oriented spatial gradient in the amount, type (i.e., basalt vs. gabbro), and extent of dehydration of the ancient recycled oceanic crust.