Permeable (sandy) sediments cover half the continental margin and are major regulators of oceanic carbon cycling. The microbial communities within these highly dynamic sediments frequently shift between aerated and anoxic states, and hence are less stratified than cohesive (muddy) sediments. A major question is therefore how these communities maintain metabolism during oxic-anoxic transitions. Here we show that molecular hydrogen (H2) accumulates in silicate sand sediments due to decoupling of fermentative and respiratory bacteria following anoxia. In situ measurements show that H2 is supersaturated by up to 250-fold in the water column overlying these sediments and has an isotopic composition consistent with fermentative production. Community and shotgun metagenomic profiling suggests that the sediments harbor diverse and specialized microbial communities with a high abundance of [NiFe]-hydrogenase genes. The hydrogenase profiles predict that H2 is primarily produced by facultatively fermentative bacteria and can be consumed by aerobic respiratory bacteria. Consistently, we demonstrate through flow-through reactor and slurry experiments that H2 is (i) rapidly produced by fermentation following anoxia, (ii) immediately consumed by aerobic respiration following reaeration, and (iii) only consumed by sulfate reduction during prolonged anoxia. We also detected high abundance and activity of hydrogenotrophic sulfur, nitrate, and nitrite reducers. In combination, these experiments confirm that fermentation dominates anoxic carbon mineralization in permeable sediments and, in contrast to cohesive sediments, is largely uncoupled from anaerobic respiration. The frequent changes in oxygen availability in these sediments may have selected for metabolically flexible bacteria while excluding strict anaerobes.