In the search for a highly active bulk material for the hydrogen evolution reaction (HER) under acidic conditions, we developed a simple solvothermal approach to synthesize Pd2B nanosheets supported on carbon, which achieves a low overpotential for the HER, 15.3 mV at 10 mA cm−2, a small Tafel slope of 22.5 mV dec−1 and a high exchange current density (j0) of 2.84 mA cm−2. The atomic structure evolution from Pd to Pd2B catalyst during synthesis is analyzed in detail via experimental and theoretical calculations, which shows that the slow insertion of B is assisted by the layer-by-layer fcc-to-hcp phase transition. Theoretical calculations further revealed that both the subsurface B and the lattice expansion after the hcp lattice formation play a key role to boost the HER activity. Since the Pd2B crystal is the global minimum in the Pd–B alloy, the success in the synthesis and demonstration of high HER activity paves the way towards further exploration of the catalytic performance for this stable metal boride material.