Ab initio calculations of electron-phonon interactions including the polar Fröhlich coupling have advanced considerably in recent years. The Fröhlich electron-phonon matrix element is by now well understood in the case of bulk three-dimensional (3D) materials. In the case of two-dimensional (2D) materials, the standard procedure to include Fröhlich coupling is to employ Coulomb truncation, so as to eliminate artificial interactions between periodic images of the 2D layer. While these techniques are well established, the transition of the Fröhlich coupling from three to two dimensions has not been investigated. Furthermore, it remains unclear what error one makes when describing 2D systems using the standard bulk formalism in a periodic supercell geometry. Here we generalize previous work on the ab initio Fröhlich electron-phonon matrix element in bulk materials by investigating the electrostatic potential of atomic dipoles in a periodic supercell consisting of a 2D material and a continuum dielectric slab. We obtain a unified expression for the matrix element, which reduces to the existing formulas for three-dimensional and 2D systems when the interlayer separation tends to zero or infinity, respectively. This expression enables an accurate description of the Fröhlich matrix element in 2D systems without resorting to Coulomb truncation. We validate our approach by direct ab initio density-functional perturbation theory calculations for monolayer BN and MoS2, and we provide a simple expression for the 2D Fröhlich matrix element that can be used in model Hamiltonian approaches. The formalism outlined in this work may find applications in calculations of polarons, quasiparticle renormalization, transport coefficients, and superconductivity, in 2D and quasi-2D materials.