We present a comprehensive investigation of the low-field hole mobility in strained Ge and III-V (GaAs, GaSb, InSb and In_{1− x}Ga_xAs) p-channel inversion layers with both SiO₂ and high-κ insulators. The valence (sub)band structure of Ge and III-V channels, relaxed and under strain (tensile and compressive) is calculated using an efficient self-consistent method based on the six-band k · p perturbation theory. The hole mobility is then computed using the Kubo-Greenwood formalism accounting for non-polar hole-phonon scattering (acoustic and optical), surface roughness scattering, polar phonon scattering (III-Vs only), alloy scattering (alloys only) and remote phonon scattering, accounting for multi-subband dielectric screening. As expected, we find that Ge and III-V semiconductors exhibit a mobility significantly larger than the “universal” Si mobility. This is true for MOS systems with either SiO₂ or high-κ insulators, although the latter ones are found to degrade the hole mobility compared to SiO₂ due to scattering with interfacial optical phonons. In addition, III-Vs are more sensitive to the interfacial optical phonons than Ge due to the existence of the substrate polar phonons. Strain—especially biaxial tensile stress for Ge and biaxial compressive stress for III-Vs (except for GaAs)—is found to have a significant beneficial effect with both SiO₂ and HfO₂. Among strained p-channels, we find a large enhancement (up to a factor of 10 with respect to Si) of the mobility in the case of uniaxial compressive stress added on a Ge p-channel similarly to the well-known case of Si. InSb exhibits the largest mobility enhancement. In_0.7Ga _0.3As also exhibits an increased hole mobility compared to Si, although the enhancement is not as large. Finally, our theoretical results are favorably compared with available experimental data for a relaxed Ge p-channel with a HfO₂ insulator.