Heating, ventilation, and air-conditioning (HVAC) systems are ideal demand-side flexible resources to provide regulation services. However, finding the best hourly regulation capacity offers for HVAC systems in a power market ahead of time is challenging because they are affected by non-Gaussian uncertainties from regulation signals. Moreover, since HVAC systems need to frequently regulate their power according to regulation signals, numerous thermodynamic constraints are introduced, leading to a huge computational burden. This paper proposes a tractable chance-constrained model to address these challenges. It first develops a temporal compression approach, in which the extreme indoor temperatures in the operating hour are estimated and restricted in the comfortable range so that the numerous thermodynamic constraints can be compressed into only a few ones. Then, a novel convexification method is proposed to handle the non-Gaussian uncertainties. This method leverages the Gaussian mixture model to reformulate the chance constraints with non-Gaussian uncertainties on the left-hand side into deterministic non-convex forms. We further prove that these non-convex forms can be approximated by mixed-integer second-order cone constraints that can be efficiently solved by off-the-shelf solvers. The optimality gap because of this approximation is marginal under mild conditions. Numerical experiments are conducted to validate the superiority of the proposed method.