To improve vehicle adaptability to low-temperature environments, this paper proposes a combined energy and thermal management strategy (C-ETM) based on twin delayed deep deterministic policy gradient (TD3) algorithm for hybrid electric vehicles (HEVs). First, a vehicle energy management system (EMS) model and a engine-battery-cabin coupled thermal management system (CTMS) model are developed. By analyzing the coupling relationship between the CTMS and the EMS, a multiobjective optimization problem is constructed to minimize fuel consumption and battery aging damage and ensure SOC stability. Facing the challenges of solving optimization problems caused by the high-order complex nonlinearity of thermal-electrical coupling systems, the optimization problems are transformed into a Markov decision process (MDP). A reinforcement learning framework based on the TD3 algorithm is designed to achieve a real-time solution to the problem from a new perspective, overcoming the reliance on the system models and accurate future traffic information. The proposed strategy has efficient performance in terms of fuel economy, battery life, ensuring SOC stability, and adaptability. The total optimization cost reaches 91.42% level of the dynamic programming (DP) strategy, which is 30.3% lower than the model predictive control (MPC) strategy. The online computing burden is only 0.19% of the MPC strategy, which has strong potential for real-time applications.