Building heating, ventilation, and air conditioning (HVAC) and associated energy consumption make up the more and more important part of the world, whose reduction provides a cost-effective path to the “dual carbon” goal. This paper proposes a two-layer management of HVAC-based multi-energy buildings under proactive demand response of fast/slow-charging electric vehicles (EVs). In this paper, the building HVAC is mathematically formulated via a R-C thermodynamic model, which coordinates with multi-energy converters and storages to form a 100% renewable building. The building management is a challenging optimization problem due to its severe constraints and strong spatio-temporal couplings. In the first layer, a day-ahead multi-energy dispatch is formulated to economically optimize the electrical, heat, gas energy carriers. In the second layer, alternating current (AC) slow charging and direct current (DC) fast charging types of EVs are considered and managed via a novel real-time supply–demand pricing mechanism. After acquiring the economical dispatch references in the first layer, the second layer implements a model predictive control (MPC)-based real-time scheduling to handle the multi-energy supply–demand fluctuations. The original two-layer optimization is further handled via mixed-integer linear program (MILP) reformulation for high-efficient solving. Comparisons have shown the advantageous performances of the proposed two-layer optimization over economics and practicability. Simulations results show that the overall system operating cost can be reduced by at most 3.01% with a higher operational flexibility in building management.