Combined heat and power dispatch (CHPD) effectively improves overall energy efficiency and operational flexibility in the integrated electricity and heating system (IEHS). However, the centralized optimization of CHPD may cause privacy infringement of different energy nodes that possess individual multi-energy resources and loads. This paper proposes a fully distributed optimization method of CHPD based on the alternating direction method of multipliers (ADMM), which promotes the optimal coordination among multi-agents in IEHS with high computational efficiency and respects agents' privacy. The nodes in IEHS are regarded as energy agents, while the energy networks connecting each agent fully model the heat transfer delay and the nonlinearity of power flow to ensure the accuracy of the CHPD results. According to ADMM, the auxiliary observation variables of adjacent agents are formulated to decompose the entire CHPD problem into many subproblems, which are solved by each energy agent in parallel iteratively. The closed-form solution of each subproblem is deduced to extremely accelerate the computation speed of distributed optimization. Case studies based on a test IEHS validate the effectiveness of the proposed method.