The rapid development of extended reality (XR), including virtual and augmented reality, offers promising solutions to mediate human-robot interactions and support robotic construction in various forms. However, there is still a lack of systematic analysis of how emerging XR technologies and robotics could be effectively integrated to achieve mutual benefits and improve overall construction performance. This paper aims to critically review the existing literature on the integration of XR and robotics in construction based on a framework integrating the concept, methodology, and value dimensions, and to identify promising areas for future research in order to unleash the power of XR and robotics in construction. The paper offers a comprehensive understanding of what, how and why XR and robots are integrated in construction. In particular, the review critically analyzes the technology characteristics of XR and robotics, the integration pattern and modalities among humans, XR, and robots, and the multifaceted purposes and benefits in construction. The review indicates that VR is primarily used for simulation, training and teleoperation of robots in construction applications, while AR is implemented to provide additional information to humans for better interaction with robots and environments. The involvement of robots is generally industrial robots or mobile robots, with few studies considering specific construction robots in real-life scenarios. The findings point future directions for research to leverage XR for construction robot development and enhance XR-empowered human-robot collaboration. The review contributes to a new typology for XR-robot integration in construction and provides useful implications for researchers, practitioners, and policymakers in integrating robotics and XR to accelerate construction digitalization.