Human pose estimation is a crucial problem in computer vision, and it has numerous applications in diverse fields such as virtual reality, surveillance, human-computer interaction, and action assistance. With the advent of edge computing, it is a promising paradigm to perform real-time artificial intelligence (AI)-assisted action based on pose estimation at the edge. However, task scheduling optimization for human pose estimation in edge computing is a challenging problem, due to the limited computing resources. In this paper, we propose a novel framework for task scheduling optimization in human pose estimation at the edge. Our framework takes computing resources scheduling and task scheduling decision into account, with the objective of maximizing the quality of service (QoS) of the system. We use multiple depth cameras at different locations to build three-dimensional (3D) poses to maintain the accuracy of estimation and to assist in guiding action. We evaluate our proposed framework on a real-world dataset. The results demonstrate its effectiveness in improving system delay and estimation accuracy in comparison with benchmark methods. We also verify the sensitivity of our proposed framework, which can provide insights into optimal parameter settings for different scenarios.