As the diversification of frequency regulation resources, it is increasingly important to find a load frequency control (LFC) scheme that can effectively coordinate different resources. This paper proposes a coordinated but differentiated LFC control method that utilizes heterogeneous frequency regulation resources with their characteristics fully considered. The optimal LFC models for the single area and multi-area power systems are formulated as partially observable Markov decision process model (POMDP) and partially observable Markov game (POMG) considering the accessibility of system information. A deep reinforcement learning method called recurrent proximal policy optimization is leveraged to solve the POMDP and POMG models. Simulations based on historical data demonstrate that the proposed control scheme can better coordinate different resources and achieve superior control performance.