The rapid compute hardware development has shifted to the hybrid architecture consisting of both CPUs and GPUs. For the class of problems described by partial differential equations (PDEs) discretized by finite difference or finite element methods, obtaining reasonably good performance on a CPU/GPU platform is still a challenge. In this paper, we propose and test an algorithm with several building layers matching the hybrid architecture. The scalability of the approach is obtained by a two-level domain decomposition method, and the GPU performance is effectuated by using various local multigrid methods with suitable smoothers. Efficiency and scalability of our algorithm are demonstrated by convincing performance analysis on a hybrid platform of CPU/GPU.