In this paper, a paradigm of momentum-based systems is introduced for nonconvex optimization. Based on the paradigm, a momentum-based system and a momentum-based multi-agent system are developed for nonconvex constrained optimization and distributed nonconvex optimization respectively, and the convergence and convergence rate to a local optimal solution are proven. In addition, a hybrid swarm intelligence algorithm is established, which consists of multiple momentum-based systems for scattering searches and a meta-heuristic rule for repositioning the states upon their local convergence. Two numerical examples are elaborated to verify and demonstrate the optimality, enhanced stability, and faster convergence of the proposed approaches.