Recently many researchers invented a wide variety of meta-heuristic optimization algorithms and they have achieved remarkable performance results. Through observing natural phenomena, clues were inspired and programmed into search logics, such as PSO, Cuckoo Search and so on. Although those algorithms have promising performance, there still exist a drawback-it is hard to find a perfect balance between the global exploration and local exploitation from the traditional swarm optimization algorithms. Like an either-or problem, algorithms that have better global exploration capability come with worse local exploitation capability, and vice versa. In order to address this problem, in this paper, we propose a novel Dynamic Group Search Algorithm (DGSA) with enhanced intra-group and inter-group communication mechanisms. In particular, we devise a formless 'group' concept, where the vectors of solutions can move to different groups dynamically based on the group best solution fitness, the better group has the more vectors. Vectors inside a group mainly focus on the local exploitation for enhancing its local search. In contrast, inter group communication assures strong capability of global exploration. In order to avoid being stuck at local optima, we introduce two types of crossover operators and an inter-group mutation. Experiments using benchmarking test functions for comparing with other well-known optimization algorithms are reported. DGSA outperform others in most cases.