Recurrent neural networks have been reported as an effective approach to solve dynamic Lyapunov equations, which widely exist in various application fields. Considering that a bounded activation function should be imposed on recurrent neural networks to solve the dynamic Lyapunov equation in certain situations, a novel bounded recurrent neural network is defined in this paper. Following the definition, several bounded activation functions are proposed, and two of them are used to construct the bounded recurrent neural network for demonstration, where one activation function has a finite-time convergence property and the other achieves robustness against noise. Moreover, theoretical analyses provide rigorous and detailed proof of these superior properties. Finally, extensive simulation results, including comparative numerical simulations and two application examples, are demonstrated to verify the effectiveness and feasibility of the proposed bounded recurrent neural network.