The recent emerging graph signal processing technologies have been widely applied to analyze signals defined on irregular domains, e.g., data collected from social networks, sensor networks, or transportation systems. Vertex frequency analysis, especially the windowed graph Fourier transform, is one of the most important tools for graph signal analysis and representations. Nevertheless, with a selected window function, it is rather challenging to construct tight frames via the windowed graph Fourier transform. To facilitate the construction of tight frames, in this paper, we consider multi-windowed graph Fourier transforms to develop novel vertex frequency analysis methods. Firstly, under the multi-windowed setting, tight graph Fourier frames are elaborately constructed to fulfill technical demands in different application scenarios. The canonical dual frames of the multi-windowed graph Fourier frames are investigated to establish the reconstruction formulas of graph signals. Additionally, we propose shift multi-windowed graph Fourier frames by directly using the shift operators, e.g., the adjacency matrix. The related tight frames, dual frames and their constructions are also discussed. Experimental results show that the proposed two types of frames can efficiently extract vertex-frequency features of synthetic graph signals. Furthermore, anomaly data can also be detected by these frames.