Graph signal processing is a new research field in the signal processing community. Recently, windowed graph Fourier frames have been proposed to analyze graph signals. However, it is not easy to construct tight frames by using signal window function. In this paper, we extend the windowed graph Fourier transform to the multi-window case. The multi-windowed graph Fourier frames offer more freedom on constructing tight frames. We provide some results for constructing multi-windowed graph Fourier frames, dual frames and tight frames. When applying the multi-windowed graph Fourier transform on synthesis graph signals, a dual localization phenomenon can be found in the original and dual windowed graph Fourier transform coefficients.