A high roll-off, third-order frequency selective surface (FSS) with quasi-elliptic bandpass response is presented. It is based on substrate integrated waveguide technology. Its unit cell includes three cavities that support at the TE101 mode. Interactions of resonators are dominated by magnetic couplings. The coupling matrix is analyzed and used to design FSS. Guidelines for a coupling-matrix-based FSS design are given. Transmission zeros realized close to the lower and upper narrow passband lead to high roll-off characteristics. In addition, a cross-polarization feature is easily achieved. Finally, a prototype is designed and fabricated to validate the proposed structure. A stable frequency response in passband has been obtained with an oblique incidence of 40° for TE polarization. Good agreement between the simulated and measured results verifies the design concept of bandpass FSS.