Image-based modeling and rendering (IBMR) techniques have been used to construct three-dimensional virtual scenes applied for VR applications by warping several two-dimensional images with depth information. In this paper, a novel approach of IBMR with depth information called double projective cylindrical texture mapping method (DPCM) is presented and implemented on an FPGA hardware platform. By the approach, based on a virtual double projective cylindrical camera model, the rendered scene is firstly projected onto the surface of a cylinder with perspective projection for the top and bottom view and parallel projection for the side view. Both color and depth images are captured after projection. Using the double projective cylindrical reference images (DPCI) as the source, a two-pass rendering process is conducted to generate new views of the 3D object and scene through a horizontal and vertical pre-warping followed by standard texture mapping. By taking the parallelism and reconfigurability of FPGA hardware, the algorithm is optimized for the architecture of circuit and well designed for memory management. Experimental results show that both improved rendering speed and good image quality could be achieved on the hardware platform.