This paper presents a new approach on 2D to 3D video automatic conversion based on Kinect. There are mainly two contributions in our approach. First, the depth maps captured by Kinect are usually of poor quality. There are a lot of depth information missing regions. An improved energy function based Inpainting algorithm is proposed, which can help to recover the missed depth information smoothly. Then a new hole filling approach in depth image based rendering (DIBR) is proposed to synthesize the 3D images by using the corresponding color images and the repaired depth maps. Experiments are conducted on videos in different scenes. The results show the effectiveness in both the quality and the running time.