As a certain case in the domain of human actions, hand gestures can be expressed by the motion of user's hand to provide nature interaction information in many applications. This paper proposed a hand gesture recognition system based on sparse 3D interest point detector, the spatial-temporal filtering achieved by separable linear Gaussian-Gabor filters detects local motion corners as strong response. The motion trajectory is extracted from the response and different feature formulations are introduced for gesture classification with several solutions. Experimental results demonstrate that the trajectory feature from spatial-temporal (ST) filtering is more reliable than cuboids descriptor, and SVM overcomes KNN and HMM for classification. © 2011 ACM.