Object classification based on tactile perception plays an essential role in robot manipulation process, as it serves for decision-making for the the downstream manipulation tasks. The demand for precise execution by industrial robots in smart factories has increased, and like humans, robots can infer tactile properties and identify object categories through brief motions. However, traditional practices only consider grasping as an instant state, resulting in the absence of time-series information. To address this issue, we propose a spatio-temporal attention-based Long Short-Term Memory (LSTM) network to solve the time-series problem for object classification. The proposed model utilizes a temporal attention mechanism that can dynamically trace the time-related features of the tactile data. Moreover, a spatial attention mechanism coordinates the integration of tactile information from various input features. The model classifies objects based on the entire temporal process of robot-object contact rather than data from a particular moment. To further enhance the model’s performance, we also incorporate PCA and Kalman filter. Our extensive experiments demonstrate the proposed model’s accuracy and efficiency, validating its ability to perform object classification based on tactile perception.