In order to deploy deep Convolutional Neural Networks (CNNs) on the mobile devices, many mobile CNNs are introduced. Currently, some online applications are usually re-trained because of the constantly-increasing data. However, compared with the regular models, it is not very efficient to train the present mobile models. Therefore, the purpose of this paper is to propose efficient mobile models both in the training and test processes through exploring the main causes of the current mobile CNNs’ inefficiency and the parameters’ properties. Finally, this paper introduces Highly Shared Convolutional Neural Networks (HSC-Nets). The HSC-Nets employ two shared mechanisms to reuse the filters comprehensively. Experimental results showed that, compared with the regular networks and the latest state-of-the-art group-conv mobile networks, the HSC-Nets can achieve promising performances and effectively decrease the model size. Furthermore, it is also more efficient in both the training and test processes.