The recorded electroencephalography (EEG) signals are usually contaminated by many artifacts. In recent years, deep learning models have been used for denoising of electroencephalography (EEG) data and provided comparable performance with that of traditional techniques. However, the performance of the existing networks in electromyograph (EMG) artifact removal was limited and suffered from the over-fitting problem. Here we introduce a novel convolutional neural network (CNN) with gradually ascending feature dimensions and downsampling in time series for removing muscle artifacts in EEG data. Compared with other types of convolutional networks, this model largely eliminates the over-fitting and significantly outperforms four benchmark networks in EEGdenoiseNet. Our study suggested that the deep network architecture might help avoid overfitting and better remove EMG artifacts in EEG.