The semi-nonnegative matrix factorization (Semi-NMF) is a promising soft K-means clustering technique. Deep Semi-NMF, which stacks one-layer Semi-NMF into multi-layer, is able to learn the hierarchical projections and can obtain the deep hidden representations according to the unknown attributes of the given data. On the other hand, the inherent structure of the each data cluster can be described by the distribution of the intraclass data. Then one hopes to learn the new deep hidden representations which can preserve the intrinsic structures embedded in the original data space perfectly. Here seamlessly integrating the benefits of the Deep Semi-NMF and the distribution preserving strategy, we propose a novel distribution preserving-based deep semi-nonnegative matrix factorization method (DPNMF) to achieve this goal. In DPNMF, by maintaining the consistency of two distributions that can approximate the manifold structures, we can seek the deep hidden features which reveal the original intrinsic structures. As a result, the manifold structures in the raw data are well preserved in the new feature space. We also devise an adaptive projected Barzilai-Borwein method to optimize the proposed constrained objective function efficiently. The experimental results on the several real-world datasets show that the proposed DPNMF can achieve advantageous clustering performance in terms of accuracy (ACC), normalized mutual information (NMI), and adjusted rand index (ARI).