Multimodal clustering has become a fundamental and important problem in the data mining community since the development of multimedia technology over the last two decades has led to a tremendous increase in unlabeled multimodal data. Although a panoply of multimodal subspace clustering methods shows promising performance via fusing information from different views of multimodal data, most of them consist of two sequential steps, i.e., learning a consensus affinity matrix from the original data and then feeding the resulting affinity matrix into the framework of spectral clustering. However, this leads to the suboptimal clustering performance due to the following limitations: 1) the two steps of learning the affinity matrix and clustering are carried out independently; 2) the affinity matrix may be unreliable; 3) the post-processing requirement, such as K-means. To address these issues, we propose a novel multimodal subspace clustering method via adaptively learning a similarity graph on a latent low-rank representation space. In particular, the number of connected components of the learned graph is precisely equal to the number of clusters, i.e., the optimal solution of the associated problem directly reveals the clustering structure of data. Extensive evaluations on several benchmark multimodal datasets demonstrate that the proposed approach outperforms state-of-the-art methods.