Noise estimation is crucial in many image processing tasks such as denoising. Most of the existing noise estimation methods are specially developed for grayscale images. For color images, these methods simply handle each color channel independently, without considering the correlation across channels. In this work, we propose a multivariate Gaussian approach to model the noise in color images, in which we explicitly consider the inter-dependence among color channels. We design a practical method for estimating the noise covariance matrix within the proposed model. Specifically, a patch selection scheme is first introduced to select weakly textured patches through thresholding the texture strength indicators. Noticing that the patch selection actually depends on the unknown noise covariance, we present an iterative noise covariance estimation algorithm, where the patch selection and the covariance estimation are conducted alternately. Experimental results show that our method can effectively estimate the noise covariance. The practical usage is demonstrated with color image denoising.