Linear discriminant analysis (LDA) is a popular technique for supervised classification problems, and it works quite well when the number of classes is small, but the accuracy deteriorates when the number of classes becomes large. In this paper, we propose a domain decomposed method and an iteratively deflated method to improve the classification accuracy. In the domain decomposed LDA, we decompose the given dataset into subsets and apply LDA separately to each subset for the training part of the algorithm. In the testing step, we project the samples into multiple subspaces, contrary to the full space as in the traditional LDA. From the multiple low-dimensional projections we determine the class or classes that the sample belongs to. An optimality theory is developed to show why the new method offers better classification under a technical assumption. In the iteratively deflated method, the traditional LDA method serves as the initial iteration from which we select separable classes to be deflated from the training set, and the remaining samples in the dataset are used for the next iteration. As the process goes on we generate a sequence of projection matrices that are used to determine which class or classes a sample belongs to using certain classification criteria. With the proper choices of the quantile radii in the separable criteria for the training and testing phases, we show that the proposed method is much more accurate than the traditional LDA. To test and compare the two proposed methods, we consider the popular datasets CIFAR-10/100 and a gene expression dataset of cancer patients, and show that the new approaches outperform the traditional LDA by a large margin.