The image down-conversion may be used in the block-based image compression because it can help save lots of bit-counts for each individual block. A straightforward way to implement the transform domain down-conversion is to truncate some high-frequency components to get a down-sized coefficient block. However, directly using this down-sized coefficient block to reconstruct a completed image block will lead to a serious quality degradation. In this paper, we propose a constrained quantization based transform domain down-conversion (CQTDD) to help compress each 16×16 macro-block and it makes the coding quality of 1/4 selected pixels (according to a regular pattern) in each macro-block much higher than that can be achieved by using the traditional truncation based approach. Meanwhile, the other 3/4 pixels will be interpolated by using those 1/4 well-reconstructed pixels. Furthermore, these 1/4 pixels are optimized before the compression to help get a more efficient interpolation. Finally, the proposed CQTDD works with the JPEG baseline coding together as two candidate coding modes in our proposed compression scheme. Experimental results demonstrate that our proposed method may offer a remarkable quality gain, both objectively and subjectively, compared with some existing methods.