A novel two-channel single-output joint transform correlator system with the Mach-Zehnder configuration using encoding technique based on HSV color space for color pattern recognition is introduced. The large zero order term can be removed directly by the Stokes relations in only one step in this structure. By the image encoding technique, the size of the liquid crystal spatial light modulators will be smaller with interlaced rearrangement of hue and saturation color components. Furthermore, the utilization of Lagrange multipliers to synthesize the reference image for reducing the correlation sidelobes is also studied. The computer numerical results are presented to verify the performance of the proposed system.