Assessing student learning outcomes has become a global trend in higher education. In this paper, we report on the validation of the Chinese HEIghten® Critical Thinking assessment with a nationally representative sample of Electrical Engineering and Computer Science students from 35 institutions in China. Key findings suggest that there was a test delivery mode effect favoring the paper tests over the online tests. In general, the psychometric quality of the items was satisfactory for low-stakes, group-level uses but there were a few items with low discrimination that awaits further investigation. The relationships between test scores and various external variables such as college entrance examination scores, university elite status and student perceptions of the test were as expected. We conclude with speculations on the key findings and discussion of directions for future research.