Background: Any infectious disease outbreak may lead to a negative detrimental psychological impact on individuals and the community at large, however; there was no systematic review nor meta-analysis that examined the relationship between the psychological/mental health impact of SARS and COVID-19 outbreak in Asia. Methods and design: A systematic search was conducted using PubMed, EMBASE, Medline, PsycINFO, and CINAHL databases from 1/1/2000 to 1/6/2020. In this systematic review and meta-analysis, we analyzed the psychological impact on confirmed/suspected cases, healthcare workers and the general public during the Severe Acute Respiratory Syndrome (SARS) outbreak and Coronavirus disease (COVID-19) epidemics. Primary outcomes included prevalence of depression, anxiety, stress, post-traumatic stress disorder, aggression, sleeping problems and psychological symptoms. Result: Twenty-three eligible studies (N = 27,325) were included. Random effect model was used to analyze the data using STATA. Of these studies, 11 were related to the SARS outbreak and 12 related to COVID-19 outbreaks. The overall prevalence rate of anxiety during SARS and COVID-19 was 37.8% (95% CI: 21.1–54.5, P < 0.001, I2 = 96.9%) and 34.8% (95% CI: 29.1–40.4), respectively. For depression, the overall prevalence rate during SARS and COVID-19 was 30.9% (95% CI: 18.6–43.1, P < 0.001, I2 = 97.3%) and 32.4% (95% CI: 19.8–45.0, P < 0.001, I2 = 99.8%), respectively. The overall prevalence rate of stress was 9.4% (95% CI: −0.4 −19.2, P = 0.015, I2 = 83.3%) and 54.1% (95% CI: 35.7–72.6, P < 0.001, I2 = 98.8%) during SARS and COVID-19, respectively. The overall prevalence of PTSD was 15.1% (95% CI: 8.2–22.0, P < 0.001) during SARS epidemic, calculated by random-effects model (P < 0.05), with significant between-study heterogeneity (I2 = 93.5%). Conclusion: The SARS and COVID-19 epidemics have brought about high levels of psychological distress to individuals. Psychological interventions and contingent digital mental health platform should be promptly established nationwide for continuous surveillance of the increasing prevalence of negative psychological symptoms. Health policymakers and mental health experts should jointly collaborate to provide timely, contingent mental health treatment and psychological support to those in need to reduce the global disease burden. Systematic review registration: CRD42020182787, identifier PROSPER.