Background: The COVID-19 pandemic has greatly affected treatment-seeking behaviors of psychiatric patients and their guardians. Barriers to access of mental health services may contribute to adverse mental health consequences, not only for psychiatric patients, but also for their guardians. This study explored the prevalence of depression and its association with quality of life among guardians of hospitalized psychiatric patients during the COVID-19 pandemic. Methods: This multi-center, cross-sectional study was conducted in China. Symptoms of depression and anxiety, fatigue level and quality of life (QOL) of guardians were measured with validated Chinese versions of the Patient Health Questionnaire – 9 (PHQ-9), Generalized Anxiety Disorder Scale – 7 (GAD-7), fatigue numeric rating scale (FNRS), and the first two items of the World Health Organization Quality of Life Questionnaire - brief version (WHOQOL-BREF), respectively. Independent correlates of depression were evaluated using multiple logistic regression analysis. Analysis of covariance (ANCOVA) was used to compare global QOL of depressed versus non-depressed guardians. The network structure of depressive symptoms among guardians was constructed using an extended Bayesian Information Criterion (EBIC) model. Results: The prevalence of depression among guardians of hospitalized psychiatric patients was 32.4% (95% CI: 29.7–35.2%). GAD-7 total scores (OR = 1.9, 95% CI: 1.8–2.1) and fatigue (OR = 1.2, 95% CI: 1.1–1.4) were positively correlated with depression among guardians. After controlling for significant correlates of depression, depressed guardians had lower QOL than non-depressed peers did [F = 29.24, p < 0.001]. “Loss of energy” (item 4 of the PHQ-9), “concentration difficulties” (item 7 of the PHQ-9) and “sad mood” (item 2 of the PHQ-9) were the most central symptoms in the network model of depression for guardians. Conclusion: About one third of guardians of hospitalized psychiatric patients reported depression during the COVID-19 pandemic. Poorer QOL was related to having depression in this sample. In light of their emergence as key central symptoms, “loss of energy,” “concentration problems,” and “sad mood” are potentially useful targets for mental health services designed to support caregivers of psychiatric patients.