Ku80 is a subunit of the Ku heterodimer that binds to DNA double-strand break ends as part of the non-homologous end joining (NHEJ) pathway. Ku80 is also involved in homologous recombination (HR) via its interaction with BRCA1. Ku80 is encoded by the XRCC5 gene that contains a variable number tandem repeat (VNTR) insertion in its promoter region. Different VNTR genotypes can alter XRCC5 expression and affect Ku80 production, thereby affecting NHEJ and HR pathways. VNTR polymorphism is associated with multiple types of sporadic cancer. In this study, we investigated its potential association with familial breast cancer at the germline level. Using PCR, PAGE, Sanger sequencing, and statistical analyses, we compared VNTR genotypes in the XRCC5 promoter between healthy individuals and three types of familial breast cancer cases: mutated BRCA1 (BRCA1+), mutated BRCA2 (BRCA2+), and wild-type BRCA1/BRCA2 (BRCAx). We observed significant differences of VNTR genotypes between control and BRCA1+ group (P < 0.0001) and BRCA2+ group (P = 0.0042) but not BRCAx group (P = 0.2185), and the differences were significant between control and cancer-affected BRCA1+ cases (P < 0.0001) and BRCA2+ cases (P = 0.0092) but not cancer-affected BRCAx cases (P = 0.4251). Further analysis indicated that 2R/2R (OR = 1.94, 95%CI = 1.26–2.95, P = 0.0096) and 2R/1R (OR = 1.58, 95%CI = 1.11–2.26, P = 0.0388) were associated with increased risk but 1R/1R (OR = 0.55, 95%CI = 0.35–0.84, P = 0.0196) and 1R/0R (OR = 0, 95%CI = 0–0.29, P = 0.0012) were associated with decreased risk in cancer-affected BRCA1+ group; 2R/1R (OR = 1.94, 95%CI = 1.14–3.32, P = 0.0242) was associated with increased risk in cancer-affected BRCA2+ group. No correlation was observed for the altered risk between cancer-affected or -unaffected carriers and between different age of cancer diagnosis in cancer-affected carriers. The frequently observed VNTR association with in BRCA1+ and BRCA2+ breast cancer group indicates that VNTR polymorphism in the XRCC5 promoter is associated with altered risk of breast cancer in BRCA1+ and BRCA2+ carriers.