As a mathematical representation of the output-to-output relationship, transmissibility function (TF) has been extensively applied in structural damage detection due to its robustness to influences of the input variations. As in most engineering fields, dealing with the problem of uncertainty in TF-based feature detection is an issue of fundamental importance. In this study, a new statistical, data-driven damage detection algorithm is proposed by rigorously modelling the variability of TF without postprocessing with circularly-symmetric complex Gaussian ratio distribution. The probabilistic distance of Symmetric Kullback-Leibler (SKL) divergence between TFs under baseline condition and potential damage scenarios which can measure the dissimilarity of probability distributions for the TFs under different states are computed as a damage index (DI) to detect structural anomaly. Compared against Mahalanobis distance which has the implicit assumption that the normal condition set is governed by Gaussian statistics, the probabilistic distance measure proposed in this study can deal with the deviations in TFs not following Gaussian distribution. A statistically rigorous threshold selection scheme integrating Bayesian inference strategy and Monte Carlo discordancy test is proposed to detect the the presence of damage by accommodating the uncertainties of measurements and the probabilistic model of TF. Numerical, experimental, and field test studies are conducted to validate the potential of probabilistic distance measure of TFs in anomaly detection under ambient vibration instead of forced vibration testing. Results demonstrate satisfactory performance of the proposed approach for detecting the existence and quantify the relative damage severity from a global perspective.