The prediction of rare events is a pressing scientific problem. Events such as extreme meteorological conditions, may aggravate human morbidity and mortality. Yet, their prediction is inherently difficult as, by definition, these events are characterised by low occurrence, high sampling variation, and uncertainty. For example, earthquakes have a high magnitude variation and are irregular. In the past, many attempts have been made to predict rare events using linear time series forecasting algorithms, but these algorithms have failed to capture the surprise events. This study proposes a novel strategy that extends existing GMDH or polynomial neural network techniques. The new strategy, called residualfeedback, retains and reuses past prediction errors as part of the multivariate sample data that provides relevant multivariate inputs to the GMDH or polynomial neural networks. This is important because the strength of GMDH, like any neural network, is in predicting outcomes from multivariate data, and it is very noise-tolerant. GMDH is a well-known ensemble type of prediction method that is capable of modelling highly non-linear relations. It achieves optimal accuracy by testing all possible structures of polynomial forecasting models. The performance results of the GMDH alone, and the extended GMDH with residual-feedback are compared for two case studies, namely global earthquake prediction and precipitation forecast by ground ozone information. The results show that GMDH with residualfeedback always yields the lowest error.