While multiple model adaptive control (MMAC) scheme provides a solution to systems with unknown and rapidly time-varying parameters, many offline samples must be obtained beforehand, and the number of models is difficult to be found if no prior knowledge is given. This paper proposes a new adaptive control strategy to handle such systems. The principle is to use a change detection mechanism to check if there is an abrupt change, and immediately train a new model if a change is detected. A novel online identification algorithm, namely initial-training-free online extreme learning machine (ITF-OELM), is also proposed to allow the model to be trained anytime without concerns on prior data. With this strategy, only one model is necessary as compared to MMAC, resulting in reduction on computational complexity and memory usage. Simulation results show that the proposed strategy is effective. Besides, although the use of forgetting factor in ITF-OELM can accelerate the convergence speed for system identification, sometimes it may lead to ill-conditioned covariance matrix in the recursively updating process. This paper shows that such issue can be solved by the change detection mechanism.