Extreme learning machine (ELM) is an emerging machine learning algorithm and widely used in various real-world applications due to its extremely fast training speed, good generalization and universal approximation capability. In order to further explore the ELM to be used in practical embedded systems, a task-parallel and reconfigurable FPGA-based hardware architecture of ELM algorithm is presented in this paper. The proposed architecture performs the on-chip machine learning for both training and prediction phases which are implemented parameterizably based on the reconfigurable parameters. Meanwhile, the task-parallel efforts are focused on the training phase to improve the computational efficiency by resolving the serial computations into subtasks for task-parallel computations. In addition, the on-chip block RAMs reuse scheme is also applied in proposed architecture for saving on-chip resource consumption. The experimental results show that the proposed ELM architecture can achieve similar accuracy compared with floating-point implementation on Matlab and outperform the recently published ELM implementations in terms of hardware performance, power consumption and resource utilization.