A new approach for binocular vision system calibration and 3D re-construction is proposed. While the system is calibrated, the sum of square distances between the vector coordinates recombined with the coordinates of feature points in the world frame and those in image frame to the fitted hyperplane is taken as an objective function. An orthogonal learning neural network is designed, where a self-adaptive minor component extracting method is adopted. When the network comes to equilibrium, the projective matrixes for the two cameras are obtained from the eigen-vectors of the autocorrelation matrix corresponding to the minimum eigen values, so the calibration of the binocular vision system is achieved. As for 3D re-construction, an autocorrelation matrix is obtained from feature point coordinates in image planes and calibration data, and an orthogonal learning network is designed. After the network is trained, the autocorrelation matrix’s eigen-vector corresponding to the minimum eigen-values is obtained, from which the 3D coordinates are obtained also. The proposed approach is a novel application of minor component analysis and orthogonal learning network in binocular vision system and 3D re-construction.