Accurate segmentation of brain tumors is very essential for brain tumor diagnosis and treatment plans. In general, brain tumor includes WT (whole tumor), TC (tumor core) and ET (enhance tumor), and TC and ET are much more important than WT clinically. However, TC and ET usually contain blurred boundaries, and occupy much fewer pixels than WT. Recently, MetricUNet based on voxel-metric learning is proposed, which considers voxel-level feature relationship in the image to obtain finer segmentation results. However, it may not be applicable in brain tumor segmentation. That is because the scales/sizes of brain tumor greatly vary between images and causing ineffective model training in MetricUNet. Moreover, it has heavy computation for considering voxel-level feature relationship in brain tumor segmentation. In this work, a Scale-adaptive Super-feature based MetricUNet (S2MetricUNet) is proposed and provides two advantages: i) higher accuracy on TC and ET since a novel scale-adaptive metric loss is proposed for learning of more context information about TC and ET while addressing the scale variation between images; ii) significant reduction on computation since a super voxel-level feature is proposed to represent a group of voxel-level features (of the same label) in non-edge regions. The experimental results on public dataset BraTS2019 have demonstrated that the improvement of our method is up to 3.38% on TC and 3.82% on ET in terms Dice. Moreover, the computation of our S2MetricUNet is reduced to about 1/11 of MetricUNet.