In frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems, accurate channel state information (CSI) needs to be acquired via CSI feedback to reap the potential benefits of massive MIMO. However, the large-scale antenna array enlarges the dimension of the CSI matrix to be fed back and thus leads to unaffordable CSI feedback overhead. In addition, the quantization and dequantization processes in CSI feedback unavoidably introduce non-neglectable quantization errors, which greatly restrict the performance of CSI feedback. To this end, in this paper, we propose a Transformer-based CSI feedback method with a hybrid learnable non-uniform quantization method to eliminate quantization errors and improve CSI feedback accuracy with reduced feedback overhead. Experimental results on a public dataset demonstrate that the proposed Transformer-based CSI feedback method can achieve higher CSI feedback accuracy with the help of the hybrid learnable non-uniform quantization method.