Except for linguistic interpretability and uncertainty-handling ability, fuzzy Apriori method (FAM) is being hurdled by both very expensive computational burdens and low generalization capability caused by serious correlation between short to long fuzzy rules generated. The novel doubly interpretable classifier DI-FAM with FAM-based hybrid structure is proposed to circumvent the above shortcomings of FAM. DI-FAM successively stacks the short rule bundles of each FAM sub-classifier on both a sampled feature subset and the outputs of the previous stacking layer. DI-FAM then finds out the output weights of short rule bundles at each stacking layer, followed by a linear sub-classifier (as a compensator) on all the original input features with one-step wide calculation. DI-FAM has four distinct merits: (1) low computational complexity stemmed from both its fast generation way of short rule bundles by FAM sub-classifiers respectively on their own features, and its one-step calculation for the output weights. (2) theoretical guarantee about no violation of the importance ranking orders of the short rules by each FAM sub-classifier on its own features at each stacking layer with regard to all the fuzzy rules by FAM on all the input features. (3) enhanced generalization capability by successively stacking short rule bundles at each stacking layer according to the stacked generalization principle. (4) double interpretability that DI-FAM shares both linguistic interpretability of all FAM sub-classifiers and feature-importance-based interpretability of a linear sub-classifier. Extensive experimental results indicate the effectiveness of DI-FAM in the sense of classification performance, training speed, incremental learning and double interpretability.