Recent researches on cancer segmentation in dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) usually resort to the combination of temporal kinetic characteristics and deep learning to improve segmentation performance. However, the difficulty in accessing complete temporal sequences, especially post-contrast images, hinders segmentation performance, generalization ability and clinical application of existing methods. In this work, we propose a diffusion kinetic model (DKM) that implicitly exploits hemodynamic priors in DCE-MRI and effectively generates high-quality segmentation maps only requiring pre-contrast images. We specifically consider the underlying relation between hemodynamic response function (HRF) and denoising diffusion process (DDP), which displays remarkable results for realistic image generation. Our proposed DKM consists of a diffusion module (DM) and segmentation module (SM) so that DKM is able to learn cancer hemodynamic information and provide a latent kinetic code to facilitate segmentation performance. Once the DM is pretrained, the latent code estimated from the DM is simply incorporated into the SM, which enables DKM to automatically and accurately annotate cancers with pre-contrast images. To our best knowledge, this is the first work exploring the relationship between HRF and DDP for dynamic MRI segmentation. We evaluate the proposed method for tumor segmentation on public breast cancer DCE-MRI dataset. Compared to the existing state-of-the-art approaches with complete sequences, our method yields higher segmentation performance even with pre-contrast images. The source code will be available on https://github.com/Medical-AI-Lab-of-JNU/DKM.