Background: This study examined the relationship between childhood circumstances and health in middle and later life. We quantified how childhood circumstances contribute to health in later life, both directly and indirectly, through their effects on potential mediators. Methods: This study used three waves of data from the national longitudinal survey of the China Health and Retirement Longitudinal Study (CHARLS). The final model in this study included 7,476 eligible respondents aged 45 years and above. We constructed a simple health status measure based on the first principal component of CHARLS survey responses with 25 health-related information. It is a multi-dimensional measurement that comprehensively reflects the individual's healthy aging. We formulated childhood circumstances factors into five domains: childhood health and nutrition, childhood socioeconomic status, access to health care, parental genetics, and adverse childhood experiences. Ordered logit regression was conducted to analyze the relationship between health in middle and later life and childhood circumstances, with other explanatory variables controlled. Results: Controlling for educational attainment, personal income, and health status in the last wave, adults who experience good childhood health (poor as the base, coefficient 0.448, p < 0.01), and better family financial status (worse as the base, coefficient 0.173, p < 0.01) have significantly better health during their middle and later life, in comparison, being inconvenient to visit a doctor (coefficient −0.178, p < 0.01), and having two or three adverse childhood experiences (0 as the base, coefficient −0.148, p < 0.01) are significantly associated with poorer health. Childhood circumstances appear to act both through a lasting effect of initial health and financial status in childhood and through their impact on achievements in adulthood. Conclusion: Our findings suggest that investments in health during childhood not only contribute to health in later life but also dynamically improve an individual's educational attainment and personal income, as well as other life prospects. All these returns may extend far beyond childhood and continue throughout the lifespan.