This paper presents a finite element method-based framework for an object withering simulation modeled with heterogeneous material, such as fruits drying or decay. We introduce diffusion procedures for both the moisture content and decay spread, which are solved directly on a tetrahedral mesh representation of the fruit flesh. Then, we use the moisture content to control shrinkage through the initial strain, which is integrated into the Lagrangian dynamic equation, and solved with the finite element method. For the complex structure of the object, another fine triangle mesh is used to represent the skin, and its deformation is solved by a thin shell technique. To couple the motion between different layers of the fruit, a tracking force is used to pull the skin and drive its deformation together with the volume mesh. In comparison with the previous work, our method provides temporally and spatially varying parameters to model the complex phenomena of object withering. Moreover, the water diffusivity can also be given by user input to present various material properties of the cut section and skin-covered area. Our algorithm is easy to implement and highly efficient in generating a realistic appearance for the withering effect. For a medium-scale model, we can achieve interactive simulation. Copyright © 2012 John Wiley & Sons, Ltd.