User interfaces have traditionally followed the WIMP (window, icon, menu, pointer) paradigm. Though functional and powerful, they are usually cumbersome for a novice user to design a complex model, requiring considerable expertise and effort. This paper presents a system for designing geometric models and image deformation with sketching curves, with the use of Green coordinates. In 3D modeling, the user first creates a 3D model by using a sketching interface, where a given 2D curve is interpreted as the projection of the 3D curve. The user can add, remove, and deform these control curves easily, as if working with a 2D line drawing. For a given set of curves, the system automatically identifies the topology and face embedding by applying graph rotation system. Green coordinates are then used to deform the generated models with detail-preserving property. Also, we have developed a sketch-based image-editing interface to deform image regions using Green coordinates. Hardware-assisted schemes are provided for both control shape deformation and the subsequent surface optimization, the experimental results demonstrate that 3D/2D deformations can be achieved in realtime.