Hand-drawn animation is one of the essential media to convey ideas, promote products, and tell a story. Despite these benefits, creating drawing animations still remains challenging even for experienced professionals [Xing, 2015]. Prior work have proposed animation authoring tools that can deform shapes [Igarashi, 2005], add kinetic texture [Kazi, 2014], and frame-by-frame animations [Xing, 2015]. However, combining hand-drawn animations with the kinematic movements in the real-world examples is still difficult and tedious process, and also requires domain-specific knowledge [Igarashi, 2005]. In this project, we propose a system to add kinematic animations into hand-drawn sketch by capturing and importing movements from the real-world examples. In the current prototype, the system has two features: sketch beautification and facial expression based animation. (See the video for more detail.)
In the sketch beautification, the system first recognizes a user’s stroke as Bezier path. Based on the Bezier path, the system categorizes three common shapes; lines, arcs, and circles by calculating distances between segments, and angles between tangents of the path. To detect the difference between arc and circle (or eclipse), the system also calculate the total angles and if the span is close to 2π (=6.28), then the arc is replaced with full circle. If we apply this method to more complex beautification such as connection, line alignment, and perpendicular detection, the search space will be too big to compute. To avoid this computation problem, we adapted to the basic algorithm discussed in [Igarashi, 1997], [Cheema, 2012], [Fišer, 2015] to reduce the computational cost by search graph pruning.
To detect face expression, we adapt Saragih et al’s algorithm [Saragih, 2012]. This technique provides roughly 70 line segments that express the facial emotion, so based on the position of these lines, we create an animation. In the current prototype, we do not implement automatic detection of the sketch. Instead, we arbitrarily label the parts of the face based on the position and shape. This will be improved in the future implementation.
Cod on GitHub: https://github.com/ryosuzuki/drawing-tool
Minimum: Detecting a face or a body in a semi-automated way, and moving these parts by manually through inverse kinematic.
Target: Based on the user’s paint stroke, automatically detect the structure such as skeleton, and move them with Kinect or face detection.
Stretch: Import movements from real-world examples such as YouTube video.
- Xing, Jun, Li-Yi Wei, Takaaki Shiratori, and Koji Yatani. “Autocomplete hand-drawn animations.” ACM Transactions on Graphics (TOG) 34, no. 6 (2015): 169.
- Igarashi, Takeo, Tomer Moscovich, and John F. Hughes. “As-rigid-as-possible shape manipulation.” ACM transactions on Graphics (TOG) 24, no. 3 (2005): 1134-1141.
- Kazi, Rubaiat Habib, Fanny Chevalier, Tovi Grossman, Shengdong Zhao, and George Fitzmaurice. “Draco: bringing life to illustrations with kinetic textures.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 351-360. ACM, 2014.
- Igarashi, Takeo, Tomer Moscovich, and John F. Hughes. “Spatial keyframing for performance-driven animation.” In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, pp. 107-115. ACM, 2005.
- Igarashi, Takeo, Satoshi Matsuoka, Sachiko Kawachiya, and Hidehiko Tanaka. “Interactive beautification: a technique for rapid geometric design.” In Proceedings of the 10th annual ACM symposium on User interface software and technology, pp. 105-114. ACM, 1997.
- Cheema, Salman, Sumit Gulwani, and Joseph LaViola. “QuickDraw: improving drawing experience for geometric diagrams.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1037-1064. ACM, 2012.
- Fišer, Jakub, Paul Asente, and Daniel Sýkora. “ShipShape: a drawing beautification assistant.” In Proceedings of the workshop on Sketch-Based Interfaces and Modeling, pp. 49-57. Eurographics Association, 2015.
- Saragih, Jason M., Simon Lucey, and Jeffrey F. Cohn. “Deformable model fitting by regularized landmark mean-shift.” International Journal of Computer Vision 91, no. 2 (2011): 200-215.