A 2023 paper from Meta AI Research describes a system that turns a child's drawing of a person into an animated character in under 10 seconds, using fine-tuned computer vision models to find the figure, trace its outline, and locate its joints. The public demo launched on December 16, 2021, and drew more than 3.2 million unique visitors in its first nine months. The work is a documented, narrow case of AI in animation: it handles drawn human figures, and its authors report where it fails.
Key takeaways
- The system chains four steps: detect the figure, cut out its mask, predict joints, then move the rig with recorded human motion.
- Models pretrained on photographs scored a mean average precision of 0.06 for finding drawn figures; fine-tuning on 2,000 drawings raised it to 0.81.
- Of 3.5 million uploaded images before April 20, 2022, users consented to data use for 1.7 million, and 178,166 became a research dataset.
- Viewers preferred the "twisted perspective" motion in 16 of 20 test videos; in the other four, no significant preference appeared.
- The method covers human figures only, and its repository was archived on September 3, 2025.
The four steps from paper to motion
The paper, "A Method for Animating Children's Drawings of the Human Figure" by Harrison Jesse Smith, Qingyuan Zheng, Yifei Li, Somya Jain, and Jessica K. Hodgins, appeared in ACM Transactions on Graphics in 2023. Its authors describe a pipeline rather than a single model. The table lists each stage, the technique the paper describes, and a failure the authors document.
| Stage | Technique | Documented failure |
|---|---|---|
| Find the figure | Object detector fine-tuned on drawings | Background objects detected as figures, hollow parts excluded |
| Cut out the figure | Image processing, not a neural network | Limbs touching at points other than joints, or an outline that is not fully connected |
| Locate joints | Pose estimator fine-tuned on drawings | Limbs confused with background or held objects; figures not facing forward |
| Move the character | Triangle mesh, skeleton rig, as-rigid-as-possible deformation, recorded human motion | Drawn proportions that differ sharply from a human body |
After segmentation, the program builds a 2D mesh from the outline and textures it with the original drawing. The skeleton uses the shoulders, elbows, wrists, hips, knees, ankles, and nose, with hips averaged into a root joint and shoulders into a chest joint. Motion clips from human performers then drive the rig.
Why models trained on photographs missed
The detectors and pose estimators were built for photographs, and children's drawings differ in line, color, and proportion. The authors tested how many annotated drawings fine-tuning needs. Pretrained weights, treating a drawn figure as an instance of the "person" class, scored 0.06 mean average precision for bounding boxes, 0.04 for segmentation, and 0.09 for pose. With 100 training drawings the scores were 0.51, 0.51, and 0.76. With 2,000 drawings they reached 0.81, 0.71, and 0.85.
Training on all 177,666 user-accepted annotations did not improve every stage. Those labels were noisy, and segmentation fell to 0.49 while pose rose to 0.90. The authors also found that mask predictions from the detector were often unsuitable for animation, so the final system uses an image-processing method for the mask instead. By their visual inspection, 42.4 percent of masks produced that way were usable without correction on the test images, which is why the demo lets users edit the box, the mask, and the joints before animating.
What the public demo recorded
The demo went online on December 16, 2021. Over the next nine months, the paper reports more than 3.2 million unique visitors, 6.7 million uploaded images, and about four animations generated per image. Promotion after launch came from users sharing the site. The authors note that parents, elementary school teachers, technology enthusiasts, and artists were among the visible groups.
Data use was opt-in. By April 20, 2022, users had uploaded over 3.5 million images and agreed to the data usage agreement for 1.7 million of them. Refusing did not limit the experience. The authors filtered out photographs of people, pets, anime characters, and logos, leaving the Amateur Drawings Dataset of 178,166 drawings with bounding boxes, masks, and joint locations. The project's code repository lists the code, model weights, and that dataset under the MIT License.
Twisted perspective: a design choice that was tested
Children often draw a body with the torso facing the viewer and the legs in profile. Retargeting recorded motion onto one flat plane can look wrong for such figures. The authors project the torso and upper limbs onto a frontal plane and the lower limbs onto a side plane, and then rotate the character's joints to match.
They tested the choice with 66 subjects on a crowdsourcing platform. Each saw side-by-side videos, one with twisted perspective and one with a single plane, and picked the more appealing motion. Across four figures and five motions (20 videos), 16 showed a significant preference for twisted perspective and four showed none. The result concerns appeal on those characters and motions, not accuracy of movement.
What the research does not show
The paper addresses human figures. It does not cover animals, vehicles, or full-length productions, and its motion comes from recorded human performers, not from a model that invents movement. It also does not describe how professional studios work. A separate page on this site covers animation union rules. The repository's owner archived it on September 3, 2025, so the code is read-only and no longer maintained.
FAQ
Does this system invent the motion?
No. Models locate the figure and its joints, but the movement comes from a library of motion clips recorded from human performers and retargeted to the character's skeleton.
Was the training data collected with permission?
According to the paper, sharing was optional and refusing did not restrict the experience. Of more than 3.5 million uploads before April 20, 2022, users consented for 1.7 million.
Can the method animate any kind of drawing?
No. It was designed and evaluated for drawn human figures, and the authors list failures such as limbs touching away from joints and figures not facing forward.
Is the code still maintained?
No. The repository was archived on September 3, 2025. The code, weights, and dataset remain available under the MIT License, according to its page.
Related reading
- AI and the future of content creation in animation
- Creative AI examples and who gets credit
- How AI learning datasets work
- AI in research and discovery
More in AI Tools and Industry.
Last verified: October 2026
Sources
- A Method for Animating Children's Drawings of the Human Figure, Smith, Zheng, Li, Jain, and Hodgins, ACM Transactions on Graphics, 2023 (arXiv version, April 2023)
- AnimatedDrawings repository, Meta Research, archived September 3, 2025