Rotoscoping—the painstaking process of tracing over film footage frame-by-frame to create mattes for visual effects—has long been considered the “grunt work” of the VFX industry. For decades, it has remained a necessary evil, demanding immense patience and surgical precision. Today, Foundry, the developer behind the industry-standard software Nuke, has officially launched SmartRoto, an AI-powered plugin designed not to replace the artist, but to fundamentally accelerate the keyframing process.
While the market is saturated with automated matting tools, SmartRoto distinguishes itself by focusing on the one thing professional compositors care about most: editable, subpixel-accurate splines.
The Philosophy of SmartRoto: Intelligence Over Automation
The first hurdle in understanding SmartRoto is dispelling the myth of the "magic button." According to Adam Cherbetji, Foundry’s Director of Product for AI Research, SmartRoto is explicitly not a replacement for artist intuition.
"This isn’t a magic bullet; it doesn’t do the full spline creation for you," Cherbetji explains. "It’s about accelerating the keyframing process. There is so much artist intuition that goes into the breakdown of a subject into smaller, logical shapes. While that preparation is time-consuming, the truly grueling part is the keyframing—that’s where we focused."
In traditional rotoscoping, an artist breaks a subject into articulated shapes on a "hero" frame. SmartRoto acts as an intelligent assistant, handling the tedious movement, deformation, and temporal tracking of those splines across an image sequence. Crucially, the system respects the artist’s authority. By distinguishing between "Smart Keys" (AI-predicted) and "User Keys" (artist-placed), the software ensures that the artist remains in total control. Once a user places a keyframe, the system treats it as "gospel," locking it against any automated overwriting.
Chronology: A Four-Year Journey to Production-Ready AI
The path to SmartRoto was far from linear. Foundry’s exploration into AI-assisted rotoscoping began over six years ago in collaboration with academic partners like the University of Bath and DNEG. While those early experiments provided a foundation, they failed to produce a tool that met the stringent requirements of professional VFX pipelines.
Four years ago, Foundry restarted the project from the ground up, internalizing the research and development. The team took a rigorous approach to training data, building a proprietary dataset derived from original, high-quality filmmaking footage. These clips were manually rotoscoped by professionals, ensuring the AI learned from the "gold standard" of human work rather than noisy, low-quality automated masks.
This development cycle was deeply iterative. The team spent years refining how the model interprets spatial relationships. By understanding that a human limb is connected to a torso—rather than treating every spline as an isolated, floating vector—the model achieved a level of temporal coherence that had previously eluded the team.
The Technical Edge: Why Optical Flow Fell Short
To understand why SmartRoto is a breakthrough, one must look at what it intentionally avoids. Early iterations relied on optical flow—a common technique for tracking motion—but it proved inadequate for the razor-sharp edges required in high-end VFX. "Optical flow breaks down at the edge," says Cherbetji. "That’s exactly where you need the precision."
Similarly, the team rejected simple segmentation models. While many AI tools currently on the market can generate a matte by identifying a person or object, these mattes are often "baked" into pixels, making them difficult to adjust or refine. SmartRoto, by contrast, latches onto image features and propagates the splines themselves.
Because the system preserves the spline as the primary "currency" of the workflow, it remains subpixel-accurate. It doesn’t force the artist to work with a rigid mask; it provides a vector-based foundation that the artist can continue to tweak, refine, or manually override at any point in the timeline.
Supporting Data and Performance Metrics
During the final phase of testing, Foundry conducted a blind study. Professional roto artists were brought into the office to perform identical tasks using both standard Nuke tools and the new SmartRoto plugin, with the order of tasks randomized to ensure impartiality.
The results were statistically significant. By the end of the two-week testing period, artists were performing tasks up to four times faster than they had previously. While Foundry is careful to manage expectations—noting that "two times faster" is a more consistent, conservative estimate for daily production—the qualitative feedback was even more telling. Many beta testers, including renowned compositor Benjamin Bratt, noted that the tool fundamentally changed their workflow.
"I’m shocked at how good SmartRoto is," says Bratt, author of Rotoscoping: Techniques and Tools for the Aspiring Artist. "It incorporates an intuitive, easily adaptable keyframe approach that meshes with manual practices. It’s easy to fix auto-generated keys without needing to start from scratch."
From a hardware perspective, the system is surprisingly efficient. It requires roughly 8GB of VRAM, making it accessible to most modern workstations without requiring the massive, expensive server-side infrastructure associated with large-scale "foundation" AI models. Furthermore, the tool runs entirely locally, meaning no sensitive production data is ever sent to the cloud, maintaining the strict security standards of major film studios.
The Landscape of AI Roto: A Crowded Field
Foundry is not alone in its quest to solve the rotoscoping problem. The industry is currently witnessing a "gold rush" of AI-assisted tools. Sam Hodge’s work at Kognat has already provided the industry with high-quality, AI-driven roto solutions, and the SIGGRAPH community is actively engaged in the challenge.
Notably, the RotoShop paper presented at SIGGRAPH 2026 by Sirak Ghebremusse (OTOY) proposes a different route. RotoShop utilizes a segmentation model to generate a mask and then fits a rig of canonical Bézier splines to that mask. It is a "bottom-up" approach compared to SmartRoto’s "top-down" propagation.
While these approaches differ, they share a singular, industry-wide realization: the spline is the future. AI tools that produce only rasterized mattes are increasingly viewed as dead-ends, as they do not offer the flexibility required for final-pixel compositing. Whether through fitting vectors to masks or propagating splines through motion, the goal is clear: removing the tedium without removing the artist’s ability to finesse the final frame.
Implications for the Future of VFX
The introduction of SmartRoto marks a turning point in the relationship between AI and the VFX artist. For years, the fear has been that AI would "automate away" the artist’s role. SmartRoto offers a different vision: the "augmented artist."
By automating the interpolation of splines across thousands of frames, the artist is liberated from the most repetitive, wrist-straining aspects of the job. This shift allows the compositor to spend their time on the nuanced, creative aspects of the craft—such as managing motion blur, dealing with complex partial occlusions, and ensuring that the rotoscoping integrates perfectly with the lighting and depth of the original shot.
Furthermore, the "user key" system ensures that the software remains a tool, not a black box. Because the AI is designed to learn from the artist’s corrections, it becomes a partner in the workflow. If the model misinterprets a movement, the artist simply corrects the shape, and that correction is immediately factored into the surrounding frames. This eliminates the "all-or-nothing" frustration common in earlier AI tools, where a single bad prediction would force a total reset of the work.
As the industry continues to push for higher-resolution deliverables and faster turnaround times, tools like SmartRoto will likely become a standard component of the VFX toolkit. By respecting the traditions of the trade while embracing the power of machine learning, Foundry has provided a blueprint for how AI can coexist with, and ultimately elevate, the professional artist.
For studios and individual artists alike, the subscription model—priced at $499 per year—suggests that Foundry is positioning this as an essential, high-utility asset rather than a niche luxury. In a world where time is the most expensive resource on a VFX production, the promise of halving the time spent on rotoscoping is not just a technological advancement; it is a financial and creative imperative.
