In the high-stakes world of visual effects, rotoscoping has long been the industry’s "necessary evil"—a labor-intensive, time-consuming process that acts as the backbone of compositing. Whether it is isolating an actor for a digital background replacement or creating precise mattes for color grading, the act of drawing and animating splines is a task that every compositor knows all too well. Today, Foundry, the software powerhouse behind Nuke, has officially launched SmartRoto, an AI-powered plugin designed to fundamentally change the economics of rotoscoping by shifting the burden from the artist’s wrist to the machine’s intelligence.
Available for Nuke, NukeX, Nuke Studio, and Nuke Indie, SmartRoto arrives not as a replacement for human judgment, but as a high-velocity assistant. By integrating seamlessly into the existing Nuke ecosystem, the tool promises to eliminate the repetitive grind of keyframing, allowing artists to focus on the nuance of the craft rather than the mechanics of frame-by-frame adjustment.
The Core Philosophy: Augmentation over Automation
The most significant hurdle in AI-driven VFX tools is the "magic bullet" fallacy. Many automated matting solutions attempt to generate perfect masks from scratch, often producing results that are either jittery or impossible to refine. Foundry’s approach with SmartRoto is distinct: it respects the artist’s workflow.
"This isn’t a magic bullet; it doesn’t do the full spline creation for you," explains Adam Cherbetji, Foundry’s Director of Product for AI Research. "It’s about accelerating the keyframing process. There is so much artist intuition that goes into the breakdown of the subject into smaller shapes. While that is somewhat time-consuming, the really time-consuming part is the keyframing. That’s where we focused."
In practice, the artist remains the architect. They define the shape, segment the subject into logical components, and set the "anchor" frames. SmartRoto then acts as the engine, animating and deforming those splines across the timeline. It is a symbiotic relationship: the artist provides the "why" and the "where," while the AI handles the "how."
A Two-Tiered Keyframe Architecture
At the heart of SmartRoto is a sophisticated, dual-keyframe system that distinguishes between "User Keys" and "Smart Keys." This hierarchy is essential for maintaining control and ensuring the final result meets the high standards of production-grade compositing.
- User Keys: These are the artist’s definitive inputs. When an artist adjusts a spline on a specific frame, the system treats this data as "gospel." These keys are immutable; they will never be overwritten by the AI, ensuring that the artist maintains absolute authority over the final matte.
- Smart Keys: These are the predictive elements generated by the model. By analyzing the motion and image features between two User Keys, the model fills in the gaps.
This workflow allows for a granular, non-destructive process. An artist can place a User Key at the start and end of a movement, and the system will interpolate the shapes in between. If the resulting Smart Keys are slightly off-target, the artist simply corrects the frame, which then promotes that frame to a new User Key. The model then re-calculates the remaining gaps based on this updated, higher-fidelity information. This iterative feedback loop ensures that the artist is constantly guiding the AI, never fighting against it.
The Chronology of Innovation
SmartRoto is not an overnight breakthrough; it is the culmination of years of rigorous research. Foundry first dipped its toes into AI-assisted rotoscoping six or seven years ago, collaborating with DNEG and the University of Bath. While those early experiments provided invaluable insights, the technology wasn’t quite ready for the demanding, high-precision requirements of professional compositing.
Four years ago, the company decided to restart the project from the ground up, building a custom model and training datasets within its internal research team. A critical aspect of this development was the ethical sourcing of training data. Foundry utilized footage from the director of research’s own film productions, which was subsequently annotated by professional roto artists.
By training the model on licensed, proprietary data, Foundry has ensured that SmartRoto is fully "commercially safe"—a growing concern in an era where AI training ethics are under heavy scrutiny. Furthermore, because the model runs entirely locally on the user’s hardware, there is no reliance on cloud processing, ensuring that sensitive studio footage never leaves the local network.
Supporting Data and Technical Requirements
The technical implementation of SmartRoto avoids the common pitfalls of optical flow or simple segmentation models. As Cherbetji notes, optical flow often fails at the edges—the very place where rotoscoping demands the most precision. Similarly, relying on a standard segmentation mask to draw a spline often results in a loss of the subpixel accuracy required for high-end film work.
Instead, the model focuses on "feature propagation." It latches onto image features to track the movement of the spline itself. This allows for the retention of subpixel accuracy, which is the "currency" of rotoscoping.
System Requirements:
The tool is surprisingly efficient compared to the massive "foundation" models seen in other sectors of AI.
- VRAM: The model typically requires approximately 8GB of VRAM.
- Scalability: Requirements may increase depending on the resolution of the footage and the complexity of the shapes being tracked simultaneously, but it remains accessible for standard workstation hardware.
Performance Gains:
Foundry conducted two-week, real-world trials with professional roto artists, comparing their speed on standard Nuke workflows against the SmartRoto-enhanced workflow. The results were compelling: artists were consistently twice as fast, with some achieving up to four times the speed on specific shots. Perhaps more importantly, the qualitative feedback was overwhelmingly positive, with many beta testers reporting that they found it difficult to return to manual rotoscoping after experiencing the tool.
Implications for the VFX Industry
The launch of SmartRoto signals a broader shift in the VFX industry. For decades, the industry has been searching for an automated solution that satisfies three distinct requirements:
- Precision at the silhouette: The ability to handle complex, fine-edged details.
- Temporal coherence: The elimination of "chatter" or jitter across a sequence.
- Editability: The output must remain in a format that a human compositor can tweak and refine.
Many previous attempts have achieved one or two of these, but SmartRoto’s focus on preserving the "spline as the deliverable" makes it uniquely suited to professional pipelines.
The industry is clearly at an inflection point. Other researchers, such as the team behind RotoShop (presented at SIGGRAPH 2026), are working on similar problems, using differentiable vector rasterizers to convert segmentation masks into editable Bèzier splines. While the technical approaches differ—SmartRoto propagates existing splines, while others work backward from raster masks—the goal is identical: moving away from pixel-based matting toward high-fidelity, vector-based control.
A New Standard for Compositors
Benjamin Bratt, a veteran compositor and author of Rotoscoping: Techniques and Tools for the Aspiring Artist, echoed the industry’s enthusiasm during the beta testing phase. "I’m shocked at how good SmartRoto is," Bratt stated. "It uses principles of rotoscoping and incorporates an intuitive, easily adaptable keyframe approach that meshes with manual practices. It’s easy to fix auto-generated keys without needing to start from scratch."
As Foundry continues to develop the tool—specifically looking at "thinning" algorithms to manage the density of keyframes—SmartRoto is poised to become an essential component of the Nuke toolkit. By leaning into the expertise of the artist rather than trying to replace it, Foundry has provided a solution that acknowledges the reality of production: in the world of VFX, the human eye is, and will remain, the ultimate arbiter of quality.
With an annual subscription model priced at US$499 per year, SmartRoto is positioning itself as a high-value investment for studios and independent artists alike, promising to turn one of the most dreaded tasks in the industry into a fluid, creative process.
