The Future of Rotoscoping: How Foundry’s SmartRoto is Transforming VFX Workflows
For decades, rotoscoping has been the "necessary evil" of the visual effects industry. It is a task that requires immense patience, precision, and a meditative commitment to detail, yet it is often the most time-consuming bottleneck in the production pipeline. Today, Foundry has officially disrupted this status quo with the release of SmartRoto, a sophisticated AI-powered plugin designed specifically for the Nuke ecosystem. By focusing on the acceleration of the keyframing process rather than attempting to automate the creative intuition of the artist, Foundry may have finally solved the "roto riddle" that has plagued the industry for years.
The Core Concept: Augmentation Over Automation
The most important distinction to understand about SmartRoto is what it is not. It is not a "magic button" that spits out a perfect, finished matte at the click of a mouse. It is not an automated segmentation tool that attempts to draw splines for you. As Adam Cherbetji, Foundry’s Director of Product for AI Research, puts it: "This isn’t a magic bullet. It’s about accelerating the keyframing process. There is so much artist intuition that goes into the breakdown of the subject into smaller, articulated shapes. We didn’t want to take that away—we wanted to remove the tedium of moving, deforming, and animating those splines frame-by-frame."
SmartRoto operates on the principle that the human artist is the final authority. An artist breaks down a character or object into sensible shapes on a clear frame, and the AI takes over the mechanical drudgery of propagating those shapes through the sequence. It is a collaborative relationship: the human provides the intent, and the machine provides the motion.
A Chronology of Innovation: Four Years in the Making
The arrival of SmartRoto is the culmination of years of internal research and development. Foundry’s journey into AI-assisted rotoscoping began in earnest over six years ago, with initial explorations alongside academic partners like the University of Bath and DNEG. While those early experiments provided foundational knowledge, they didn’t yield a production-ready solution.
Four years ago, the company decided to reset the approach, bringing the research fully in-house. The team spent years building a custom model and training it on high-quality, proprietary datasets. Notably, the training footage was sourced from the research team’s own filmmaking projects and annotated with top-tier rotoscoping work, ensuring that the model learned from "gold-standard" production practices.
Crucially, in an era of concerns over generative AI, Foundry has emphasized that SmartRoto is fully commercially safe. The model is trained exclusively on licensed data, and all processing happens locally on the artist’s workstation. No frames are sent to the cloud, ensuring that sensitive studio data remains secure within the facility’s firewall.
The Workflow: User Keys and the Hierarchy of Truth
At the heart of SmartRoto is a two-tier keyframe system that balances machine-generated efficiency with human-mandated precision.
The Two-Tier System
- User Keys: These are the artist’s inputs. When a user adjusts a spline, the system treats that frame as "gospel." It is a locked point of truth that the model will never overwrite.
- Smart Keys: These are the frames predicted by the AI. They fill in the gaps between User Keys.
The workflow is designed to be iterative. An artist sets their shapes on a key frame, hits a button, and the model snaps the splines into position across the sequence. If the prediction is slightly off, the artist makes a quick adjustment; that adjustment is instantly promoted to a "User Key," and the model automatically re-calculates the surrounding frames based on this new, refined information.
This creates a feedback loop where the model gets smarter as the artist works. The artist is never "fighting" the software; they are simply providing more direction. Because the system is shape-specific—meaning you can lock certain shapes while letting the AI handle others—the artist maintains total control over the composition.
Technical Superiority: Beyond Optical Flow
When asked why previous attempts at automated roto often failed, Cherbetji was candid about the limitations of existing technologies. "Optical flow was the first thing we tried," he noted, "but it breaks down at the edge. And the edge is exactly where you need the most precision."
SmartRoto also avoids the common pitfall of relying on image-based segmentation masks. While many current market solutions use AI to create a pixel-based matte and then attempt to trace a spline around it, this often leads to "jittery" results and a loss of the sub-pixel accuracy that professional compositors require.
Instead, SmartRoto uses a proprietary machine learning model that latches onto image features to propagate the splines themselves. By keeping the workflow in the vector-based world of B-splines, the tool remains compatible with standard VFX pipelines, allowing for natural motion blur, easy retiming, and precise edge control. The model also understands the spatial relationship between shapes, meaning that as one shape moves, the others maintain their logical articulation.
Empirical Results and Industry Feedback
To validate the tool before its public release, Foundry invited a cohort of professional rotoscopers to their offices for a rigorous two-week testing period. The results were striking. By the end of the second week, artists were, on average, four times faster than those using traditional methods. While Foundry is careful to temper expectations—suggesting that a 2x speed increase is a more standard production-level estimate—the qualitative feedback was even more telling.
Benjamin Bratt, a veteran compositor and author of Rotoscoping: Techniques and Tools for the Aspiring Artist, was among the beta testers. "I’m shocked at how good SmartRoto is," Bratt remarked. "It meshes perfectly with manual roto practices. It’s easy to fix auto-generated keys without needing to start from scratch."
Implications for the Industry
The release of SmartRoto comes at a time when the entire visual effects industry is grappling with how to integrate AI into established workflows. The consensus among professionals has been that AI must be "production-grade"—it cannot produce results that require a complete overhaul in the final compositing stages.
Foundry’s approach aligns with a broader trend in the industry, represented by projects like the RotoShop paper presented at SIGGRAPH 2026. Like SmartRoto, these initiatives prioritize the creation of editable vector data rather than static raster mattes. Whether through fitting splines to masks or propagating splines directly from image features, the goal is the same: to turn a manual, labor-intensive chore into an intelligent, artist-driven process.
The implications for studios are significant. Rotoscoping is often one of the largest line items in a film or series budget. By cutting the time required for this task in half—or even by a factor of four—studios can reallocate resources to more creative aspects of the pipeline. Furthermore, for the individual artist, SmartRoto removes the most monotonous parts of the job, allowing them to focus on the nuance of the edge, the complexity of the motion, and the final artistic integration.
Conclusion: A New Standard for Roto
Foundry has successfully navigated the thin line between "over-automation," which frustrates artists, and "manual labor," which drains productivity. By building a tool that respects the traditional language of rotoscoping—using splines, keyframes, and user-defined anchor points—they have created a solution that feels like a natural extension of the compositor’s existing skillset.
Available now for $499 per year, SmartRoto is poised to become an essential component of the Nuke toolkit. It serves as a blueprint for how AI can be implemented in high-end creative software: not as a replacement for human judgment, but as a high-powered assistant that handles the math, leaving the artistry exactly where it belongs—in the hands of the artist.
