Rotoscoping / Video Segmentation & Matting¶
Video
Open Source¶
-
SAM2Matting Discovered by AI
2026
A unified tracker-to-matting framework that bridges high-level tracking and low-level matting, utilizing Segment Anything 2 (SAM 2) to achieve generalized, high-quality image and video matting.
mattingrotoscopingsam2segmentation -
VideoMaMa Discovered by AI
2026
VideoMaMa (Video Mask-to-Matte Model) is a mask-guided video matting framework that converts coarse segmentation masks into pixel-accurate alpha mattes by leveraging pretrained video diffusion models as generative priors. It demonstrates strong zero-shot generalization to real-world footage and complex backgrounds.
diffusion-priorrotoscopingsegmentationvideo-matting -
EdgeTAM: On-Device Track Anything Model Discovered by AI
2025
EdgeTAM is an on-device executable variant of SAM 2 designed for promptable video segmentation and tracking. It introduces a 2D Spatial Perceiver to compress frame-level memories, running 22x faster than SAM 2 and achieving 16 FPS on mobile devices like the iPhone 15 Pro Max without quantization.
mobilesam2segmentationtracking -
Generative Video Matting (GVM) Discovered by AI
2025
A generative video matting framework that redefines video matting as a conditional video generation task, leveraging large-scale video diffusion priors to produce highly realistic alpha mattes and foregrounds.
diffusiongenerativemattingrotoscoping -
MatAnyone Discovered by AI
2025
MatAnyone is a target-assigned video matting framework that introduces a consistent memory propagation module with region-adaptive memory fusion to integrate memory from previous frames. This ensures semantic stability in core regions while preserving fine details in complex or ambiguous backgrounds.
background-removalmemory-propagationrotoscopingvideo-matting -
MatAnyone 2 Discovered by AI
2025
An advanced human video matting framework that leverages a learned Matting Quality Evaluator (MQE) to assess semantic and boundary quality, enabling high-quality alpha matte generation without green screens.
alpha-mattebackground-removalmattingvideo-matting -
Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos Discovered by AI
2025
Sa2VA is a unified model that couples SAM 2 with a Multimodal Large Language Model (MLLM) to support dense grounded understanding of both images and videos. It enables high-quality video referring segmentation, allowing users to segment and track objects in videos using natural language prompts.
mllmsam2segmentationtracking -
SAM 3 Discovered by AI
2025
A unified foundation model for promptable concept segmentation and tracking in both images and videos, allowing users to segment and track all instances of open-vocabulary concepts using text or visual prompts.
foundation-modelsam3segmentationtracking -
Segment Any Motion in Videos Discovered by AI
2025
SegAnyMo is a zero-shot moving object segmentation framework that combines long-range trajectory motion cues with semantic features from DINO and SAM 2. It identifies and segments moving objects in video sequences without requiring manual annotations or training on video segmentation datasets.
motionsam2segmentationzero-shot -
A Distractor-Aware Memory for Visual Object Tracking with SAM2 Discovered by AI
2024
DAM4SAM introduces a Distractor-Aware Memory (DAM) mechanism for SAM 2 to address the challenge of tracking objects in the presence of visually similar distractors. It maintains separate target and distractor memories to prevent tracking drift, significantly improving tracking robustness in complex video scenes.
memorysam2segmentationtracking -
BiRefNet Discovered by AI
2024
A bilateral reference framework for high-resolution dichotomous image and video segmentation. It utilizes a localization module for global semantic positioning and a reconstruction module with bilateral reference to produce highly detailed and clean background removal masks.
background-removalhigh-resolutionrotoscopingsegmentation -
Grounded SAM 2: Ground and Track Anything in Videos Discovered by AI
2024
An open-source pipeline that integrates Grounding DINO, Florence-2, and SAM 2 to detect, segment, and track any object in videos based on text prompts, enabling automated rotoscoping.
grounding-dinorotoscopingsam2video-segmentation -
LiVOS Discovered by AI
2024
A lightweight video object segmentation (VOS) framework that replaces traditional quadratic softmax matching with a gated linear matching mechanism, significantly reducing memory and computational overhead while maintaining high segmentation quality.
real-timevideo-segmentationvos -
SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree Discovered by AI
2024
An enhancement for SAM 2 video segmentation designed for long video sequences, utilizing a training-free memory tree to address error accumulation. It maintains multiple segmentation pathways and selects optimal paths to prevent cascading errors over long durations.
long-videorotoscopingsam2video-segmentation -
SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory Discovered by AI
2024
SAMURAI adapts SAM 2 for zero-shot visual object tracking by incorporating motion-aware memory. It integrates motion cues and a memory selection mechanism to handle challenging scenarios like fast-moving objects, self-occlusion, and crowded scenes without requiring any retraining or fine-tuning.
motionsam2trackingzero-shot -
Segment Anything Model 2 (SAM 2)
2024
A promptable segmentation model that tracks objects across video frames, the backbone of modern AI-assisted rotoscoping and matting tools.
mattingsegmentationtracking -
Cutie Discovered by AI
2023
A video object segmentation network featuring top-down object-level memory reading, which significantly reduces matching noise and improves tracking accuracy in the presence of distractors.
object-trackingsegmentationvideo-editingvos -
DEVA Discovered by AI
2023
A decoupled video segmentation approach (DEVA) that combines task-specific image-level segmentation with class-agnostic bi-directional temporal propagation. It enables open-world video tracking and segmentation of any target object without requiring video-specific training.
open-worldrotoscopingsegmentationtracking -
HQTrack Discovered by AI
2023
HQTrack (Tracking Anything in High Quality) is a framework for high-performance video object tracking and segmentation. It combines a Video Multi-Object Segmenter (VMOS) with a Mask Refiner (MR) to track multiple target objects simultaneously and output highly accurate object masks.
rotoscopingsamsegmentationvideo-tracking -
Segment and Track Anything (SAM-Track) Discovered by AI
2023
An interactive video object tracking and segmentation framework that combines the Segment Anything Model (SAM) with advanced temporal propagation algorithms to track and segment any object in a video.
interactivesamsegmentationtracking -
Segment Anything in High Quality Discovered by AI
2023
An extension of the Segment Anything Model (SAM and SAM 2) designed to preserve fine-grained details and intricate boundaries in both images and videos by introducing a high-quality output token and a global-local feature fusion block.
high-qualitysamsegmentation -
Track-Anything Discovered by AI
2023
An interactive and flexible tool for video object tracking and segmentation, combining Segment Anything (SAM), XMem, and E2FGVI to allow users to easily track and segment any object via simple clicks.
interactivesamsegmentationtracking -
ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformers Discovered by AI
2023
ViTMatte is an image and video matting system based on pretrained plain Vision Transformers (ViTs). It utilizes a hybrid attention mechanism and a convolutional neck to achieve high-quality alpha matte estimation with fine-grained boundary details.
alpha-mattemattingvision-transformervitmatte -
XMem++ Discovered by AI
2023
XMem++ (XMem2) is a semi-supervised video object segmentation model that introduces a permanent memory module to effectively handle multiple user-selected frames with varying appearances of the same object. It provides production-level video segmentation from a few annotated frames.
memory-networkrotoscopingsemi-supervisedvideo-segmentation -
XMem Discovered by AI
2022
A video object segmentation architecture designed for long videos, utilizing a multi-store feature memory system inspired by human cognitive memory models to maintain high accuracy and low memory usage.
long-term-trackingmemory-modelsegmentationvos -
Robust Video Matting Discovered by AI
2021
A real-time, high-resolution human video matting method that uses a recurrent neural network to process videos with temporal memory, achieving high temporal coherence without requiring extra inputs like trimaps.
mattingreal-timervmvideo-matting
Proprietary / Industry Tools¶
-
Beeble AI Discovered by AI
2026
Beeble AI is an AI-powered VFX platform that offers advanced background removal and rotoscoping tools to generate clean alpha mattes. It integrates with professional DCC tools like Nuke, After Effects, and DaVinci Resolve, supporting high-resolution production pipelines.
background-removalrotoscopingvfx -
Boris FX Silhouette Discovered by AI
2026
An Academy Award and Emmy Award-winning advanced rotoscoping, paint, and tracking application widely used in Hollywood visual effects pipelines. It features AI-driven matte generation, 3D head tracking, and integrated node-based compositing tools.
compositingrotoscopingtrackingvfx -
Silhouette Discovered by AI
2026
An Academy and Emmy Award-winning visual effects software widely recognized as the industry standard for professional rotoscoping, paint, and compositing. It features advanced node-based workflows, integrated Mocha tracking, and AI-assisted masking tools to streamline complex VFX pipelines.
compositingpaintrotoscopingvfx -
Smart Roto Discovered by AI
2026
Smart Roto is an online, browser-based automatic rotoscoping tool that propagates user-drawn splines across video frames while keeping every spline fully editable. It is designed for professional VFX workflows, supporting up to 8K plates and EXR export.
online-toolrotoscopingvfx -
Autodesk Flow Studio Discovered by AI
2025
A cloud-based, AI-powered 3D toolset that automatically animates, lights, and composes CG characters into live-action scenes, featuring automated actor masking, rotoscoping, and clean plate generation.
compositingrotoscopingvfx -
Boris FX Mocha Pro Discovered by AI
2025
Mocha Pro is an industry-standard planar tracking and rotoscoping software that features advanced AI-powered masking tools, including Matte Refine ML, Object Brush, and Matte Assist. It is widely used in professional VFX and post-production studios to generate clean mattes and track organic surfaces.
matte-refineplanar-trackingrotoscopingvfx -
Mocha Pro Discovered by AI
2025
An industry-standard planar tracking and masking software by Boris FX, featuring advanced AI-powered rotoscoping, face detection, and automated matte refinement tools.
maskingrotoscopingtrackingvfx -
Adobe After Effects Roto Brush Discovered by AI
2024
An industry-standard AI-powered rotoscoping tool integrated into Adobe After Effects that automates the separation of foreground subjects from backgrounds using advanced machine learning.
adobeafter-effectsmattingrotoscoping -
Beeble Discovered by AI
2024
An AI-powered video-to-VFX platform featuring studio-grade rotoscoping, background removal, and a node-based AI compositor for relighting and 3D compositing workflows.
compositingrelightingrotoscopingvfx -
Mask Prompter Discovered by AI
2024
An AI-powered rotoscoping plugin for After Effects and Premiere Pro that allows users to generate precise masks and mattes using text prompts, bounding boxes, or points.
after-effectsmaskingplugin -
Slapshot AI Discovered by AI
2024
An enterprise-grade AI-powered VFX toolkit that automates rotoscoping, camera tracking, depth map estimation, and generative retiming for high-resolution footage up to 8K.
camera-trackingdepth-maprotoscopingvfx -
Runway Green Screen Discovered by AI
2023
A professional web-based AI tool for instant video background removal and rotoscoping, allowing creators to isolate subjects with just a few clicks.
background-removalrotoscopingrunwayweb-tool
