Video Editing¶
Video
Open Source¶
-
EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing Discovered by AI
2026
An efficient video inpainting and editing framework that dynamically allocates computation proportional to the edit mask size. It uses a local video context module operating on masked tokens combined with a lightweight temporal global context embedder to achieve real-time, high-fidelity generative editing.
diffusioninpaintingreal-timevideo-editing -
EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing Discovered by AI
2026
A video object removal framework that jointly learns object removal and insertion. Built on the Wan2.1 video generation system, it effectively eliminates dynamic target objects along with their complex visual effects like shadows and reflections.
object-removalvideo-editingvideo-inpaintingwan2.1 -
FireRed-OpenStoryline Discovered by AI
2026
An open-source AI video editing agent that transforms manual editing into intention-driven directing. It leverages Large Language Models (LLMs) for autonomous planning, script generation, voiceover creation, and precise tool orchestration to automate video creation and editing workflows.
agentautomationllmvideo-editing -
JoyAI-Video-Edit Discovered by AI
2026
A 16B-parameter autoregressive diffusion framework designed for real-time, open-ended streaming video editing. It processes incoming video frames causally with low latency, supporting diverse editing tasks such as object addition/removal, background replacement, and style transfer at 30 FPS.
diffusionreal-timestreamingvideo-editingGitHub · arXiv · Hugging Face
-
LiveEdit Discovered by AI
2026
A diffusion-based streaming video editing framework designed for real-time interactive scenarios, optimizing background stability and low latency.
real-timestreamingvideo-editing -
LoomVideo Discovered by AI
2026
A unified video generation and editing model that integrates multimodal large language models to interpret interleaved text-image inputs and perform zero-overhead video editing.
multimodalvideo-editingvideo-generation -
Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editing Discovered by AI
2026
A memory-augmented framework for video-to-video diffusion models that treats prior edits as structured constraints for subsequent generations. It maintains an external memory of previous outputs, retrieves task-relevant edits, and integrates them to ensure cross-turn consistency in multi-turn novel view synthesis and long-video editing.
diffusionmemory-augmentedvideo-editingvideo-to-video -
Qwen-Video-Edit Discovered by AI
2026
An instruction-based video editing framework that repurposes a pre-trained image editing model (Qwen-Image-Edit) to edit videos by operating directly on video-VAE latents. It arranges latent frames as tiles of a large virtual image, bypassing the need for expensive video-pretrained transformers.
diffusionimage-to-videolatent-spacevideo-editing -
SANA-Streaming Discovered by AI
2026
A system-algorithm co-designed framework by NVIDIA for high-resolution, real-time streaming video-to-video editing on consumer GPUs using a hybrid diffusion transformer.
diffusion-transformerreal-timevideo-editing -
Seen-to-Scene Discovered by AI
2026
A video outpainting framework that unifies propagation-based and generation-based paradigms to expand video boundaries. It preserves existing visual content while generating temporally coherent and spatially consistent unseen regions.
diffusionoutpaintingtemporal-consistencyvideo-editing -
Vera: A Layered Diffusion Model for Content-Preserving Video Editing Discovered by AI
2026
A layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, it generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design.
alpha-mattelayered-diffusionnetflixvideo-editing -
DiffuEraser Discovered by AI
2025
A diffusion-based video inpainting model designed to achieve high content completeness and temporal consistency for large-mask object removal and video completion.
diffusion-modelobject-removalvideo-inpainting -
DynVFX: Augmenting Real Videos with Dynamic Content Discovered by AI
2025
A method for augmenting real-world videos with newly generated dynamic content based on simple text instructions. It synthesizes dynamic objects or complex scene effects that naturally interact with the existing scene over time, ensuring temporal and spatial consistency.
diffusiondynamic-contentvfxvideo-editing -
EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning Discovered by AI
2025
A unified framework that enables both image and video editing and generation within a single model. By representing text, images, and videos as a unified token sequence, it leverages full self-attention to achieve robust in-context learning and effective knowledge transfer between modalities.
diffusionimage-editingin-context-learningvideo-editing -
EgoEdit Discovered by AI
2025
A real-time, instruction-guided autoregressive model and benchmark tailored for egocentric video editing. It addresses challenging hand occlusions, complex interactions, and large egomotion to perform precise object substitution and removal in first-person videos.
autoregressiveegocentricreal-timevideo-editing -
EraserDiT: Fast Video Inpainting with Diffusion Transformer Model Discovered by AI
2025
A fast video inpainting and object removal framework based on the Diffusion Transformer (DiT) architecture. It interactively removes specified objects, automatically generates corresponding prompts, and processes high-resolution videos with strong temporal consistency and content fidelity.
ditinpaintingobject-removalvideo-editing -
FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editing Discovered by AI
2025
A training-free, inversion-free video editing framework that operates directly in data space using Ordinary Differential Equations (ODEs). It achieves spatiotemporal coherent editing and uses attention-guided masking for precise localized control without requiring model fine-tuning.
inversion-freeodetraining-freevideo-editing -
InstructX: Towards Unified Visual Editing with MLLM Guidance Discovered by AI
2025
A unified framework for image and video editing that integrates Multimodal Large Language Models (MLLMs) with diffusion models. It leverages MLLM guidance to improve instruction-driven editing across diverse tasks, enhancing spatial and semantic accuracy.
diffusionmllmvideo-editing -
MTV-Inpaint Discovered by AI
2025
A unified multi-task video inpainting framework that supports both traditional scene completion and text/image-guided object insertion in long videos.
diffusionvideo-editingvideo-inpainting -
VACE Discovered by AI
2025
A unified video creation and editing model that supports reference-to-video generation, video-to-video editing, masked video-to-video editing, and free composition of these tasks.
diffusion-transformervideo-editingvideo-generation -
VEGGIE Discovered by AI
2025
A unified video generative framework that handles instructional video editing (adding, removing, or changing concepts) and video reasoning based on multimodal instructions.
diffusioninstructionalvideo-editing -
VideoPainter Discovered by AI
2025
A video inpainting and editing framework that introduces plug-and-play context control to handle arbitrary-length videos while balancing background preservation and foreground generation.
context-controlvideo-editingvideo-inpainting -
AnyV2V Discovered by AI
2024
AnyV2V is a tuning-free, plug-and-play framework for video-to-video editing. It simplifies video editing by converting it into an image editing task on the first frame, and then propagating the edits across the remaining frames using a modified video diffusion model.
diffusionvideo-editingvideo-to-video -
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions Discovered by AI
2024
A framework that automatically creates realistic and dynamic visual effects (VFX) videos from a single video and natural language instructions. It integrates neural scene modeling, LLM-based code generation, and physical simulation to provide physically-grounded, photorealistic editing effects.
llmphysics-simulationvfxvideo-editing -
COCOCO Discovered by AI
2024
A video inpainting framework that improves text-guided video inpainting consistency, controllability, and compatibility using Segment Anything 2 (SAM 2).
sam2video-editingvideo-inpainting -
CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility Discovered by AI
2024
A text-guided video inpainting framework designed to improve temporal consistency, controllability, and compatibility. It integrates Segment Anything 2 (SAM2) to enable seamless object removal, video completion, and personalized video inpainting.
diffusionsam2video-editingvideo-inpainting -
COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing Discovered by AI
2024
A zero-shot video editing framework that leverages the inherent feature correspondence of pre-trained text-to-image diffusion models. By establishing cross-frame correspondence, it enforces strong temporal constraints to produce highly consistent and high-quality edited videos.
diffusiontemporal-consistencyvideo-editingzero-shot -
Elevating Flow-Guided Video Inpainting with Reference Generation Discovered by AI
2024
A robust video inpainting framework that combines a large generative model for reference generation with an advanced pixel propagation algorithm. It avoids resampling artifacts while maintaining sub-pixel accuracy, enabling high-quality object removal and text-guided content synthesis in missing areas.
inpaintingobject-removalpixel-propagationvideo-editing -
FFF-VDI Discovered by AI
2024
A video inpainting model that leverages pre-trained image-to-video diffusion models to fill missing regions. By using a first-frame filling strategy combined with noise latent propagation and DDIM inversion, it achieves high temporal consistency and visual quality without relying on optical flow propagation.
diffusioninpaintingtemporal-consistencyvideo-editing -
FloED Discovered by AI
2024
An advanced video inpainting framework that incorporates optical flow guidance into diffusion models to enhance temporal coherence and computational efficiency.
optical-flowvideo-editingvideo-inpainting -
I2VEdit Discovered by AI
2024
A first-frame-guided video editing framework that propagates edits made on the first frame to the entire video using image-to-video diffusion models, ensuring temporal consistency and preserving original motion.
diffusioni2vvideo-editing -
MotionFollower Discovered by AI
2024
MotionFollower is a score-guided video editing framework designed to alter object motion trajectory and pose while preserving background details and protagonist identity across video frames.
diffusionmotion-editingvideo-editing -
Streaming Video Diffusion (SVDiff) Discovered by AI
2024
A streaming video diffusion framework designed for online video editing. It incorporates compact spatial-aware temporal recurrence into off-the-shelf Stable Diffusion models, enabling real-time, consistent editing of streaming video frames.
diffusionreal-timestreamingvideo-editing -
Towards Language-Driven Video Inpainting via Multimodal Large Language Models Discovered by AI
2024
Introduces language-driven video inpainting, a task that uses natural language instructions instead of manual binary masks to guide video object removal and replacement. It presents the Remove Objects from Videos by Instructions (ROVI) dataset and a multimodal framework leveraging large language models to generate precise inpainting results.
mllmobject-removalvideo-inpainting -
VideoDirector Discovered by AI
2024
A precise video editing framework that leverages pretrained text-to-video (T2V) diffusion models, utilizing spatiotemporal decoupled guidance and multi-frame null-text optimization to achieve high temporal consistency and edit fidelity.
diffusiont2vvideo-editing -
Videoshop Discovered by AI
2024
A training-free localized semantic video editing algorithm that propagates user edits from the first frame to subsequent frames using noise-extrapolated diffusion inversion.
diffusiontraining-freevideo-editing -
AVID: Any-Length Video Inpainting with Diffusion Model Discovered by AI
2023
AVID is a diffusion-based framework for any-length video inpainting and text-guided editing. It addresses temporal consistency and memory constraints by utilizing a segment-aware temporal attention mechanism and a sliding window inference strategy, allowing seamless object removal and localized video editing.
diffusioninpaintingvideo-editingvideo-inpainting -
ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond Discovered by AI
2023
A training-free framework for text-driven video editing that adapts pre-trained text-to-image diffusion models to generate temporally consistent videos. It incorporates structural conditions such as edge maps and fine-tunes key-frame and temporal attention to preserve the layout of the source video.
diffusiontraining-freevideo-editing -
Fairy Discovered by AI
2023
A fast, parallelized instruction-guided video-to-video synthesis framework that utilizes anchor-based cross-frame attention to propagate diffusion features. It ensures high temporal coherence and scales to long videos without memory issues.
cross-frame-attentiondiffusionvideo-editingvideo-to-video -
FateZero Discovered by AI
2023
A zero-shot text-based video editing framework that manipulates real-world videos without per-prompt training or user-specified masks. It achieves temporal consistency by fusing attention maps from the source video inversion process into the editing process.
attention-fusiondiffusionvideo-editingzero-shot -
FLATTEN: Optical Flow-Guided Attention for Consistent Text-to-Video Editing Discovered by AI
2023
A text-to-video editing framework that introduces optical flow-guided attention to enforce temporal consistency. By aligning attention maps along optical flow trajectories, it prevents visual flickering and ensures seamless texture and style propagation across frames.
attentiondiffusionoptical-flowvideo-editing -
ProPainter Discovered by AI
2023
ProPainter is a video inpainting framework that improves propagation and spatiotemporal Transformers. It integrates recurrent flow completion, dual-domain propagation, and a mask-guided sparse Transformer to achieve high-quality video completion and object removal.
inpaintingobject-removalvideo-editing -
RAVE Discovered by AI
2023
RAVE is a zero-shot video editing method that uses randomized noise shuffling across video grids in pre-trained text-to-image diffusion models, enabling fast and temporally consistent text-driven video edits without additional training.
diffusiontext-to-videovideo-editingzero-shot -
StableVideo: Text-driven Consistency-aware Diffusion Video Editing Discovered by AI
2023
An ICCV 2023 paper that introduces temporal dependency to text-driven diffusion models for video editing. It utilizes an inter-frame consistency loss and a mapping mechanism to ensure that edited objects maintain a stable and consistent appearance over time.
diffusiontemporal-consistencyvideo-editing -
TokenFlow Discovered by AI
2023
TokenFlow is a training-free framework that enables consistent video editing using a pre-trained text-to-image diffusion model. It ensures temporal consistency by enforcing the consistency of diffusion features across frames based on inter-frame correspondences.
diffusiontemporal-consistencyvideo-editing -
Track Anything (TAM) Discovered by AI
2023
Track Anything (TAM) is an interactive video object tracking and editing framework that combines SAM with video tracking (XMem) and inpainting (E2FGVI) models, allowing users to select, track, and edit video objects seamlessly via click or text prompts.
inpaintingsamtrackingvideo-editing -
Video-P2P Discovered by AI
2023
A framework for real-world video editing with cross-attention control, adapting pre-trained image diffusion models to video tasks while preserving original poses and scenes.
cross-attentiondiffusionvideo-editing -
VideoSwap Discovered by AI
2023
VideoSwap provides customized video subject swapping using interactive semantic point correspondences, enabling subject replacement with distinct shapes and identities while retaining temporal consistency.
diffusioninpaintingsubject-swappingvideo-editing -
VidToMe Discovered by AI
2023
A zero-shot video editing framework that enhances temporal consistency and reduces memory consumption by merging redundant tokens across frames. It aligns and aggregates self-attention tokens from a pre-trained image diffusion model to produce temporally coherent video edits.
diffusiontoken-mergingvideo-editingzero-shot -
E2FGVI Discovered by AI
2022
E2FGVI is an end-to-end framework for flow-guided video inpainting. It integrates flow completion, feature propagation, and content hallucination into a single trainable network, achieving high efficiency and temporal consistency for video object removal and completion.
inpaintingoptical-flowvideo-editing
Proprietary / Industry Tools¶
-
Adobe Premiere Pro Generative Extend Discovered by AI
2025
An AI-powered feature in Adobe Premiere Pro, powered by the Firefly Video Model, that allows video editors to extend the beginning or end of video clips by up to two seconds and audio clips by up to ten seconds to smooth transitions.
fireflyindustry-standardvideo-editingvideo-extension -
Diffusion Studio Discovered by AI
2024
A professional browser-based AI video editor featuring an infinite canvas, timeline-based sequencing, and deep integration with coding agents for automated video workflows.
ai-agentbrowser-basedvideo-editing -
Runway Inpainting Discovered by AI
2024
An AI-powered video editing tool that allows users to seamlessly remove objects, people, or watermarks from video footage using simple brush strokes. It leverages advanced temporal consistency algorithms to ensure clean, artifact-free results.
inpaintingobject-removalvfxvideo-editing -
Runway AI Magic Tools Discovered by AI
2023
A comprehensive suite of cloud-based generative AI tools for video editing, featuring advanced video inpainting, motion brush, object removal, and style transfer. It is widely adopted in professional film and game production pipelines for rapid visual effects prototyping.
cloud-basedinpaintingproprietaryvideo-editing -
Descript Discovered by AI
2020
Descript is an AI-powered collaborative audio and video editor that allows users to edit video by editing the transcribed text. It features automated transcription, AI voice generation, filler word removal, and studio sound enhancement.
text-based-editingtranscriptionvideo-editing
