Facial Animation & Lip-Sync¶
Animation & Motion
Open Source¶
-
AVTR-1 Discovered by AI
2026
An open-source stack for real-time interactive avatar conversations, featuring a compact 153M-parameter autoregressive flow-matching motion generator conditioned on both participants' audio.
audio-drivenavatarflow-matchingreal-time -
EmoTaG Discovered by AI
2026
An emotion-aware talking head synthesis framework based on 3D Gaussian Splatting that enables few-shot personalization with expressive facial motion and stable geometry.
emotion-awarefew-shotgaussian-splatting -
LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation Discovered by AI
2026
A novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. It addresses the latency-quality trade-off by utilizing a single-step generation process to prevent error accumulation and identity drift.
lip-syncreal-timesingle-steptalking-head -
Xemo-Talker Discovered by AI
2026
An audio-driven talking portrait synthesis framework that achieves explicit and precise emotion control by decoupling neutral speech-to-motion mapping from emotion-related variations.
audio-drivenemotion-controltalking-head -
AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinement Discovered by AI
2025
An audio-driven framework for generating multi-person talking videos. It features an extensible multi-stream processing architecture to scale multiple identities while ensuring seamless inter-identity interactions and realistic lip synchronization.
lip-syncmulti-persontalking-headvideo-generation -
Audio2Face-3D Discovered by AI
2025
An open-source suite of AI-driven facial animation models and SDKs by NVIDIA that generates 3D facial blendshapes, jaw, tongue, and eye movements in real time from speech audio.
3d-facial-animationaudio-drivenfaciallip-sync -
EchoMimicV3 Discovered by AI
2025
An efficient 1.3B parameter framework that unifies multi-task and multi-modal human animation. It generates high-quality talking human videos from a reference image, audio, and text prompt without requiring predefined 2D or 3D poses.
diffusionlip-syncmultimodaltalking-head -
EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters Discovered by AI
2025
EmoHead is an emotional talking head generation framework that manipulates semantic expression parameters to synthesize highly realistic, emotion-specific talking head videos from audio inputs.
audio-drivenemotional-expressionlip-synctalking-head -
FantasyTalking Discovered by AI
2025
A framework that leverages a pretrained video diffusion transformer (DiT) model to generate high-fidelity, coherent talking portraits with controllable motion dynamics. It uses a dual-stage audio-visual alignment to capture subtle facial expressions, global body movements, and dynamic backgrounds.
audio-drivendiffusion-transformerlip-synctalking-head -
Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization and Temporal Motion Modulation Discovered by AI
2025
An advanced portrait animation framework that utilizes direct preference optimization (DPO) and temporal motion modulation to improve lip-audio synchronization, expression vividness, and body motion coherence.
audio-drivendiffusionfaciallip-sync -
HunyuanVideo-Avatar Discovered by AI
2025
A multimodal diffusion transformer (MM-DiT) based model by Tencent that generates dynamic, emotion-controllable, and multi-character talking avatar videos from audio conditions. It supports animating any input avatar image with high fidelity and precise lip-sync.
diffusion-transformerlip-synctalking-headGitHub · arXiv · Hugging Face
-
JoyGen: Audio-Driven 3D Depth-Aware Talking-Face Video Editing Discovered by AI
2025
A two-stage framework for talking-face video editing that integrates audio-driven lip motion generation with 3D depth-aware visual appearance synthesis. It enables precise lip-audio synchronization and high-fidelity facial rendering.
3d-depthlip-synctalking-headvideo-editing -
Livatar-1: Real-Time Talking Heads Generation with Tailored Flow Matching Discovered by AI
2025
A real-time audio-driven talking head generation framework based on flow matching, designed to address limited lip-sync accuracy and long-term pose drift.
flow-matchinglip-syncreal-timetalking-head -
LLIA Discovered by AI
2025
A real-time, low-latency audio-driven portrait video generation framework based on diffusion models. It optimizes computational requirements to enable deployment in interactive avatar applications requiring high speed and minimal latency.
lip-synclow-latencyreal-time -
MultiTalk (Let Them Talk) Discovered by AI
2025
An audio-driven multi-person conversational video generation framework that handles multi-stream audio inputs. It addresses the incorrect binding problem between audio and multiple individuals, enabling synchronized facial movements and conversation dynamics for multiple characters.
lip-syncmulti-persontalking-head -
OmniHuman-1 Discovered by AI
2025
A Diffusion Transformer-based framework for end-to-end human animation that scales up data by mixing motion-related conditions. It generates highly realistic human videos from a single portrait and weak driving signals like audio, supporting various aspect ratios and body proportions.
diffusion-transformerlip-synctalking-head -
PersonaLive Discovered by AI
2025
A real-time, diffusion-based portrait animation framework designed for live streaming. It uses hybrid implicit signals (implicit facial representations and 3D implicit planes) to achieve high-quality, expressive facial animation and lip-sync with 7-22x speedup over prior diffusion models.
facial-animationlip-synclive-streamingreal-time -
SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting Discovered by AI
2025
SyncTalk++ is a high-fidelity and efficient synchronized talking head synthesis framework that leverages 3D Gaussian Splatting to achieve realistic coordination of subject identity, lip movements, facial expressions, and head poses.
3d-avatargaussian-splattinglip-synctalking-head -
X-Actor Discovered by AI
2025
X-Actor is an audio-driven portrait animation framework designed to generate lifelike, emotionally expressive, and long-form talking head videos from a single reference image.
expressivelip-synctalking-head -
AniPortrait Discovered by AI
2024
An audio-driven portrait animation framework that extracts 3D intermediate representations from audio and projects them into 2D landmarks to guide a diffusion model.
facial-animationlip-synctalking-head -
AniTalker Discovered by AI
2024
A framework that decouples identity from facial motion to animate vivid and diverse talking faces from a single portrait and audio input, enhancing natural head movements and expressions.
audio-drivenfaciallip-sync -
DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis Discovered by AI
2024
A 3D Gaussian Splatting (3DGS)-based talking face synthesis method that addresses the challenge of rendering realistic long hair and complex head motions by decomposing the scene into static, dynamic, and hair-specific Gaussian fields.
3dgsgaussian-splattinghair-renderingtalking-head -
Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis Discovered by AI
2024
A motion-space diffusion model designed for controllable, real-time talking head synthesis, achieving high-fidelity lip-sync and facial expressions with low latency.
diffusionlip-syncreal-timetalking-headGitHub · arXiv · Hugging Face · Website
-
EchoMimic Discovered by AI
2024
An audio-driven portrait animation framework that utilizes editable landmark conditioning to generate lifelike facial expressions and natural head movements.
facial-animationlip-synctalking-head -
EchoMimicV2 Discovered by AI
2024
An audio-driven human animation framework that simplifies portrait and semi-body animation by generating expressive facial expressions, natural head poses, and upper-body movements from audio inputs.
audio-drivenfaciallip-sync -
EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis Discovered by AI
2024
An efficient disentanglement framework for emotional talking head synthesis that enables precise control over mouth shape, head pose, and emotional expression. It utilizes orthogonal bases and an audio-to-motion module to achieve realistic audio-driven facial animation.
audio-drivenfacial-animationlip-synctalking-head -
EMO: Emote Portrait Alive Discovered by AI
2024
An expressive audio-to-video diffusion model that directly synthesizes vocal avatar videos with rich facial expressions and natural head poses from a single reference image and audio.
facial-animationlip-synctalking-head -
EmoVOCA: Speech-Driven Emotional 3D Talking Heads Discovered by AI
2024
EmoVOCA is a speech-driven emotional 3D talking head generation framework that synthesizes realistic 3D facial animations with rich emotional expressions from audio inputs.
3d-facial-animationemotional-expressionlip-synctalking-head -
FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portrait Discovered by AI
2024
A flow-matching-based audio-driven talking portrait generation method that shifts generative modeling from pixel-based latent space to a learned orthogonal motion latent space, enabling efficient and temporally consistent video generation from a single source image and audio.
flow-matchinglip-syncportrait-animationtalking-head -
Hallo Discovered by AI
2024
A hierarchical audio-driven visual synthesis framework for portrait image animation, achieving high-fidelity lip synchronization and diverse natural expressions.
facial-animationlip-synctalking-head -
Hallo2 Discovered by AI
2024
An audio-driven portrait image animation model designed for long-duration and high-resolution video synthesis. It addresses appearance drift and temporal artifacts to generate controllable, high-fidelity talking head videos at 4K resolution.
diffusionhigh-resolutionlip-synctalking-head -
Hallo3 Discovered by AI
2024
A diffusion transformer-based portrait animation framework that integrates speech audio conditioning to generate highly dynamic, realistic, and identity-preserving talking head videos from non-frontal perspectives.
audio-drivenfaciallip-sync -
JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation Discovered by AI
2024
A diffusion-based audio-driven facial animation framework that decouples facial appearance from motion, enabling realistic lip-sync and head pose generation for both human portraits and animal faces.
audio-drivendiffusionfaciallip-sync -
LatentSync Discovered by AI
2024
An end-to-end lip-sync framework based on audio-conditioned latent diffusion models that bypasses intermediate motion representations to generate high-fidelity, temporally consistent mouth movements.
audio-drivendiffusionlip-sync -
LivePortrait Discovered by AI
2024
An efficient, high-quality portrait animation framework based on implicit keypoints, enabling precise stitching and retargeting control for facial expressions and head poses.
facial-animationlip-synctalking-head -
MuseTalk Discovered by AI
2024
A real-time, high-fidelity video dubbing and lip-syncing framework that operates in latent space to preserve identity consistency and dental details.
facial-animationlip-synctalking-head -
Sonic Discovered by AI
2024
A portrait animation model that shifts focus to global audio perception, enabling stable, seamless, and highly synchronized lip-syncing and facial expressions over extended video durations.
audio-drivenfaciallip-sync -
TalkingGaussian Discovered by AI
2024
A structure-persistent 3D talking head synthesis framework based on 3D Gaussian Splatting that reconstructs high-fidelity facial details and achieves real-time rendering speeds.
3d-gaussian-splattinglip-syncreal-timetalking-head -
V-Express Discovered by AI
2024
A portrait video generation framework that uses conditional dropout and progressive training to balance control signals like audio, pose, and reference images.
facial-animationlip-synctalking-head -
DreamTalk Discovered by AI
2023
A diffusion-based framework designed to generate highly expressive and emotional talking heads from audio inputs, maintaining visual quality and lip synchronization across diverse emotional styles.
diffusionemotional-talking-headexpressivelip-sync -
EmoTalk Discovered by AI
2023
An end-to-end neural network architecture for speech-driven 3D facial animation that disentangles emotion and speech content to generate highly expressive, emotion-controllable 3D blendshapes and facial meshes.
audio-drivenemotionfaciallip-sync -
GeneFace++ Discovered by AI
2023
A generalized and stable real-time 3D talking face generation framework using NeRF. It achieves high-fidelity audio-lip synchronization, natural head motion, and real-time rendering speeds.
3d-avatarlip-syncnerftalking-head -
SyncTalk Discovered by AI
2023
SyncTalk is an audio-driven talking head synthesis framework that focuses on the synchronization of subject identity, lip movements, facial expressions, and head poses using a NeRF-based architecture.
lip-syncnerftalking-head -
Talking Head (3D) Discovered by AI
2023
A JavaScript class and web application for real-time lip-sync and facial expressions using full-body 3D avatars (such as Ready Player Me GLB models) in the browser. It supports integration with major text-to-speech (TTS) APIs and handles Mixamo animations, markdown text, and subtitles.
3d-avatarlip-syncreal-timewebgl -
VividTalk Discovered by AI
2023
VividTalk is a two-stage framework for generating high-visual-quality talking head videos from a single image and audio, utilizing a 3D hybrid prior to model expressive facial expressions and natural head poses.
3d-priorlip-synctalking-head -
SadTalker Discovered by AI
2022
A widely used framework that generates realistic 3D motion coefficients (head pose, expression) from audio to animate a single static portrait image.
facial-animationlip-synctalking-head -
VideoReTalking Discovered by AI
2022
A generic system for audio-based talking-head video editing that decouples the process into expression editing, lip-syncing, and face enhancement, producing high-quality results on in-the-wild videos.
audio-drivenfaciallip-syncvideo-editing -
MeshTalk Discovered by AI
2021
A cross-modal disentanglement framework for audio-driven 3D facial animation that synthesizes dynamic expressions across the entire face, including upper-face motion independent of audio content.
3d-facial-animationaudio-drivenfaciallip-sync -
uLipSync Discovered by AI
2021
A real-time, MFCC-based lip-sync plugin for Unity. It uses Unity's Job System and Burst Compiler to perform high-performance audio analysis, allowing 3D characters (such as VRM models) to lip-sync dynamically to microphone input or audio sources.
lip-syncreal-timeunityvrm -
Wav2Lip Discovered by AI
2020
A classic and robust lip-syncing model designed to accurately morph the lip movements of arbitrary talking face videos to match target speech audio.
facial-animationlip-synctalking-head -
Rhubarb Lip Sync Discovered by AI
2016
A command-line tool that automatically generates 2D mouth animation (visemes) from voice recordings. It is widely used in game development and 2D animation to sync character mouths with audio files, supporting output formats for Spine, After Effects, and custom game engines.
2d-animationgame-devlip-syncviseme
Proprietary / Industry Tools¶
-
InfiniteTalk Discovered by AI
2025
InfiniteTalk is an AI lip-sync video API that adds audio-driven synchronization on top of Wan 2.1, aligning lips, facial expressions, and head motion with the input audio.
apilip-syncwan-2.1 -
Hedra Discovered by AI
2024
An advanced AI video creation platform that generates lifelike talking avatars with highly accurate lip-syncing and natural facial expressions from text or audio.
facial-animationlip-synctalking-head -
Rask AI Discovered by AI
2023
A popular commercial video localization and dubbing platform that features advanced multi-speaker AI lip-syncing to match translated audio with natural mouth movements.
dubbinglip-synclocalizationvoice-cloning -
Sync Labs Discovered by AI
2023
A commercial platform and API providing real-time, studio-grade AI lip-syncing and video dubbing. It allows users to animate people speaking any language in any video, including movies, games, and animations.
apidubbinglip-syncreal-time -
NVIDIA Omniverse Audio2Face Discovered by AI
2021
An AI-powered application by NVIDIA that simplifies the animation of 3D characters to match any voice-over track. It generates realistic facial expressions and lip-sync in real-time, supporting integration with Unreal Engine (MetaHumans) and other 3D pipelines.
3d-avatarfacial-animationlip-sync -
JALI Research Discovered by AI
2020
A high-end procedural animation technology and software suite that automates multilingual lip-sync and expressive facial animation. It integrates with Maya and game engines like Unreal Engine, providing animators with precise directorial control.
facial-animationlip-syncprocedural -
D-ID Discovered by AI
2017
A leading commercial platform and API that animates face images into talking-head videos from text or audio input. It generates realistic lip-sync, facial expressions, and natural head movements for virtual presenters and localization.
apiavatarlip-synctalking-head -
Speech Graphics SGX Discovered by AI
An industry-standard procedural facial animation and lip-sync platform that generates natural, localized facial motion and lip-sync from speech audio across thousands of game dialog lines.
audio-drivenfacialgame-productionlip-sync
