# LatentSync > LatentSync is an open-source (Apache 2.0) lip-sync model from ByteDance, first published in December 2024 (arXiv 2412.09262), that takes an existing video and a speech track and regenerates the mouth area so the lips match the new audio. It works in the latent space of Stable Diffusion with Whisper audio features and SyncNet supervision; version 1.6 (June 2025) is trained at 512×512. On Kenerate AI it runs in Avatar Talk at 3 credits a second, for clips up to 5 minutes. URL: https://kenerateai.com/models/bytedance/latentsync Publisher: Kenerate AI (https://kenerateai.com) Last updated: 31 October 2026 ## Key facts - Official name: LatentSync - Repo ID: bytedance/LatentSync - Maker: ByteDance - First published: December 2024 (arXiv 2412.09262) - Latest version: 1.6 — 11 June 2025 - Licence: Apache 2.0 (open weights) - Inputs: Video + speech audio - Length on Kenerate: 3 s to 5 min - Self-host VRAM: 8 GB (1.5) · 18 GB (1.6) - On Kenerate: 3 credits a second ## Steps 1. Open Avatar Talk — Go to /app/avatar-talk, choose Lip sync a video and pick LatentSync (the button on this page selects it). 2. Upload the video — Add a clip of up to 5 minutes where one person faces the camera and the mouth is clearly visible. Use your own footage or footage you have the rights and consent to re-voice. 3. Add the new speech — Upload a clean voice track, or make one in Kenerate's voice studio. Remove music and background noise first for the tightest sync. 4. Generate — Check the credit cost (3 a second, 3 s minimum) and generate. Download the MP4 when it's ready. ## In depth ### What a LatentSync video costs LatentSync is billed per second of speech, at one flat rate: - Under 3 s — 9 credits (billed as 3 seconds). - 30 s — 90 credits. A social clip or ad variant. - 1 min — 180 credits. A short explainer in a new language. - 5 min — 900 credits. The longest single clip. ### How LatentSync works Older lip-sync tools first predict mouth landmarks or 3D motion, then render a face from them. LatentSync skips that middle step: it runs diffusion directly in Stable Diffusion's latent space, using masked frames of your video plus a reference frame so identity and background stay put, and Whisper audio embeddings to decide the mouth shape for each frame [1][2]. During training it decodes an estimate of each clean frame and scores it with SyncNet (lip sync), LPIPS (image quality) and TREPA (temporal consistency), which is how it keeps lips in sync without the flicker diffusion models are known for [1][2]. Version 1.5 added a temporal layer; version 1.6 moved training to 512×512 to sharpen the mouth [1]. ### Getting good results Start from footage where the speaker faces the camera and the mouth is clearly visible for the whole clip. Match the new speech to roughly the length of the video, cut out music and noise, and keep one face in frame. If the result looks too static for a big change in tone, try InfiniteTalk Fast · Video, which also moves the head and expression. Only re-voice people who have agreed to it. References: [1] bytedance/LatentSync — GitHub: https://github.com/bytedance/LatentSync [2] LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision — arXiv 2412.09262: https://arxiv.org/abs/2412.09262 ## FAQ Q: What is LatentSync? A: LatentSync is ByteDance's open-source lip-sync model. Given a video of someone talking and a new audio track, it regenerates the mouth region frame by frame so the lips match the new speech. The paper (arXiv 2412.09262) was first published in December 2024, and the code and weights are on GitHub under bytedance/LatentSync. Q: How does LatentSync work? A: It's an audio-conditioned latent diffusion model built on Stable Diffusion. Whisper converts the audio into embeddings that steer the U-Net through cross-attention, while reference and masked frames from your video keep the face and scene intact. Training adds SyncNet, LPIPS and TREPA losses so the lips stay in sync and the frames stay steady. Q: What's new in LatentSync 1.6 vs 1.5? A: LatentSync 1.5 (14 March 2025) added a temporal layer for steadier frames, improved Chinese-language videos and cut training memory. LatentSync 1.6 (11 June 2025) was trained on 512×512 video to reduce the blurry mouth area of earlier versions, and needs about 18 GB of VRAM to run locally versus 8 GB for 1.5. Q: Is LatentSync free? A: The model is free to download and self-host under the Apache 2.0 licence, if you have a suitable GPU. On Kenerate it costs 3 credits a second; Avatar Talk only needs a verified email, so the 25 starter credits on a new account cover a short test clip of about 8 seconds. Q: How much does LatentSync cost on Kenerate? A: 3 credits per second of speech, with a 3-second minimum. A 30-second clip is 90 credits, one minute is 180 and a full 5-minute video is 900. That makes it the cheapest video lip sync in Avatar Talk. Q: Can I use LatentSync in ComfyUI? A: Yes, community ComfyUI nodes and hosted ComfyUI workflows exist for LatentSync, but you need a local GPU and the model files. On Kenerate you skip the setup: upload a video and audio in the browser and it runs on our side. Q: Is there a LatentSync API? A: ByteDance publishes the code, not a hosted API; several cloud platforms host it. If you want to use it without code, Kenerate runs LatentSync in Avatar Talk: upload the video and the speech, and the credit cost shows before you generate. Q: What's a good LatentSync alternative? A: In the same studio: InfiniteTalk Fast · Video (4.5 credits a second, up to 10 minutes) also moves the head and expression; VEED Lipsync 2 (22.5) gives clean, quick re-voicing; Sync Lipsync 3 (40.5) has the most precise mouth shapes; Sync React-1 (50.1, up to 15 s) changes the emotion too. You can run each on the same clip and compare. Q: Which videos work best with LatentSync? A: Front-facing talking heads with one person, steady framing, good light and an unobstructed mouth. Clean speech without music helps a lot. Profiles, hands over the face, beards hiding the lips or several faces in one shot give weaker results. ## Related pages - AI lip sync generator: https://kenerateai.com/ai-lipsync-generator - Lip sync AI: https://kenerateai.com/lip-sync-ai - InfiniteTalk: https://kenerateai.com/models/meigen-ai/infinitetalk - VEED Fabric 1.0: https://kenerateai.com/models/veed/fabric-1-0 - Best AI lip sync tools 2026: https://kenerateai.com/blogs/best-ai-lip-sync-tools-2026 ## Sources - bytedance/LatentSync — GitHub: https://github.com/bytedance/LatentSync - LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision — arXiv 2412.09262: https://arxiv.org/abs/2412.09262