1. Abstract
LongCat-Video-Avatar is an audio-driven avatar (virtual human) video generation model based on the LongCat-Video architecture, which is suitable for "long time sequence, strong consistency, realistic and dynamic" scenarios. It natively supports Audio-Text-to-Video (AT2V), Audio-Text-Image-to-Video (ATI2V), and Video Continuation, and emphasizes stability and identity consistency in multi-character and long-form video generation.
2. Core features
1. Integration of three native modes: the same framework covers AT2V, ATI2V and video continuation, and is suitable for single/multiple people and single/multiple audio inputs.
2. Stable image quality for long videos: Cross-Chunk Latent Stitching reduces pixel degradation and error accumulation in long sequences by reducing repeated VAE decode-encode loops across segments, improving the quality of seamless stitching.
3. Long-term identity consistency: Reference Skip Attention suppresses the stiff copy-paste feeling caused by "conditional image leakage" while retaining the character ID characteristics.
4. More natural human dynamics: Through decoupling unconditional guidance, speech signals and action dynamics are more reasonably separated, reducing the unnatural phenomenon of "mouth movement but stiffness".
5. Evaluation alignment with real perception: The model card provides a display of human evaluation results based on EvalTalker, which is used to measure the naturalness and realism of single/multiple people (the specific conclusion is subject to the official report and reproduction experiment).
3. Installation
- Clone the code and create an environment:
- git clone --single-branch --branch main https://github.com/meituan-longcat/LongCat-Video
- Recommended Python 3.10 + Conda environment
- 2. Install dependencies (select the corresponding torch according to the CUDA version):
- Install torch/torchvision/torchaudio
- and install flash-attn (FlashAttention-2 is enabled by default in the model configuration, which can be switched as needed).
- pip install -r requirements.txt
- Avatar Additional dependencies: librosa, ffmpeg, pip install -r requirements_avatar.txt
- 3. Download weight: Use huggingface-cli to connect LongCat-Video with LongCat-Video-Avatar Download to the local weights directory.
4. Typical use cases
- Virtual anchor/oral broadcast: Generate long videos with stable lip shapes and expressions based on audio and scripts.
- Podcast/interview: Support long-term speaking and multiple people to rotate speeches, emphasizing identity consistency and natural splicing.
- Singing/stage performance: Make the rhythm of the action more synchronized with the vocals, suitable for the continuous generation of short clips to long paragraphs.
- Customer service/sales explanation: Transition between silent paragraphs and pauses more naturally to reduce the look and feel of "stuck".
- Video continuation: Expand generation to multi-paragraph splicing, suitable for "unlimited length" content production and iterative editing.
5. Ecology and competitors
- Ecology: LongCat-Video provides text/image-to-video and writing capabilities; Avatar complements the audio driver and multi-person conversation video generation, and gives quick inference scripts and parameter suggestions with the Hugging Face model card.
- Competing products/related directions: InfiniteTalk (sparse frame dubbed long video), StableAvatar (emphasizing infinite length and end-to-end), MultiTalk (multi-person dialogue generation) and other solutions have their own emphasis on "identity preservation, long sequence stability, and multi-person interaction". The differences between LongCat-Video-Avatar are more focused on the cross-segment stabilization mechanism and the way the reference information is injected.
6. Limitations and precautions
- Long videos and multiple characters have high requirements for video memory and computing power, and the inference speed is strongly related to resolution/frame rate/number of segments.
- Multi-audio mode needs to be aligned in length or spliced according to rules, and improper input organization will affect the naturalness of lip sync and rotation.
- Generative video may have detail drift, occasional inaccurate lip shape, or defects in high-frequency areas such as hands/teeth, so it is recommended to manually review and post-retouch key clips.
- Applications involving portraits and voices must comply with privacy, authorization, and content compliance requirements, especially in commercial and public release scenarios.
7. Project address
https://github.com/meituan-longcat/LongCat-Video
8. FAQ
Question: Is LongCat-Video-Avatar supported? AT2V (Audio + Text to Video)?
Answer: Yes, the model natively provides an AT2V inference entry and suggests adding clear action cues like "speak/talk" to the prompt to improve lip-syncing and dynamics.
Question: What scenarios is LongCat-Video-Avatar's ATI2V (Audio + Text + Image to Video) suitable for?
Answer: It is suitable for virtual human generation that needs to fix the character's image/clothing style, enhancing the consistency between ID and appearance through reference diagrams.
Question: How does LongCat-Video-Avatar reduce the problem of long videos becoming more and more blurry?
Answer: Its Cross-Chunk Latent Stitching is designed to reduce VAE loops that repeat across segments, resulting in lower pixel degradation and error accumulation.
Question: Can LongCat-Video-Avatar do multi-character dialogue and take turns?
Answer: Multiplayer mode and multiple audio input formats are supported, but they need to be configured according to the parameters and audio organization of the official script to get a more natural rotation effect.
Q: What is the core difference between LongCat-Video-Avatar and InfiniteTalk?
Answer: InfiniteTalk is more inclined to the path of "dubbing/aligning existing videos in a long time"; LongCat-Video-Avatar emphasizes audio-driven generation and continuation under a unified architecture, and improves the consistency of long sequences through cross-clip stabilization.
