How to Lip-Sync Any Video for Free: The Wav2Lip Colab Blueprint

Phase 1: The Raw Materials (Quality Control)

Wav2Lip is sensitive. If you feed it garbage, it gives you glitches. You need pristine input files to get a professional result.

1. The Video Source (The Face)

   You need a video of a character looking at the camera. You might generate this in Sora, Runway Gen-2, or Pika Labs. Do not screen record this. Screen recordings have variable frame rates that confuse the AI. Use BulkAiDownload to extract the raw .mp4 file directly from the server. Rename this file to input_video.mp4.

2. The Audio Source (The Voice)

   Go to ElevenLabs or OpenAI TTS. Generate your script. Ensure the audio is clear with no background music. Background music confuses the lip detection model. Rename this file to input_audio.wav.

Phase 2: Setting Up the Colab Environment

We will use Google Colab to "rent" a Tesla T4 GPU for free. This GPU will handle the heavy math required to reshape the mouth movements.

Step 1: Open and Configure

Go to colab.research.google.com and create a "New Notebook." In the top menu, go to Runtime > Change runtime type and select T4 GPU. This is mandatory. The code will not run on a CPU.

Step 2: Install Dependencies

Copy and paste this code into the first cell and click the "Play" button. This installs the Wav2Lip software and the necessary libraries.

# Clone the Repository
!git clone https://github.com/Rudrabha/Wav2Lip.git

# Install Dependencies
!pip install -r Wav2Lip/requirements.txt
!pip install librosa==0.8.0
!wget "https://www.adrianbulat.com/downloads/python-fan/s3fd-619a316812.pth" -O "Wav2Lip/face_detection/detection/sfd/s3fd.pth"

print("✅ Installation Complete")

Step 3: Download the AI Model

Now we need the "Brain" of the AI. There are two versions: "Wav2Lip" (Standard) and "Wav2Lip GAN" (Better Visuals). We will use the GAN version because it produces sharper teeth and lips.

# Download the GAN Model
!wget "https://huggingface.co/camenduru/Wav2Lip/resolve/main/checkpoints/wav2lip_gan.pth" -O "Wav2Lip/checkpoints/wav2lip_gan.pth"

Phase 3: Running the Sync

Now, look at the "Files" tab on the left sidebar of Colab. Drag and drop your input_video.mp4 and input_audio.wav into the Wav2Lip folder.

Once they are uploaded, run this final command to generate your video:

# Run Inference
!cd Wav2Lip && python inference.py \
    --checkpoint_path checkpoints/wav2lip_gan.pth \
    --face "input_video.mp4" \
    --audio "input_audio.wav" \
    --pads 0 10 0 0 \
    --resize_factor 1

Understanding the settings:
--pads 0 10 0 0: This adds padding to the chin area. If the mouth looks cut off at the bottom, increase the "10" to "20".
--resize_factor 1: This keeps the resolution as high as possible.

Phase 4: Fixing the "Blurry Mouth" Issue

When the video finishes, you will find it in the results folder. You might notice the lip area looks slightly softer than the rest of the 4K video. This is normal. The AI model operates at a lower resolution (96x96 pixels) and pastes the mouth back on.

To fix this, you must run the video through an Upscaler. Do not skip this step if you want to be professional. Using the FFmpeg upscaler script we shared in our previous guide will blend the sharp edges of the new mouth with the rest of the face, making the edit invisible to the naked eye.

Troubleshooting Common Errors

Here are the fixes for the red error messages you might see in Colab.