- Clone the repo: git clone https://github.com/EliasLindbergs/ai-video-dubber.git
- Download the following pre-trained models and place them in the corresponding folders:
- epoch_2nd_00050.pth (download link: https://drive.google.com/file/d/1nm0yB6Y5QWF3FYGfJCwQ6zYNlOAYVSet/view) in Models/LibriTTS
- g_00935000 (download link: https://drive.google.com/file/d/1RDxYknrzncGzusYeVeDo38ErNdczzbik/view) in Vocoder/LibriTTS
- In the inference notebook, replace API_KEY with your OpenAI API key (https://platform.openai.com/account/api-keys) and AUTH_TOKEN with your pyannote access token (visit hf.co/pyannote/embedding and accept user conditions, then visit hf.co/settings/tokens to create an access token)
- Re-name your video to “video.mp4” (or whatever extension it has) and place it in a Media folder
- Run the inference notebook and collect the generated dubbed video "final_video_output.mp4" in the Media folder