There was a time when watching YouTube videos or movies in an unfamiliar language meant missing out entirely. Today, however, auto-generated subtitles in your language are just a click away, thanks to advancements in AI.

If you’re a developer interested in how this works, trying it yourself can be an enlightening experience. Take, for example, a video in Marathi. Indian cinema, celebrated for its rich cultural diversity, often sees films remade across different languages. The movie Superboys of Malegaon, for instance, follows an aspiring filmmaker creating a Marathi version of the Hindi classic Sholay. It shows how hard it used to be for people to watch regional films if they didn’t speak the language.

The process starts with transcribing the audio using a script. Large language models can now accurately detect and convert dialogue into text. Even general speech-to-text models perform competently, though specialized models trained for Indian languages deliver even better results. At this stage, you’ll have an SRT file in the original language. That’s the good news. The bad news? For a three-hour movie, you might have around 2,000 lines to translate!

Direct translation seems like the obvious solution, but it typically falls flat, especially because models often lack context—such as the plot and characters—that is essential for accurate translation in film. Instead, a better approach is to leverage large language models as professional Marathi-English translators. You supply the plot, character descriptions, and the tone expected in different scenes. To preserve meaning, you provide guidelines for handling idioms and culture-specific terms. Since you must translate in groups rather than all at once, make sure each subtitle chunk overlaps, ensuring the model retains important context.

At this point, you’ll discover that there are user-friendly free tools that streamline this process. Subtitle Edit, for example, integrates directly with AI models like OpenAI’s Whisper, making it easier to review and adjust subtitles.

After considerable trial and error, you’ll be able to enjoy a film with personalized subtitles. Results may be imperfect, especially if you’re not familiar with the original language, but you’ll gain insight into the technology behind auto-generated subtitles—and perhaps even pick up a few words of Marathi. As always, exploring the "how" behind the so-called magic is eye-opening. So, next time you notice errors in auto-generated subtitles, you’ll appreciate the complexity and ingenuity that makes cross-language viewing possible.