A video file can carry several audio tracks, and not all of them say what they are. An unlabelled one comes up as “Unknown” wherever you play it, so choosing between them means guessing which is the original, which is a dub, and which is somebody talking over the top. Across a whole collection it adds up to hundreds of tracks nobody ever got around to labelling.
Seen through ffprobe, the standard way to ask a video file what it holds, a labelled track shows its language right after the stream number, like #0:1(nor). These two have nothing there at all:
$ ffprobe "Interviews/Bergen 2018.mkv"
...
Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080, 25 fps
Stream #0:1: Audio: aac (LC), 48000 Hz, stereo (default)
Stream #0:2: Audio: aac (LC), 48000 Hz, stereo
Metadata:
title : Interpreter
audio-lang-tagger fills that in. It works out what language each unlabelled track is in and writes that into the file. Since that means editing my own files, it can also just look and report what is still missing a label, which is the safer place to start. Same file, but the gap now has a name:
$ audio-lang-tagger.py --list ~/video
[1/1] Interviews/
Bergen 2018.mkv
track a1: aac 2ch - language missing
track a2: aac 2ch, "Interpreter" - language missing
It gets there by listening. A few seconds of the track go through whisper.cpp, a speech recognition program running on my own machine, and I get back the words it heard alongside the language it believes they are. Where I have a catalogue that already knows what the original language ought to be, that shows up beside the guess as a second opinion. I press Enter to accept it, or type the right language myself.
Saving the label leaves the picture and the sound untouched. It only rewrites the note attached to them, which is why even a 40 GB file is finished in well under a second. Every change is written down as it happens, so anything I get wrong can be put back:
$ audio-lang-tagger.py --ledger 2
Ledger 2 rows manual 2
~/.local/state/audio-lang-tagger/lang_tagger_tags.tsv
2026-08-27 19:42:11 und->nor manual p0.96 a1 Interviews/Bergen 2018.mkv
2026-08-27 19:42:29 und->eng manual p0.91 a2 Interviews/Bergen 2018.mkv
It can also be left to run on its own, and the bar for labelling anything without asking is worth stating plainly. Every sample taken from a track has to come back as the same language at 0.90 confidence or better, there has to be enough varied speech behind that verdict to rule out singing, and an outside source has to agree independently. Past that it refuses whole categories where it knows it is unreliable: anything made before 1940, where dialogue tends to be thin; music and concert recordings; files carrying more than one audio track; and the Norwegian, Danish and Swedish cluster it mixes up too often to be trusted with. Everything that falls short of all of that waits for me.
Setting the label by hand has always been possible, and MKVToolNix does it in seconds once you know the answer. Working out the answer is the part that does not scale, and ULDAS already automates it, along with a good deal more. What it lacks is a setting between a dry run and writing every label it is confident about, and that middle is what I wanted, because the two mistakes here are not the same size. Being asked costs a second. A wrong label is silent, stays in the file, and gets acted on later by whatever reads it.
Nor does confidence sort the safe cases from the risky ones. A musical short came back confidently labelled off a transcript that was a 219-word “la la la” loop, while a sparse cartoon with 99 perfectly clear words scored badly. They fail in opposite directions, so no single number puts them in the right order. I would rather be shown what was heard.
Cut at 0.35 and both pass, at 0.65 only the music passes, and at 0.90 neither does.
In practice the listening happens ahead of time, unattended on the machine that holds the files, so by the time I sit down to work through the results there is nothing left to wait for.