Measurement
Three ways to match two subtitle files, measured on the same file, with the same faults put into it. The test files are on this page. You can repeat the test in your own tool.
A text diff shows you the lines that are different. In a translation, every line is different. That is the purpose of a translation. So a normal diff reports the whole file and tells you nothing.
A subtitle comparer must therefore do something else. It must first decide which line in file A is the same line as which line in file B. That decision is called matching, and it is where these tools differ from each other.
The free subtitle comparers match by position. They compare line 1 with line 1, line 2 with line 2, and so on. One of them says so in its own help text:
"The tool compares entries by index position. Entry #1 in File A is compared with entry #1 in File B, and so on." SubtitleWise, Subtitle Diff, frequently asked questions. Read on 20 August 2026.
Position matching works only while the two files hold the same number of lines in the same order. A translator breaks that condition as a matter of normal work. A translator joins two short lines into one line, because the target language is shorter. A translator divides one long line into two lines, because the target language is longer.
Each join and each division moves every line after it by one place. From that point the tool compares line 40 with line 41, then line 41 with line 42, and it reports each of them as an error. One merge near the start of a file is enough to make the rest of the file read as wrong.
The source is an English subtitle track of 187 cues, with a Spanish translation made by a person. Both come from Wikimedia Commons. This is volunteer work and not broadcast work, which makes it a harder test and not an easier one.
I made a copy of the good Spanish file and put 16 known changes into it. Six of the changes are real faults:
Ten of the changes are not faults. They are normal translator work:
A tool must find the six faults. A tool must stay silent about the ten changes that are not faults. Those ten are the drift that moves the later lines.
| Matcher | Flags raised | Wrong flags | Real faults found |
|---|---|---|---|
| Position (the free comparers) | 127 | 123 | 0 of 6 |
| Start time | 11 | 5 | 3 of 6 |
| Time overlap | 10 | 0 | 5 of 6 |
A fault counts as found only when the tool names the cue that holds it. This is the part that is easy to measure wrongly. Position matching does raise 4 flags of the correct type, and a count of flag types alone would read that as a pass. The 4 flags name cues 184 to 187, which are at the end of the file. The deleted cues are 151, 152 and 153. The tool names the wrong lines, so it finds nothing.
The 10 flags from overlap matching are 3 deleted cues, 2 untranslated cues, and 5 notes that name the merges and the divisions as what they are. None of the 10 is wrong.
The result is the same in 12 language pairs, which are Arabic, Greek, Spanish, Italian, Japanese, Korean, Dutch, Polish, Russian, Thai, Ukrainian and Chinese.
Overlap matching finds 5 of the 6 real faults. It misses the untranslated numeric line. That line is mostly digits, so the language of the text cannot be read from it, and the digits are correct in both files. This is a real limit and I state it here because the measurement found it.
The seeded faults sit at cues 26, 27 and 28, and at cues 151, 152 and 153. I mention the numbers because they matter to the test. Cues 26, 27 and 28 are next to each other, so a scorer that accepts a flag one cue away from the fault reports 6 of 6 for overlap matching. That answer is wrong. The numbers above use the exact cue only.
These are the files. Put the source and the seeded translation into any subtitle comparer, then count the flags it raises and read how many of them are real.
Subtitle Check uses overlap matching. It reads the two files in your browser, and the files do not leave your computer.
Open the free subtitle checker
The subtitle tracks come from Wikimedia Commons. The video is "(Nachhaltiger Urlaub in Praxis) E-Bike Urlaub 2021 - Bahnrueckreise von Verona nach Deutschland nach Via Claudia Augusta.webm" by SaftRAD. The video and its subtitle tracks are under the Creative Commons Attribution 3.0 Unported licence.
The seeded file is a changed copy of that work and is under the same licence. Keep the attribution to SaftRAD if you share these files.
Licence text: creativecommons.org/licenses/by/3.0/
Subtitle Check (2026). "A subtitle diff raises 123 false errors on a good translation." https://subtitlecheck.keelworks.io/subtitle-diff-accuracy-test/