SUBTITLE COMPARISON TEST FILES
==============================

These three files are a test set for subtitle comparison tools. Anyone can use
them. Put them into any subtitle comparer and read what it reports.

THE FILES

  source-en.srt              The English source. 187 cues.
  translation-es-good.srt    The Spanish translation, by a person. 187 cues.
                             It has no faults in it.
  translation-es-seeded.srt  The same Spanish translation with 16 known
                             changes in it. 183 cues.

WHAT IS IN THE SEEDED FILE

Six of the changes are real faults:

  3 cues deleted             The line is not in the translation.
  2 cues left in English     The line was never translated.
  1 numeric cue left in
    English                  The same fault, on a line that is mostly digits.

Ten of the changes are NOT faults. They are normal translator work:

  3 pairs of cues merged     Two lines joined into one line.
  2 cues split               One line divided into two lines.

The merges and the splits are the important part of this test. They move every
cue after them. A tool that compares cue 1 with cue 1 reports the whole rest of
the file as different.

HOW TO USE THE TEST

Compare source-en.srt against translation-es-seeded.srt in your tool. Then
count two numbers:

  1. How many of the 6 real faults did the tool find?
  2. How many flags did it raise that are not faults?

The second number is the one that decides if a tool is usable. A tool that
raises 123 wrong flags hides the 6 that matter.

WHERE THE FAULTS ARE

  Cues 26, 27, 28    the untranslated lines (28 is the numeric one)
  Cues 151, 152, 153 the deleted lines
  Cues 61, 71, 81    the merged pairs (not faults)
  Cues 101, 111      the divided cues (not faults)

MEASURED RESULTS

  Matcher                       Flags   Wrong flags   Real faults found
  Position (cue 1 to cue 1)      127        123            0 of 6
  Start time (+/- 100 ms)         11          5            3 of 6
  Time overlap                    10          0            5 of 6

A fault counts as found only when the tool names the cue that holds it.
Position matching raises 4 flags of the right type, but they name cues 184 to
187 at the end of the file, and the deleted cues are 151 to 153. So it finds
none of the six.

The result is the same in 12 language pairs.
Method and full results: https://subtitlecheck.keelworks.io/subtitle-diff-accuracy-test/

SOURCE AND LICENCE

The subtitle tracks come from Wikimedia Commons. The video is
"(Nachhaltiger Urlaub in Praxis) E-Bike Urlaub 2021 - Bahnrueckreise von Verona
nach Deutschland nach Via Claudia Augusta.webm" by SaftRAD.

The video and its subtitle tracks are under the Creative Commons Attribution
3.0 Unported licence (CC BY 3.0).
https://creativecommons.org/licenses/by/3.0/

The seeded file is a changed copy of that work, and it is under the same
licence. You must keep the attribution to SaftRAD if you share these files.