Skip to content

Croatian is read as Serbo-Croatian and Serbian as no language, from their current ISO 639-2 codes #138

Description

@malard

A stream tagged hrv - the ISO 639-2 code for Croatian, and what ffmpeg reports for a Matroska
track declaring Croatian - is read by Kodi as Serbo-Croatian. A stream tagged srp, the code
for Serbian, is read as no language at all.

Both are the current codes. Neither is exotic input.

The mechanism

CIso639::Alpha3ToAlpha2 maps a terminological code to its bibliographic form before looking the
language up, because the lookup table is keyed by the bibliographic code:

// Iso639.cpp
// The table is keyed by the bibliographic form, so a terminological code is mapped over first
const std::string bCode{CIso639_2::TCodeToBCode(code).value_or(code)};

const auto it = std::ranges::lower_bound(LanguageCodesByIso639_2b, bCode, {}, &ISO639::iso639_2b);

The T-to-B map still carries the two pairs ISO withdrew on 2008-06-28, and says so in its own
comments:

// Iso639_2_Table.h
{StringToLongCode("hrv"), StringToLongCode("scr")}, // 2008-06-28 scr was deprecated. The T code remains.
{StringToLongCode("srp"), StringToLongCode("scc")}, // 2008-06-28 scc was deprecated. The T code remains.

But the table it maps into was written the other way round - it holds the codes that survived:

// TableLanguageCodes.h
{"hr", "hrv"},
{"sh", "scr"},   // <- Serbo-Croatian, which is what hrv now lands on
{"sr", "srp"},

So:

tagged mapped to found read back as
hrv (Croatian) scr {"sh", "scr"} sh, Serbo-Croatian
srp (Serbian) scc nothing nothing

Neither code is in system/language-subtag-registry.txt either, so CBcp47::ParseTag cannot
answer first and the ISO fallback is the whole path.

The registry's own entry for what Croatian becomes says what is wrong with the result:

Subtag: sh
Description: Serbo-Croatian
Scope: macrolanguage
Comments: sr, hr, bs are preferred for most modern uses

What follows from it

  • A user whose audio language preference is Croatian never matches a Croatian track, because the
    track's language is not Croatian by the time anything compares it.
  • Croatian and Bosnian tracks tagged hrv and bos do not stay distinguishable from each other
    in the way the preference expects.
  • Serbian is worse: srp resolves to nothing, so ParseStreamLanguage logs it and stores und.
  • AsIso6392B() also emitted the withdrawn scr / scc on the way back out.

The fix

Delete both pairs from the T-to-B map. hrv and srp became both forms in 2008, so there is
nothing to map: with the entries gone, each is looked up as itself, finds its own row, and returns
hr / sr.

Nothing else uses those mappings - TCodeToBCode has exactly two callers, Alpha3ToAlpha2 and
CLanguageTag::AsIso6392B, and both are corrected by the same removal.

Pinned by TestIso639.ResolvesTheLanguagesWhoseBibliographicCodeWasWithdrawn.

How it surfaced

Writing the table test #36 asked for, over the RDS slow-labelling language table. The test asserts
every cell the standard assigns resolves to a language, and Serbian failed it. Croatian passed -
it resolved to something - which is the reason a test that only asks "did this parse" would not
have found it, and why the RDS test now pins the language each index names rather than only its
validity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: i18nLanguage codes, BCP 47, localisationtype: bugDefect in existing behaviour

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions