The Turkish I Problem: How Case Conversion Broke in Five of My Projects

Anyone who builds software for Turkish users runs into the same bug sooner or later: case conversion. In English, I and i are one pair. Turkish has two pairs: I–ı (dotless) and İ–i (dotted). Most programming languages convert case with English rules by default, and some use the language of the machine they run on. Both choices break in the wrong place. This post is about the Turkish I problem: how I hit it in five of my projects, in five different shapes, and how I fixed each one in Java, JavaScript, Python and C#.

The problem in one table

Conversion	English rule	Turkish rule
`I` to lower case	`i`	`ı`
`i` to upper case	`I`	`İ`
`İ` to lower case	`i` + combining dot (two characters)	`i`

Any code that lowercases a string and compares it with another one can trip over this table: search, category matching, email addresses, file extensions, protocol keywords. The nasty part is that English test data never shows it.

1. Java: the "INBOX" folder became "ınbox"

My email client shows an icon for each folder: inbox, sent, trash. An IMAP server names the inbox INBOX. The code lowercased the folder name and looked for words like "inbox", "sent" and "trash".

In Java, toLowerCase() without an argument uses the machine's default locale. On a Turkish Windows install that is Turkish, and "INBOX".toLowerCase() returns "ınbox". Because of the dotless ı, nothing matched and the inbox had no icon. The code that picks server settings from an email address had the same problem: for someone who typed their address in capitals, GMAIL.COM became gmaıl.com and the Gmail preset was never found.

The fix is to ask for the locale-independent rule explicitly for technical strings:

// Locale.ROOT: with Turkish rules "INBOX" -> "ınbox" (dotless ı)
final String name = fullName.toLowerCase(Locale.ROOT);

I made it a rule: Locale.ROOT for machine text such as folder names, domain names and protocol words; Turkish rules for Turkish text shown to people.

2. JavaScript: the wrong letter in Hangman

In my Hangman game, players can also type letters on the keyboard. In the Turkish game, a player who presses I means ı. JavaScript's toLowerCase() ignores language entirely and always turns I into i. Had I used it, a player hunting for the word "KIŞ" (winter) would press I, the game would check for i, and the player would lose a life for a wrong letter. Having been bitten by this in the email client, I used toLocaleLowerCase with an explicit language from the start:

export function normalizeLetter(input, lang) {
  if (typeof input !== 'string' || [...input].length !== 1) return null;
  const letter = input.toLocaleLowerCase(lang);
  return ALPHABETS[lang].includes(letter) ? letter : null;
}

The game can also be played in English, so the language is not fixed; it is the game's current language. The capital letters in the letter boxes go through toLocaleUpperCase(lang) for the same reason, otherwise the i in a Turkish word would show up as I. The tests check both languages:

assert.equal(normalizeLetter('I', 'tr'), 'ı');
assert.equal(normalizeLetter('İ', 'tr'), 'i');
assert.equal(normalizeLetter('I', 'en'), 'i');

3. JavaScript: typing "kadikoy" did not find Kadıköy

City search in my weather app uses two services. Open-Meteo's geocoding is fast for world cities, but it did not handle the Turkish "ı" and could not find Kadıköy, a district of Istanbul. So I added Photon, which is built on OpenStreetMap, for Turkish provinces and districts.

I had to compare the results from both services with what the user typed. On a phone, someone may type "kadikoy" or "KADIKÖY". So I wrote a function that folds text into one comparable form:

export function foldText(value) {
  return value
    .toLocaleLowerCase('tr')
    .normalize('NFD') // ş → s + cedilla; the marks are removed on the next line
    .replace(/[̀-ͯ]/g, '')
    .replace(/ı/g, 'i');
}

The order matters. Lowercasing with Turkish rules first gets I → ı and İ → i right. Then Unicode decomposition (NFD) splits letters such as ş, ö and ü into a base letter plus a mark, and the marks are removed. Finally ı becomes i, because ı does not decompose: it is a letter of its own. The test:

assert.equal(foldText('ŞANLIURFA'), 'sanliurfa');
assert.equal(foldText('ısparta'), foldText('Isparta'));
assert.equal(foldText('Kadıköy'), 'kadikoy');

4. Python: the name "İrem" became "Irem"

My KVKK Document Assistant masks people's names in a document before a language model ever sees it, and puts fake names in their place. While looking names up in a dictionary and writing the fake ones, I ran into two Python behaviours:

>>> "İ".lower()
'i̇'          # i + combining dot (U+0307), two characters
>>> "irem".capitalize()
'Irem'

The first broke matching an all-caps "İREM" against "irem" in the dictionary: the lowercased text looks the same but carries an extra invisible character. The second wrote "Irem" instead of "İrem" in the fake data and in email text.

Python's standard library has no language-aware case conversion, so I wrote two small functions:

def tr_lower(word: str) -> str:
    return word.replace("I", "ı").replace("İ", "i").lower()

def tr_capitalize(word: str) -> str:
    if not word:
        return word
    first = {"i": "İ", "ı": "I"}.get(word[0], word[0].upper())
    return first + tr_lower(word[1:])

In this project a miss was expensive: a name that failed to match could reach the model unmasked. So the tests generate every name in lower case, upper case, with Turkish suffixes and without Turkish letters, and check that none of them gets through unmasked.

5. C#: searching "ip13" did not find "IP13"

The stock page in Tamirhane searches by part code or name. Part names are Turkish ("Şarj soketi", "iPhone 13 ekran paneli") while codes use English letters (IP13-SCR). The first version searched case-insensitively with the Turkish culture. Typing "şarj" found "Şarj soketi", but typing "ip13" did not find IP13-SCR: under Turkish rules the capital of i is İ, so ip13 and IP13 are not the same word.

Switching to the invariant culture was no fix either; then "ince" would not find "İnce kablo". Real data has both kinds of text side by side, so a match under either rule counts:

// In Turkish the capital of "i" is "İ": Turkish rules alone would not find "IP13-SCR" for "ip13".
// Codes use English letters, names are Turkish; a match under either rule counts.
private static bool Contains(string text, string query) =>
    Turkish.IndexOf(text, query, CompareOptions.IgnoreCase) >= 0
    || CultureInfo.InvariantCulture.CompareInfo.IndexOf(text, query, CompareOptions.IgnoreCase) >= 0;

The test covers both kinds together:

[InlineData("ip13", "iPhone 13 ekran paneli")] // Turkish rules: "i" ≠ "I"; codes are still found
[InlineData("şarj", "Şarj soketi USB-C")]
[InlineData("ŞARJ", "Şarj soketi USB-C")]

What I took away

All five bugs come down to one question: whose text is this?

Machine text (folder names, domains, file extensions, part codes, protocol words): the locale-independent rule. Locale.ROOT in Java, InvariantCulture in C#, plain toLowerCase() in JavaScript.
Human text (names, words, cities): the rule of the text's language, passed explicitly. Locale.forLanguageTag("tr") in Java, toLocaleLowerCase('tr') in JavaScript, tr-TR in C#, a hand-written function in Python.
Search over both: either check both rules, or fold the text into one form without diacritics.

And one testing rule: every test for a function that changes case should contain at least one I, one İ and one ı. Those three letters catch, on the first run, the bugs English test data never sees.