Localization Using .po

Mar 11, 2017

Preface

Translating a game into multiple languages is a very effective way to expose it to new audiences. However, it can turn into a real headache if you don't architect it in a maintainable and expandable way. A .po system is a great way to manage translations, and is a useful system to know for more than just games. I will go into ways you can expand the system to be more robust, but most of the details of those are too dependent on your setup to be fully detailed in a tutorial.

Something to keep in mind when deciding to translate is that your game should already be in a pretty mature state, as getting translations done (especially by fans) is a hard thing to get them to update every time you change or add some text.

A few things you should already have set up going into this are some sort of settings system as well as a UI system and a global DontDestroyOnLoad object that your UI has access to.

Parts of this is based off of this, but I have expanded on it greatly.

I will most likely update this at some point, as even writing this I noticed some optimizations I could be making.

Getting Translations

Skip this section if you just want to look at the workflow, but I found this system useful for getting fans to translate things for me. The best thing to do is make a Google Drive spreadsheet and share the link with fans, who you can find by making posts at the various hubs for your game. Below is a link to my sample spreadsheet that you can save onto your own Drive and use.

Drive Spreadsheet Template

The .po File

What is a .po (Portable Object) file? It's a part of the gettext standard, and is described by the GNU project like this (link):

A PO file is made up of many entries, each entry holding the relation between an original untranslated string and its corresponding translation. All entries in a given PO file usually pertain to a single project, and all translations are expressed in a single target language. One PO file entry has the following schematic structure:

white-space
#  translator-comments
#. extracted-comments
#: referenceā€¦
#, flagā€¦
#| msgid previous-untranslated-string
msgid untranslated-string
msgstr translated-string

So what? It's just a file with a key and a value and some meta data? Not quite - there are two more parts to using a .po system that make it really shine. The .pot files, and the editors.

A .pot (Portable Object Template) is what is used to generate your .po files in an editor, and can be used to update your .po files when you add new translations. It has the exact same structure as a normal .po file, but with an empty translated-string.

Do not try to manage translations by adding entries to .po files, update from .pot to ensure consistency!

The most popular editor is Poedit. Which looks like this, I have circled the 'Update from POT' button:

poedit

Overview of Unity Implementation

This system will probably need some tweaking to your exact situation, but I'm going to try to keep it as generic as possible. Though some of the steps you are going to have to figure out where to call some things on your own. We are going to be using the English strings as our keys - this is very important! Keep your .pot updated. The idea is that since the game will start in English every time we can just grab the default values for the text box and use them as both keys and the English language values.

The main files we will be working with are:
template.pot - Stored in Resources/Languages (create if doesn't exist)
spanish.po - Stored in Resources/Languages (create if doesn't exist)
LanguageManager.cs - Attached and referenced by global persistent game manager
UILocalizeText.cs - Attached to each UI text object controls updating that text

You must attach UILocalizeText.cs to every text box you want to translate, and have a value for it in the .po and .pot files.

TL;DR for pros: Make LanguageManger on a global don't destroy on load object, call LoadLanguage() when you switch languages and then UpdateAllTextBoxes() when you need to update your UI.

Note: Since I updated Unity, it seems to not recognize .po files as text files and failing to find the translation files. Adding .txt after .po fixes it (ie. spanish.po.txt), but to load it to Poedit you must type the name in the open file dialog.*

The Scripts

In the following scripts everything that says YourGameManager should be replaced with your systems. I have tried to comment the files to be pretty descriptive.

To set up and use these:
1 - Create three text boxes with values that correspond to the values in template.pot
2 - Attach UILocalizeText.cs to these text boxes
3 - Init LevelManager.cs
4 - LoadLanguage("spanish")
5 - UpdateAllTextBoxes()

Grab a zip here!

template.pot

#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: Test\n"
"POT-Creation-Date: 2017-02-12 09:12-0500\n"
"PO-Revision-Date: 2015-10-27 19:11-0400\n"
"Last-Translator: \n"
"Language-Team: \n"
"Language: en\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Poedit 1.8.11\n"
"X-Poedit-Basepath: .\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"X-Poedit-KeywordsList: --keyword[-]\n"
"X-Poedit-SearchPath-0: .\n"

msgid "GOLD"
msgstr ""

msgid "SILVER"
msgstr ""

msgid "BRONZE"
msgstr ""

spanish.po.txt

#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: Test\n"
"POT-Creation-Date: 2017-02-12 09:12-0500\n"
"PO-Revision-Date: 2015-10-27 19:11-0400\n"
"Last-Translator: \n"
"Language-Team: \n"
"Language: en\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Poedit 1.8.11\n"
"X-Poedit-Basepath: .\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"X-Poedit-KeywordsList: --keyword[-]\n"
"X-Poedit-SearchPath-0: .\n"

msgid "GOLD"
msgstr "ORO"

msgid "SILVER"
msgstr "PLATA"

msgid "BRONZE"
msgstr "BRONCE"

LanguageManager.cs

using UnityEngine;
using System.Collections;
using System.IO;
using System.Collections.Generic;
using System;

public class LanguageManager : MonoBehaviour
{
    //This is where the current loaded language will go
    private static Hashtable textTable;

    [HideInInspector]
    //Just a reference for the current language, default to english
    public string CurrentLanguage = "english";

    //Run this when you are ready to start the language process, you usually want to do this after everything has loaded
    public void Init()
    {
        //You should use an enum for storing settings that are a unique list,
        //This gets a string representation of an enum, 
        CurrentLanguage = Enum.GetName(typeof(Settings.Languages), (int)YourGameManager.Settings.Language);

        //Pass that language into LoadLanguage, remember we are in init so this should only run once.
        LoadLanguage(CurrentLanguage);
    }

    //You call this when you want to update all text boxes with the new translation.
    //Run this after Init
    //Run this whenever you run LoadLanguage
    //Run this whenever you load a new scene and want to translate the new UI
    public void UpdateAllTextBoxes()
    {      
        //Find all active and inactive text boxes and loop through 'em
        UILocalizeText[] temp = Resources.FindObjectsOfTypeAll<UILocalizeText>();
        foreach (UILocalizeText text_box in temp)
        {
            //Run the update translation function on each text 
            text_box.UpdateTranslation();
        }
    }

    //Run this whenever a language changes, like in when a setting is changed - then run UpdateAllTextBoxes
    //This is based off of http://wiki.unity3d.com/index.php?title=TextManager, though heavily modified and expanded
    public void LoadLanguage(string lang)
    {
        CurrentLanguage = lang;

        if(lang == "english")
        {
            UpdateAllTextBoxes();
        }
        else if (lang != "english")
        {
            string fullpath = "Languages/" + lang + ".po"; // the file is actually ".txt" in the end

            TextAsset textAsset = (TextAsset)Resources.Load(fullpath);
            if (textAsset == null)
            {
                Debug.Log("[TextManager] " + fullpath + " file not found.");
                return;
            } else
            {
                Debug.Log("[TextManager] loading: " + fullpath);

                if (textTable == null)
                {
                    textTable = new Hashtable();
                }

                textTable.Clear();

                StringReader reader = new StringReader(textAsset.text);
                string key = null;
                string val = null;
                string line;
                while ((line = reader.ReadLine()) != null)
                {
                    if (line.StartsWith("msgid \""))
                    {
                        key = line.Substring(7, line.Length - 8).ToUpper();
                    }
                    else if (line.StartsWith("msgstr \""))
                    {
                        val = line.Substring(8, line.Length - 9);
                    }
                    else
                    {
                        if (key != null && val != null)
                        {
                            // TODO: add error handling here in case of duplicate keys
                            textTable.Add(key, val);
                            key = val = null;
                        }
                    }
                }

                reader.Close();
            }
        }
    }

    //This handles selecting the value from the translation array and returning it, the UILocalizeText calls this
    public string GetText(string key)
    {
        string result = "";
        if (key != null && textTable != null)
        {
            if (textTable.ContainsKey(key))
            {
                result = (string)textTable[key];

            } else
            {

            }
        }
        return (string)result;
    }
}

UILocalizeText.cs

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class UILocalizeText : MonoBehaviour {

    //This instances key, as well as the english translation
    [HideInInspector]
    public string TranslationKey = "";

    [HideInInspector]
    public Text TextToTranslate;

    //References to text values
    string OriginalText = "";
    string TranslatedText = "";

    //This gets run automatically if the original text hasn't been set when you go to update it. 
    //You shouldn't need to manually run this from anywhere
    public void Init()
    {
        //Grab the TextToTranslate if we haven't
        if(TextToTranslate == null)
            TextToTranslate = GetComponent<Text>();

        //Grab the original value of the text before we update it
        if(TextToTranslate != null)
            OriginalText = TextToTranslate.text;

        //Set the translation key to the original english text
        if(TranslationKey == "")
            TranslationKey = OriginalText;
    }

    //This gets called from LanguageManager
    //One thing I noticed is that it might be nicer to just pass in the correct string to this rather than go grap it from LanguageManager
    public void UpdateTranslation()
    {
        //If original text is empty, then this object hasn't been initiated so it should do that 
        if (OriginalText == "")
            Init();

        //If the object has no Text object then we shouldn't try to set the text so just stop
        if (TextToTranslate == null) return;

        if(YourGameManager.LanguageManager.CurrentLanguage != "english" && TranslationKey != "")
        {
            string new_text = YourGameManager.LanguageManager.GetText(TranslationKey.ToUpper());

            if (new_text != "")
            {
                TextToTranslate.text = new_text;
            } else
            {
               // Debug.Log("Key " + OriginalText + " doesn't have an entry in this language");
            }
        } else if(YourGameManager.LanguageManager.CurrentLanguage == "english")
        {
            if(TextToTranslate != null && OriginalText != null)
                TextToTranslate.text = OriginalText;
        }

    }

}