Implementing translation support

Implementing translation support

Postby Edward_Lii » 31 Mar 2012, 06:44

Hello all,

It's time to tackle the task of adding translation support to meandmyshadow.
There are three parts that need translation support.

Game
The first one is the most obvious one, the game itself.
We need to make everything translatable, from the console output to the window title.
I think the easiest solution for this is to use GNU gettext.
My only concern is that non-latin characters won't show probably with the fonts we use.

Levelpacks
With levelpacks a translation folder can be added in which translations files can be placed.
The translation files should contain the text from all the levels inside the pack but also the description (and name?).
The actual translating could still be done by gettext (or tinygettext, which STK uses).

Levels
I think this one is the hardest to do, but also the least important one.
The best way to do it is to threat it just like levelpack translations, but that would mean that a single level can now exist out of more than one file.
Adding the translations to the level file isn't such a good idea either, the md5sum would changes just because a translation is added/updated.

Another possible solution would be to do the game translation using (tiny)gettext and the level(pack)s using a custom system.
Still adding the translations to the level files but calculate the md5sum without it.

Any suggestions?
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: Implementing translation support

Postby odamite » 31 Mar 2012, 10:14

Hello Edward_Lii,

Yeah, I agree that that translating the game itself is will be easy: couple of macros and functions and "_(" and ")" around every string. But also you're right that our font doesn't support non-latin characters...So I made some mockups how the things could work:
finnish.png
Finnish

russian.png
Russian (probably doesn't make any sense because Google Translate :D )

The Russian one looks better that I would have thought. What do you think?

I don't actually how one can expand gettext functionality outside of code like our level files. Does tinygettext make this easier to accomplish?

I agree that one translation file per levelpack is good. But do we really need translations for single levels? I think that usually people will share their level which don't contain much text and people aren't able to translate levels by themselves. Levelpacks are bigger and might contain help or even a story, then possibility of translation must be there. And of course levelpacks which come with the game must have many translations.
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: Implementing translation support

Postby Edward_Lii » 31 Mar 2012, 10:26

Hello odamite,

odamite {l Wrote}:The Russian one looks better that I would have thought. What do you think?

It's not the same but I think the added value of the game in for example Russian is for Russian users more important than the font that is used.

odamite {l Wrote}:I don't actually how one can expand gettext functionality outside of code like our level files. Does tinygettext make this easier to accomplish?

I haven't looked that much in it, but the translating of levelpacks will be almost the same as for the game itself.
The strings will still be parsed by gettext _(message) but instead of looking the text in the default translation file it should fetch it from the levelpack translation file.

What tinygettext can do is instead of reading the .mo files (compiled .po files) it can directly use the .po files.
Meaning one step less in translating the game, beside that you can easily create new Dictionaries and using them.
So if a levelpack is loaded, the dictionary (.po file) can be parsed and used for the text of the levelpack.

One problem, though, tinygettext isn't in the openSUSE repository, not sure about Ubuntu or other distros.
We could include it (just like STK)?

odamite {l Wrote}:I agree that one translation file per levelpack is good. But do we really need translations for single levels? I think that usually people will share their level which don't contain much text and people aren't able to translate levels by themselves. Levelpacks are bigger and might contain help or even a story, then possibility of translation must be there. And of course levelpacks which come with the game must have many translations.

Yeah I think you're right, the text in levels is virtually negligible, if a it's a separate level it will most likely be a (hard) puzzle or big level.
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: Implementing translation support

Postby KroArtem » 31 Mar 2012, 10:38

Translating it into Russian won't be hard as I'm a native russian speaker :) The question is: what do you use for translating? Some web services like Launchpad, transifex, pootle or just .po files that can be translated locally on a computer?
KroArtem
 
Posts: 375
Joined: 26 Aug 2010, 19:04

Re: Implementing translation support

Postby Edward_Lii » 31 Mar 2012, 10:49

Hello KroArtem,

KroArtem {l Wrote}:Translating it into Russian won't be hard as I'm a native russian speaker :) The question is: what do you use for translating? Some web services like Launchpad, transifex, pootle or just .po files that can be translated locally on a computer?

We don't use anything yet, but we hope to implement it soon. ;)
It's probably going to be a .po file, about the difficulty of translating the problem primarily lies in the font that doesn't support Russian characters.

Anyway, it would be great if you could translate it into Russian when we've implemented translating.
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: Implementing translation support

Postby odamite » 31 Mar 2012, 11:35

Hello,

KroArtem {l Wrote}:Translating it into Russian won't be hard as I'm a native russian speaker :) The question is: what do you use for translating? Some web services like Launchpad, transifex, pootle or just .po files that can be translated locally on a computer?

Thanks! We need as many translators as possible. I'm glad you would like to help in translation.

I also think that it's a great idea that we could use some translation service. It isn't so smart to handle translations here in the forums or by email. It will also be easier for non-technical users to get in touch and help this project. I would suggest that we could use Launchpad or Transifex for the job. They both allow open source projects to use them for free. They make it's easy as seeing a sentence in English and then writing it in a textbox in one's own language. Also most importantly these services support gettext .po format.
Last edited by odamite on 31 Mar 2012, 14:06, edited 1 time in total.
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: Implementing translation support

Postby ctdabomb » 31 Mar 2012, 13:48

are you going to be translating the logo and title of the game or just one?
Some people are like slinkies... not really good for anything, but you still can't help smiling when you shove them down the stairs.
ctdabomb
 
Posts: 1075
Joined: 13 Dec 2011, 21:21
Location: halfway there

Re: Implementing translation support

Postby odamite » 31 Mar 2012, 14:14

Hello,

ctdabomb {l Wrote}:are you going to be translating the logo and title of the game or just one?

Generally I think that translating names of games is a bad idea. The name represents the whole game and its brand and having names translated might confuse people. Also I'm not used to translation of names and that's why they sound stupid to me. Ok, this is just my opinion and it might depend on language or culture. So I would like to hear your opinions. :)
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: Implementing translation support

Postby Knitter » 31 Mar 2012, 14:37

odamite {l Wrote}:I also think that it's a great idea that we could use some translation service. It isn't so smart to handle translations here in the forums or by email. It will also be easier for non-technical users to get in touch and help this project. I would suggest that we could use Launchpad or Transifex for the job. They both allow open source projects to use them for free. They make it's easy as seeing a sentence in English and then writing it in a textbox in one's own language. Also most importantly these services support gettext .po format.


I doubt that such a service will be useful in the beginning, and even then, it will probably cause more work with maintenance than provide any real usefulness. I do various translation works and simple .po/.pot files are easy enough that they can be managed in source control systems, send around to translators and tracked without much effort. And besides, how many strings are we talking about? The game seems simple enough that the translations will be mostly a one time job with only levels being added.

Thing is, there is not real need for a more complex system, if you end you needing something to manage translations than it is easy to start using some of those services, but I would suggest that you wait until you really feel the need to use such a system.

I can help with the Portuguese internationalization, and even managing translations if you need. I've got some experience with it.

I can also suggest PoEdit as a tool for anyone wanting to translate .po files, it's available in most operating systems and it's open source. Easy to use for any translator.

ctdabomb {l Wrote}:are you going to be translating the logo and title of the game or just one?

This will greatly depend on the country/culture. I know some countries like to translate everything, but most will want to retain the original name. I also agree that the name and log are an important part of the project's identity so I would, personally, not allow the translation of such elements.
Knitter
 
Posts: 237
Joined: 03 Jul 2011, 22:52
Location: Portugal

Re: Implementing translation support

Postby ctdabomb » 31 Mar 2012, 15:51

odamite {l Wrote}:Hello,

ctdabomb {l Wrote}:are you going to be translating the logo and title of the game or just one?

Generally I think that translating names of games is a bad idea. The name represents the whole game and its brand and having names translated might confuse people. Also I'm not used to translation of names and that's why they sound stupid to me. Ok, this is just my opinion and it might depend on language or culture. So I would like to hear your opinions. :)

knitter {l Wrote}:This will greatly depend on the country/culture. I know some countries like to translate everything, but most will want to retain the original name. I also agree that the name and log are an important part of the project's identity so I would, personally, not allow the translation of such elements.

okay, good, that is what I was hoping :)
Some people are like slinkies... not really good for anything, but you still can't help smiling when you shove them down the stairs.
ctdabomb
 
Posts: 1075
Joined: 13 Dec 2011, 21:21
Location: halfway there

Re: Implementing translation support

Postby acme_pjz » 31 Mar 2012, 15:53

Hi,

I'd like to help with Simplified Chinese translations :) As for the font, that is a problem... AFAIK WenQuanYi MicroHei and Droid Sans Fallback are two good open-source fonts containing Simplified Chinese characters...

Edward_Lii {l Wrote}:We need to make everything translatable, from the console output to the window title.


I'm not sure if we need to translate console output :|
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby Tedium » 31 Mar 2012, 16:33

I would be happy to help with the Dutch translation :).

I also think we shouldn't translate the title.
User avatar
Tedium
 
Posts: 122
Joined: 19 Oct 2011, 10:40

Re: Implementing translation support

Postby Edward_Lii » 31 Mar 2012, 16:55

Hello all,

Knitter {l Wrote}:I doubt that such a service will be useful in the beginning, and even then, it will probably cause more work with maintenance than provide any real usefulness. I do various translation works and simple .po/.pot files are easy enough that they can be managed in source control systems, send around to translators and tracked without much effort. And besides, how many strings are we talking about? The game seems simple enough that the translations will be mostly a one time job with only levels being added.

Thing is, there is not real need for a more complex system, if you end you needing something to manage translations than it is easy to start using some of those services, but I would suggest that you wait until you really feel the need to use such a system.

I agree with you, we can just maintain the .pot file in the svn repository and let the translators make/maintain the .po files.
One thing what we needs to do when this is implemented is to have a string freeze to give translators the chance to translate the new/updated string before the release.

odamite {l Wrote}:Hello,

ctdabomb {l Wrote}:are you going to be translating the logo and title of the game or just one?

Generally I think that translating names of games is a bad idea. The name represents the whole game and its brand and having names translated might confuse people. Also I'm not used to translation of names and that's why they sound stupid to me. Ok, this is just my opinion and it might depend on language or culture. So I would like to hear your opinions. :)

Personally I think we should keep the name/logo the same among different languages.
IIRC I saw "yo y mi sombra" in a Spanish review. :)
I think one problem with translating the name is that it will be harder to find the game's site when searching on a translated title.

acme_pjz {l Wrote}:I'm not sure if we need to translate console output :|

Well, it might help people if they can read the output and perhaps know what the problem is.
But on the other hand we might not help directly if we need to reverse translate the output back to english. :think:

Knitter {l Wrote}:I can help with the Portuguese internationalization, and even managing translations if you need. I've got some experience with it.

acme_pjz {l Wrote}:I'd like to help with Simplified Chinese translations :)

Tedium {l Wrote}:I would be happy to help with the Dutch translation :).

The more translations, the better. :D
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: Implementing translation support

Postby odamite » 01 Apr 2012, 16:38

Hello everyone!

Good news, I've started implementing translations system for the game! Currently it's possible to translate strings that are in the code which means there isn't yet support for levelpack translations. But this is a very good start. :D

So it's in the newest svn (r.318) and I've included an example translation for Finnish (BTW I can be the translator for Finnish). There is a translation system but DO NOT start translating anything yet. It's still under development and all strings aren't even there. I think that a proper time for translation will be when 0.4 is near to its release. Maybe between RC and final? Also there is support for just our two fonts which supports just latin-derived alphabet. This can be fixed by using some other font for different languages (for example FreeSans should support wide-range of writing systems) But it's nice to see that there are many people willing to help with translations. Thanks you!

Because of all this the game is now using 3 extra libraries (tinygettext, tinyformat and findlocale). These aren't big and that's why they are included right into the source code (and their license should allow this). All these libraries will of course impact the performance of the game but not much and I think it's worth it. User can change the language anything he/she wants from config file (lang). But if not, findlocale will step in and guess it for the user.
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: Implementing translation support

Postby Edward_Lii » 04 Apr 2012, 17:29

Hello all,

More good news, levelpacks can now be translated as well.
I've added a .pot file for the tutorial and the default levelpack, everything can be translated except the title of the levelpack.

A python script is in the latest svn (r. 323) and can be used to extract translatable strings from a levelpack.
It isn't perfect since it doesn't parse the file but tries to detect a key=value line by line.
If it finds one it will check the key against a list with keys that are translatable.
This means that the same text twice won't work properly.
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: Implementing translation support

Postby acme_pjz » 05 Apr 2012, 11:15

Hi Edward_Lii,

Unfortulately the code can't compile under Windows :( two errors: Can't find iconv.h and dirent.h under Windows...

I looked at STK's tinygettext, they disabled iconv.hpp completely, so it no longer uses iconv.h, and they implements stk_file_system instead of unix_file_system, so they doesn't used opendir function. I think we should avoid using opendir function, instead write a function in Functions.cpp whick takes care platform-dependent things...
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby acme_pjz » 05 Apr 2012, 11:40

And static inline const char* _(const std::string& message) doesn't work under MSVC compiler; when this function returns, the pointer get freed :| and the program crashes...
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby acme_pjz » 05 Apr 2012, 11:49

After checking SuperTuxKart code there is a conclusion: don't define _ as a function, it should be a macro :| ...
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby acme_pjz » 05 Apr 2012, 13:17

Finally I commited the bug fix; but I'm not sure if we should remove iconv.hpp like SuperTuxKart :|
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby odamite » 05 Apr 2012, 14:14

Hello acme_pjz,

Thank you for making the game more cross-platform and language. :D Great work! You're right that using droidsans is much better that freesans. It seems to support better other languages. It was my fault, sorry. :cry:

While compiling on Linux with gcc it gives me an error. This can be easily fixed by adding #include <iostream> in unix_file_system.cpp. I'm going to commit something soon and so I can fix it.
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: Implementing translation support

Postby acme_pjz » 13 Apr 2012, 05:52

Hi Edward_Lii,

I forget to tell you that the .pot file of levelpack is buggy, some messages are duplicated so Poedit refused to open it :|
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: Implementing translation support

Postby Edward_Lii » 13 Apr 2012, 13:29

Hello acme_pjz,

acme_pjz {l Wrote}:I forget to tell you that the .pot file of levelpack is buggy, some messages are duplicated so Poedit refused to open it :|

For me poedit opened it fine, perhaps a different version.
Anyway I fixed this bug in svn rev. 337 and updated the messages.pot files of the levelpacks.

It now handles duplicate strings properly and only extracts translatable strings from the current directory.
This is to prevent from extracting strings from the .svn directory.
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Who is online

Users browsing this forum: No registered users and 1 guest