Why Internationalization is Hopelessly Broken in ASP.NET

I wrote an article last week describing ASP.NET's Internationalization (i18n) scheme in less than favorable terms, and it occurs to me that I should probably offer up a proper justification if I'm going to start throwing terms like 'Hopelessly Broken' around.

As several members of the ASP.NET community so eloquently pointed out in response to that article, ASP.NET does in fact offer a way to translate web sites from one language to another, and it does indeed work perfectly fine, thank you very much. That fact, I omitted to mention last week, is not in dispute and I apologize for implying as much.

To clarify, I don't mean to say that ASP.NET i18n is Hopelessly Broken to the point where it's not possible to do it, but rather that ASP.NET handles i18n in a fashion that is demonstrably worse than the accepted industry standard way of doing things which, incidentally, pre-dates ASP.NET.

Here's why.

First, let me give a quick rundown on the industry standard way of localizing websites: gettext. It's a set of tools from the GNU folks that can be used to translate text in computer programs. The ever-humble GNU crowd have a lot of documentation you can read about these tools explaining why they're so well suited for i18n and how they're a milestone in the history of computer science and incidentally how much smarter the GNU folks are than, say, you. And why you should be using emacs.

But anyway, to demonstrate why the gettext way of doing things makes so much more sense than the Microsoft way, let me run down a short list of the things you need to do to translate a website. For each task, I'll give an indication of how ASP.NET would have you do it, along with how you'd do it using hacky fixes I've put in place for the FairlyLocal library I discussed at length last week. Also, if there's a difference, I'll talk briefly about how "Everybody Else" (meaning gettext, which is in fact used by Everybody Else in the world to localize text) does it.

Identifying strings that should be marked for translation

ASP.NET: Find them by hand
FairlyLocal: Find them by hand
Everybody Else: Find them by hand, (unless you're using a language that supports the emacs gettext commands for finding text and wrapping them automatically)

Marking text for translation in code

ASP.NET: Ensure that they're wrapped in some form of runat="server" control
FairlyLocal: Wrap with _()
Everybody Else: Wrap with _()

ASP.NET actually does offer one advantage here, in that many of the text messages in need of translation will already be surrounded by a runat="server" control of some description. Unfortunately, that advantage is compensated for by the sheer amount of typing (or copy/pasting or Regex Replacing) involved in surrounding all the static text in your application with "<asp:literal runat="server"></asp:literal>", and by the computational overhead involved in instantiating Control objects for every one of those text fragments.

Everybody Else gets to suffer through the steady-state habit of surrounding all their text with _(""), or with a long copy/paste or Regex Replace session similar to the ASP.NET experience. It's still not all that much fun, but at least it's less typing.

Compiling a list of text fragments for use in translation

ASP.NET: Pull up each file in Design View, right click and select Create Local Resources
FairlyLocal: Build the project (thus running xgettext automatically)
Everybody Else: run xgettext

ASP.NET uses a proprietary XML file format called .resx, which is incomprehensible to humans in its raw form, but has an editor in Visual Studio.NET. Everybody Else uses .po files, which is a text format that's simple enough to be read and edited by non-technical translators, but there are also a variety of good standalone editors available.

Updating that list of text fragments as code changes

ASP.NET: Pull up each file in Design View (again), right click and select Create Local Resources (again)
FairlyLocal: Build the project (thus running xgettext automatically (again))
Everybody Else: run xgettext again

Specifying languages for translation:

ASP.NET: Copy the .resx file for each page on your site to a language-specific version, such as .es-ES.resx.
FairlyLocal and Everybody Else: create a language-specific folder under /locale and copy a single .po file there.

Surely there must be a tool to copy and rename the hundreds of locale-specific .resx files that ASP.NET needs for every single language, but I haven't found it yet. Please ASP.NET camp, point me in the right direction here so I don't need to go off on a rant about this one…

Translating strings from one language to another

ASP.NET: Translator opens the project in Visual Studio.NET (seriously!) so that he can use the .resx editor there to edit the cryptic XML files containing the text.
FairlyLocal & Everybody Else: Give your translator a .po file and have him edit it as text or with a 3rd party tool such as POedit

Identifying the language preference of the end user

Everybody: Automatically happens behind the scenes, but you can specify language preference too.

Referencing Translated Text (by using):

ASP.NET: Uniquely named Resource Keys
FairlyLocal: The text itself
Everybody Else: The text itself

When Visual Studio.NET does its magic, every runat="server" control will get a new attribute called meta:resourceKey containing a unique key with a helpful name such as "Literal26" or "HyperLink7" that is used to relate the text in the .resx file back to the control that uses it.

This is not actually as unhelpful as it seems, since translators will still see the Original Text in the .resx file alongside that meaningless key, so they will in fact know what text they're translating. Just not its context. Further, as ASP.NET developers we've learned to put up with a certain amount of VS.NET's autogenerated metagarbage, so we can generally gloss over these strange XML attributes that suddenly appear in our source.

Everybody else simply uses the text itself as the lookup key.

Displaying text to the end user in his preferred language

ASP.NET: Automagic. Can also ask for text directly from AppLocalResources
FairlyLocal: Automagic. Can also ask for translated text directly.
Everybody Else: Automagic. Can also ask for translated text directly.

In ASP.NET, you can add keys to your .resx file by hand if there are any messages you need that didn't get sniffed from the source. Other technologies don't need to bother with this step as often, since any text appearing in the source code will be marked for translation, whether it's associated with a control or not.

Wrapping Up

A short interlude...

I'm a believer in Sturgeon's Law, which states that "90% of everything is crap." Even ASP.NET, which I feel is still miles ahead of every other web development framework is not immune.

We've learned to avoid using pretty much all of the "Rich" controls and Designer Mode garbage that shipped with 1.1 and has plagued .NET ever since, and every new release brings a few things with it (including, alas, System.Globalization) that are best avoided.

In my opinion, that's fine, since the rest of the framework is so ridiculously productive. Don't worry though, any honest Django or Rails veteran will tell you that their frameworks also have bits that are best left alone. And hey, the most popular platform in the world for building web apps is 100% crap, so we're still miles ahead of the game here in the land of MS.

Anybody still following along will notice that while ASP.NET offers workable solutions to every stage of the i18n process, it's generally not quite as straightforward or convenient as the alternative way of doing things. ASP.NET also tends to pollute your codebase with a lot of extraneous noise in the form of meta:resourceKey attributes (why couldn't they have at least shortened that to "key" and made it part of the Control class so you could easily add it to anything) and .resx file collections for every single page in your site, and it leaves you a little short in the Tools department when it comes time to translate those files.

So while it's certainly possible to localize a website the way that ASP.NET recommends, it is definitely a lot of work, and it tends to be quite confusing. Doing it in another technology, say Django for instance, just doesn't seem like that big a deal. That's the sort of experience that I'm trying to bring to ASP.NET with the FairlyLocal library, and I hope it's at least a good first step.

If you have any suggestions (or better still, code contributions) to make it better, I look forward to hearing from you.

Fixing Internationalization in ASP.NET

I've been building websites with ASP.NET for a little over 10 years now, and I have a dirty little secret to confess: I've never Internationalized a single one of them.

It's not from lack of trying, I can tell you. I've got a good dozen false starts under my belt, and plenty of hours spent studying the code from other people's sites that implement Internationalization (abbreviated as i18n for us lazy typists) the way that Microsoft wants you to do it. And my conclusion is that it's just plain not worth the effort.

I18n is hopelessly broken in ASP.NET. Let's look at this nice snippet of sample code to see why:

<!-- STEP ONE, in MyPage.aspx: Create Runat="Server" Literal Control: --> <asp:Literal ID="lblPages" runat="server" meta:resourcekey="lblPagesResource1" Text="Pages"/> <!-- STEP TWO, in MyPage.es-ES.resx: Create Message Key/Value: --> <data name="lblPagesResource1.Text" xml:space="preserve"> <value>Browse</value> </data> ...and that's for EVERY piece of text in your whole site!

Notice that you need to make every single piece of localized text into a runat="server" control. And that you then need to add this crazy long attribute (that Intellisense doesn't know about, so you have to type out in full) to each one of those controls so that ASP.NET can find them in one of the Resource files that you need to generate by hand for every text fragment in your entire website.

If it sounds like a ridiculous amount of work for your developers, you're probably being charitable. In practice, it's so much extra work that nobody actually does it. That, my friends, is the reason you hardly ever see any multi-language websites written with ASP.NET.

Recently, however, my hand was truly forced. We're getting pretty close to launching FairTutor to the public, and since it has target audiences in both the United States and Latin America it pretty much needs to work in Spanish as well as English. This is the part where I start wistfully looking back to a couple Django projects we did not too long ago, and the absolute breeze it was localizing those sites. If only the rest of Django wasn't so crap, we could just port this project across and… Hang on a sec. Port. Yeah, how about we simply port that amazing Django i18n stuff over to ASP.NET instead.

That was a week ago.

Today, I'm releasing some code that I hope will single-handedly fix i18n in ASP.NET. It's based on the way that everybody else does it. Let's pause a minute to let that sink in, since many of my fellow .NET devs might not have been aware of this fact: There's another way of doing i18n, and it's so simple and straightforward that every other web framework uses it in some form or another to do multi-language websites.

In Django, PHP, Java, Rails, and pretty much everything else out there, you simply call a function called gettext() to localize text. Usually, you alias that function to _(), so you're looking at like 5 keystrokes (including quotes) to mark a piece of text for internationalization. That's simple enough that even lazy developers like me can be convinced to do it.

Better still, frameworks that use this gettext() library (it's actually a chunk of open source code from the GNU folks), also tend to come with a program that will sift through your source and automagically generate translation files for you (in .PO format, which is basic enough to be edited in notepad by non-tech-savvy translators, but is popular enough that there are several existing editors built just for it), containing every text fragment that was marked for i18n.

The whole process is so simple and straightforward that you're left to wonder why Microsoft felt compelled to spend so much time and effort reinventing it all to be worse.

Introducing FairlyLocal

I really want ASP.NET to stop forcing people to monkey with XML files and jump through hoops just to show web pages in Spanish, so I'm going to package up all this code and release it as Open Source:

FairlyLocal - Gettext Internationalization for ASP.NET

At the moment, there's not a whole lot to it. It'll find where you're using the FairlyLocal.GetText() (or its _() alias) and generate .PO files for you. And it'll suck in various language versions of those files and translate text on your website. Not much there, eh? But then that's the whole point: i18n is supposed to be simple and straightforward. Hopefully, FairlyLocal will make that an actuality for the ASP.NET community.

I look forward to hearing your feedback.

FairTutor is our latest project here at Expat. It's a website that connects Spanish teachers in South America with students in the US and lets them hold live Spanish classes online.

We'll be starting Beta classes soon, so if you want to score some free Spanish lessons, you might want to go sign up for the waiting list!

Sprinting

I didn't get anything done last week. Nothing at all. Most of my days were spent reading random stuff on the internet, making minor tweaks to Blogabond, and obsessing over traffic stats for Twiddla (why did 5000 people suddenly show up from StumbleUpon in one day???)

This week, on the other hand, I've been on fire. In the last 4 days, here's what I've accomplished:

• Built a Reddit clone from the ground up for Rootdown's soon-to-be-live Clinical Pearls section.
• Built a Google-Maps powered acupuncture chart, cut up 3000 tiles for it, incorporated effective Lat/Lon coordinates into Rootdown's database, and built a little Ajax data entry tool to drag & drop acupuncture points and meridians onto the chart.
• Reworked the Photo uploading and Photo management pieces of Blogabond.
• Tore out and streamlined the installation process for Regressor.NET
• Wrote this lame article

Trust me, that's a lot of stuff.

I've noticed this same pattern happening over and over again. I think of it as Sprinting, and I think I'm getting better at harnessing it. There are a few factors that play into it, but I think the key is knowing that I have an entire day to Sprint on whatever it is that I want to do. Knowing Absolutely that the door to my office won't open, the phone won't ring, and no little IM popups will bother me for the Next Twelve Hours. Knowing that I'm free to get as deep into what I'm doing as I need to get whatever I'm doing Done and Done for good. Those are the days that I get the most accomplished.

Sprinting for a different reason:
Team Expat, Running with the Bulls in 1998

Another thing that seems to help, at least for me, is to have more than one ball in the air at a time. When I've only got one project, I seem more content to move slowly, check my email, read the occasional blog, and essentially stuff my productivity. But when I've got 3 things the Need to Get Done, and there just aren't enough hours in the day to do it all, I find that I work a lot faster.

Better still is to have something ELSE that I should really be doing. You should SEE the days I spend blowing off paid work to Sprint on a side project.

But it more than just blowing off one project for another. The real advantage of having more than one project going at a time is that if I get blocked for any reason, I can switch over to another project and continue Sprinting at the same pace. If I'm motivated to move fast, it doesn't really matter all that much what I'm moving on, provided I'm moving. If I can ignore the little bottlenecks and keep Sprinting until the inspiration fades, I can get a lot more done overall. If I only had a single thing to work on, any minor distraction, such as missing graphics from a screen designer, could derail me and send me off to check Reddit (and thus get stuck there for six hours.)

I don't think that any of this is new. It's common knowledge that developers tend to work in bursts. I guess the difference for me is that I'm starting to work on ways to facilitate those bursts. To keep them going once they get started. To finally look up and find it's dark out, and I haven't eaten for 16 hours and wow, did I really get all that done in a single day??? That's where I want to be. That's Sprinting.