Why Internationalization is Hopelessly Broken in ASP.NET

I wrote an article last week describing ASP.NET's Internationalization (i18n) scheme in less than favorable terms, and it occurs to me that I should probably offer up a proper justification if I'm going to start throwing terms like 'Hopelessly Broken' around.

As several members of the ASP.NET community so eloquently pointed out in response to that article, ASP.NET does in fact offer a way to translate web sites from one language to another, and it does indeed work perfectly fine, thank you very much. That fact, I omitted to mention last week, is not in dispute and I apologize for implying as much.

To clarify, I don't mean to say that ASP.NET i18n is Hopelessly Broken to the point where it's not possible to do it, but rather that ASP.NET handles i18n in a fashion that is demonstrably worse than the accepted industry standard way of doing things which, incidentally, pre-dates ASP.NET.

Here's why.

First, let me give a quick rundown on the industry standard way of localizing websites: gettext. It's a set of tools from the GNU folks that can be used to translate text in computer programs. The ever-humble GNU crowd have a lot of documentation you can read about these tools explaining why they're so well suited for i18n and how they're a milestone in the history of computer science and incidentally how much smarter the GNU folks are than, say, you. And why you should be using emacs.

But anyway, to demonstrate why the gettext way of doing things makes so much more sense than the Microsoft way, let me run down a short list of the things you need to do to translate a website. For each task, I'll give an indication of how ASP.NET would have you do it, along with how you'd do it using hacky fixes I've put in place for the FairlyLocal library I discussed at length last week. Also, if there's a difference, I'll talk briefly about how "Everybody Else" (meaning gettext, which is in fact used by Everybody Else in the world to localize text) does it.

Identifying strings that should be marked for translation

ASP.NET: Find them by hand
FairlyLocal: Find them by hand
Everybody Else: Find them by hand, (unless you're using a language that supports the emacs gettext commands for finding text and wrapping them automatically)

Marking text for translation in code

ASP.NET: Ensure that they're wrapped in some form of runat="server" control
FairlyLocal: Wrap with _()
Everybody Else: Wrap with _()

ASP.NET actually does offer one advantage here, in that many of the text messages in need of translation will already be surrounded by a runat="server" control of some description. Unfortunately, that advantage is compensated for by the sheer amount of typing (or copy/pasting or Regex Replacing) involved in surrounding all the static text in your application with "<asp:literal runat="server"></asp:literal>", and by the computational overhead involved in instantiating Control objects for every one of those text fragments.

Everybody Else gets to suffer through the steady-state habit of surrounding all their text with _(""), or with a long copy/paste or Regex Replace session similar to the ASP.NET experience. It's still not all that much fun, but at least it's less typing.

Compiling a list of text fragments for use in translation

ASP.NET: Pull up each file in Design View, right click and select Create Local Resources
FairlyLocal: Build the project (thus running xgettext automatically)
Everybody Else: run xgettext

ASP.NET uses a proprietary XML file format called .resx, which is incomprehensible to humans in its raw form, but has an editor in Visual Studio.NET. Everybody Else uses .po files, which is a text format that's simple enough to be read and edited by non-technical translators, but there are also a variety of good standalone editors available.

Updating that list of text fragments as code changes

ASP.NET: Pull up each file in Design View (again), right click and select Create Local Resources (again)
FairlyLocal: Build the project (thus running xgettext automatically (again))
Everybody Else: run xgettext again

Specifying languages for translation:

ASP.NET: Copy the .resx file for each page on your site to a language-specific version, such as .es-ES.resx.
FairlyLocal and Everybody Else: create a language-specific folder under /locale and copy a single .po file there.

Surely there must be a tool to copy and rename the hundreds of locale-specific .resx files that ASP.NET needs for every single language, but I haven't found it yet. Please ASP.NET camp, point me in the right direction here so I don't need to go off on a rant about this one…

Translating strings from one language to another

ASP.NET: Translator opens the project in Visual Studio.NET (seriously!) so that he can use the .resx editor there to edit the cryptic XML files containing the text.
FairlyLocal & Everybody Else: Give your translator a .po file and have him edit it as text or with a 3rd party tool such as POedit

Identifying the language preference of the end user

Everybody: Automatically happens behind the scenes, but you can specify language preference too.

Referencing Translated Text (by using):

ASP.NET: Uniquely named Resource Keys
FairlyLocal: The text itself
Everybody Else: The text itself

When Visual Studio.NET does its magic, every runat="server" control will get a new attribute called meta:resourceKey containing a unique key with a helpful name such as "Literal26" or "HyperLink7" that is used to relate the text in the .resx file back to the control that uses it.

This is not actually as unhelpful as it seems, since translators will still see the Original Text in the .resx file alongside that meaningless key, so they will in fact know what text they're translating. Just not its context. Further, as ASP.NET developers we've learned to put up with a certain amount of VS.NET's autogenerated metagarbage, so we can generally gloss over these strange XML attributes that suddenly appear in our source.

Everybody else simply uses the text itself as the lookup key.

Displaying text to the end user in his preferred language

ASP.NET: Automagic. Can also ask for text directly from AppLocalResources
FairlyLocal: Automagic. Can also ask for translated text directly.
Everybody Else: Automagic. Can also ask for translated text directly.

In ASP.NET, you can add keys to your .resx file by hand if there are any messages you need that didn't get sniffed from the source. Other technologies don't need to bother with this step as often, since any text appearing in the source code will be marked for translation, whether it's associated with a control or not.

Wrapping Up

A short interlude...

I'm a believer in Sturgeon's Law, which states that "90% of everything is crap." Even ASP.NET, which I feel is still miles ahead of every other web development framework is not immune.

We've learned to avoid using pretty much all of the "Rich" controls and Designer Mode garbage that shipped with 1.1 and has plagued .NET ever since, and every new release brings a few things with it (including, alas, System.Globalization) that are best avoided.

In my opinion, that's fine, since the rest of the framework is so ridiculously productive. Don't worry though, any honest Django or Rails veteran will tell you that their frameworks also have bits that are best left alone. And hey, the most popular platform in the world for building web apps is 100% crap, so we're still miles ahead of the game here in the land of MS.

Anybody still following along will notice that while ASP.NET offers workable solutions to every stage of the i18n process, it's generally not quite as straightforward or convenient as the alternative way of doing things. ASP.NET also tends to pollute your codebase with a lot of extraneous noise in the form of meta:resourceKey attributes (why couldn't they have at least shortened that to "key" and made it part of the Control class so you could easily add it to anything) and .resx file collections for every single page in your site, and it leaves you a little short in the Tools department when it comes time to translate those files.

So while it's certainly possible to localize a website the way that ASP.NET recommends, it is definitely a lot of work, and it tends to be quite confusing. Doing it in another technology, say Django for instance, just doesn't seem like that big a deal. That's the sort of experience that I'm trying to bring to ASP.NET with the FairlyLocal library, and I hope it's at least a good first step.

If you have any suggestions (or better still, code contributions) to make it better, I look forward to hearing from you.

Fixing Internationalization in ASP.NET

I've been building websites with ASP.NET for a little over 10 years now, and I have a dirty little secret to confess: I've never Internationalized a single one of them.

It's not from lack of trying, I can tell you. I've got a good dozen false starts under my belt, and plenty of hours spent studying the code from other people's sites that implement Internationalization (abbreviated as i18n for us lazy typists) the way that Microsoft wants you to do it. And my conclusion is that it's just plain not worth the effort.

I18n is hopelessly broken in ASP.NET. Let's look at this nice snippet of sample code to see why:

<!-- STEP ONE, in MyPage.aspx: Create Runat="Server" Literal Control: --> <asp:Literal ID="lblPages" runat="server" meta:resourcekey="lblPagesResource1" Text="Pages"/> <!-- STEP TWO, in MyPage.es-ES.resx: Create Message Key/Value: --> <data name="lblPagesResource1.Text" xml:space="preserve"> <value>Browse</value> </data> ...and that's for EVERY piece of text in your whole site!

Notice that you need to make every single piece of localized text into a runat="server" control. And that you then need to add this crazy long attribute (that Intellisense doesn't know about, so you have to type out in full) to each one of those controls so that ASP.NET can find them in one of the Resource files that you need to generate by hand for every text fragment in your entire website.

If it sounds like a ridiculous amount of work for your developers, you're probably being charitable. In practice, it's so much extra work that nobody actually does it. That, my friends, is the reason you hardly ever see any multi-language websites written with ASP.NET.

Recently, however, my hand was truly forced. We're getting pretty close to launching FairTutor to the public, and since it has target audiences in both the United States and Latin America it pretty much needs to work in Spanish as well as English. This is the part where I start wistfully looking back to a couple Django projects we did not too long ago, and the absolute breeze it was localizing those sites. If only the rest of Django wasn't so crap, we could just port this project across and… Hang on a sec. Port. Yeah, how about we simply port that amazing Django i18n stuff over to ASP.NET instead.

That was a week ago.

Today, I'm releasing some code that I hope will single-handedly fix i18n in ASP.NET. It's based on the way that everybody else does it. Let's pause a minute to let that sink in, since many of my fellow .NET devs might not have been aware of this fact: There's another way of doing i18n, and it's so simple and straightforward that every other web framework uses it in some form or another to do multi-language websites.

In Django, PHP, Java, Rails, and pretty much everything else out there, you simply call a function called gettext() to localize text. Usually, you alias that function to _(), so you're looking at like 5 keystrokes (including quotes) to mark a piece of text for internationalization. That's simple enough that even lazy developers like me can be convinced to do it.

Better still, frameworks that use this gettext() library (it's actually a chunk of open source code from the GNU folks), also tend to come with a program that will sift through your source and automagically generate translation files for you (in .PO format, which is basic enough to be edited in notepad by non-tech-savvy translators, but is popular enough that there are several existing editors built just for it), containing every text fragment that was marked for i18n.

The whole process is so simple and straightforward that you're left to wonder why Microsoft felt compelled to spend so much time and effort reinventing it all to be worse.

Introducing FairlyLocal

I really want ASP.NET to stop forcing people to monkey with XML files and jump through hoops just to show web pages in Spanish, so I'm going to package up all this code and release it as Open Source:

FairlyLocal - Gettext Internationalization for ASP.NET

At the moment, there's not a whole lot to it. It'll find where you're using the FairlyLocal.GetText() (or its _() alias) and generate .PO files for you. And it'll suck in various language versions of those files and translate text on your website. Not much there, eh? But then that's the whole point: i18n is supposed to be simple and straightforward. Hopefully, FairlyLocal will make that an actuality for the ASP.NET community.

I look forward to hearing your feedback.

FairTutor is our latest project here at Expat. It's a website that connects Spanish teachers in South America with students in the US and lets them hold live Spanish classes online.

We'll be starting Beta classes soon, so if you want to score some free Spanish lessons, you might want to go sign up for the waiting list!

How close to Zero Friction is your signup process?

StackOverflow.com just launched this last week, and it looks pretty cool. It seems like it might be our best shot at getting back to the sort of useful discussion that we used to have on the Usenet back in the 90's. Lots of signal, hardly any noise, and even the occasional correct answer. Sign me up!

Uh... wait a sec... I can't sign up.

StackOverflow has made the inexplicable blunder of requiring its users to sign in via OpenID. That means you can't simply pick a username and password, but must instead go away and find yourself an OpenID provider, sign up for that, and bring it back to StackOverflow. It's like 14 steps, depending on which provider you choose. Observe:

• Click login
• Read a ton of instructions
• Locate and click the "get one" link
• Dismiss the javascript error popup from openid.net
• Read a bunch more instructions
• Find and click the "ClaimID" link (it's the first one on the list of providers)
• Click "Create a new account"
• Type in your information
• Open your email, find their email, click the link
• Go back to StackOverflow, click login again
• Paste in that giant URL that is now your OpenID
• Type in your Username & Password
• Type in a bunch of Personal Info
• ... and you're in! Easy as that!

Now, for sake of comparison, let's take a look at the steps required to start using Twiddla (the web meeting playground that we've been working on these last several months here at Expat):

• Click the "New Meeting" button

Can you spot the difference?

Look, it's not just me saying this. Talk to any Usability expert you like, and they'll tell you that every barrier that you put in front of your users will cause a certain percentage of them to leave and not come back. For most sites, even stopping to ask for a Username & Password is too intrusive. That's why we built Twiddla the way we did.

Our stated goal with Twiddla is to get the hell out of your way so that you can get some work done. We've taken that idea so far that most of our users will never see a login screen of any description. Some might not ever know they've used Twiddla at all, since we keep our Logo hidden away in the corner where it's not in your way.

Can we say the same about StackOverflow's new registration system? Unfortunately not. For me, it was 10 minutes of grumbling "StackOverflow", "F'ng StackOverflow" under my breath while stumbling through the painful OpenID signup process. Complete usability failure. I can only hope they'll come to their senses and put in a reasonable username/password login like everybody else.

Canvas Rendering Issue in Google Chrome

Pop Quiz. What's wrong with this picture?

That's what you'll see if you use Google Chrome to draw a rounded box in a Twiddla meeting today, and it highlights a minor cosmetic issue in Chrome's Canvas rendering engine. Oops.

If you think about how you would draw a rounded box on a canvas (straight line, curve, straight line, curve...), you can quickly see what's going on in that picture. We've told it to "turn right 90 degrees", and it though we meant "270 degrees". Or, in math terms, π/2 vs. -π/2, which is the same as 3π/2, since it ends you up in the same place.

Here is a Test Page that reproduces the Issue. Try it in a few browsers and see it for yourself.

The strange thing here is that Chrome is supposed to be using WebKit's Canvas engine. WebKit runs Safari, and Safari draws that box just fine. Funky.

As far as I'm concerned, Chrome is a great browser. With this one minor exception, it ran Twiddla perfectly right out of the box, which is certainly more than I can say for FireFox 3 (but that's another post...), and it's rendering engine is a bit faster than Safari (but not quite as fast as Internet Explorer. Go figure.)

Keep up the good work!

UPDATE: Google fixed this issue on December 11, 2008. Here'a a link to their bug report on the issue.

How to do all that website optimizing stuff that Yahoo recommends if you're running ASP.NET and storing your content at Amazon S3

If you've come within 30 feet of the internet this last month, you'll have come across this list of best practices at least a dozen times. Everybody seems to be writing about it and linking to it and building little tools that tell you you're not doing it right.

Most of the stuff on that list is low hanging fruit. You can spend 5 minutes in IIS, flipping compression on and telling all your /images/ directories not to expire content until we're all driving flying cars, and suddenly you'll find your site loading a lot faster.

That's cool and all, but what if you also followed their advice and stuck a bunch of your static content out on Amazon S3? I guess you just fire up S3Fox and start playing with the metadata on all those… whoa, hang on… hey, you can't change that stuff once it's written. Crap. You've gotta upload all those files again. And you can't use that cool Firefox tool to do it anymore, because it has no way to set an "Expires" header when you upload a file. Crap. Crap. Crap.

Well if you're running C# and ASP.NET, you're in luck. Because I just went through that pain for a few of my sites, and now I'm going to let you mooch off my code.

First step: download the right library from Amazon

In this case, you're going to need the Amazon S3 REST Library for C#. No, not the SOAP library, because evidently that one is crap. Either drop the source straight into your project or build it elsewhere and link it in.

Last step: swipe this code

This zip contains everything you'll need. Just airlift it into your project and you'll be good to go. Now, since this is an article about programming, I'm legally obligated to provide at least one code sample for you to gloss over. So here is the meat of what we're doing:

public void PushToAmazonS3ViaREST(string bucket, string relativePath, HttpServerUtility server)
{
    relativePath = relativePath.TrimStart('/');
    string fullPath = _basePath + relativePath.Replace(@"/", @"\");

    AWSAuthConnection s3 = new AWSAuthConnection(_publicKey, _secretKey);
    string sContentType = "image/jpeg";
    SortedList sList = new SortedList();
    sList.Add("Content-Type", sContentType);

    // Set access control list to "publicly readable"
    sList.Add("x-amz-acl", "public-read"); 

    // Set to expire in ten years
    sList.Add("Expires", GetHttpDateString(DateTime.Now.AddYears(10))); 

    S3Object obj = new S3Object(FileContentsAsString(fullPath), sList);
    s3.PutObjectAsStream(bucket, relativePath, fullPath, obj.Metadata);
}

There's only two lines you need to care about if you're using S3 to host web content, and they're both commented. One sets the file to be readable by the public, and the other tells it not to expire until after you've left the company. Sorted.

I've included a cheesy .aspx page that you can use to push your files by hand. Hopefully you can figure out how to change which directories it's putting in the list, and how to add your own. It's actually pretty ugly code, but hey, it's just an admin tool that you'll only run a few times in your life.

Be Warned though: I've stripped out the security that keeps people from the outside world (and GoogleBot) from hitting this page and bogging your server. If there's any chance that this might escape to the live site, be sure to lock it down so that you can't see it unless you're logged in as an admin!

Anyway, I hope you find some use out of that code. I certainly wasn't planning to publish it, so please refrain from mentioning the 47-odd things in it that you should never do in production!

Enjoy!

paint chat software

Navigating The Minefield that is Visual Source Safe

I have a new candidate for the Most Infuriating Feature Ever. It's an innocuous little part of the source control implementation for Visual Studio.NET.

Let's say you're working on a new and risky set of changes to a project in Visual Studio.NET. You set off and start breaking things in existing files, safe in the knowledge that if you can't make it all work in the end, you'll be able to roll everything back in source control. Cut to half an hour later: things are hopelessly broken, and it's apparent that you're heading in the wrong direction. Best to cut your losses and start again from scratch, so you right click the solution in VS.NET and select "Undo Checkout" to roll everything back. As if to confirm, the following dialog pops up:

Note the default option. It's not really very descriptive, but what it's actually saying is "Roll these changes back in a half-baked way that virtually guarantees I'll accidentally re-implement them all the next time I modify any of these files."

You see, what it's doing by default is leaving a copy of your broken code sitting on your local machine. Forever. Getting latest won't even overwrite it. Neither will checking the file out. So the next time you want to modify that file, it will pull up the changes you thought you had un-done and not even warn you about it. You'll make some innocuous little text modification, check in, and find that the whole application is broken.

This is just one of many hazardous dialogs that developers running VSS have to tiptoe their way past every day. Dialogs with FIVE BUTTONS, only one of which does what Source Control was intended to do, and that one is hidden second from the left. It's enough to make you want to switch over to subversion.

Oh, and in case you're wondering, the correct response (and the only one that anybody should ever use) to that dialog above is to tick the "Replace your local file…" radio button, check the box, and hit OK. Any other combination and you're screwed.

ps. We're currently rebranding Twiddla as a design collaboration tool for distributed teams. If you're in the industry, we'd love to hear your feedback!