How to make Pidgin Unicode compatible

About a year ago I was looking for an alternative to the original QQ software, the biggest chat network in China. One programme I stumbled upon was LumaQQ. But recently, when Gaim was renamed to Pidgin, its developers have re-introduced a working QQ plugin for this free and popular multi-protocol instant messenger. Pidgin is not the most friendly programme in terms of usability, though, and frequently causes problems for users who need to exchange messages in languages that don't use latin characters. I therefore decided to give a few hints on how to make Pidgin more equipped for this type of task.

Background information

First a little background information: Pidgin has been around at least as long as Trillian, another popular multi-protocol instant messenger. Trillian has not seen a new release since 2004, though, and suffers from two problems: software bloat and commercialization. From the first "Pro" version onwards, the free version of Trillian retained only a few basic features and its users were excluded from the ability to extend their application by plugins. Both versions kept becoming more hardware demanding, although the core functionality ramained more or less unchanged. The reason many people use it anyhow is because the original ICQ, MSN and AOL clients are no better. ICQ is bloated, MSN suffers from limited configurability and most importantly: each programme's interface is cluttered with advertising and other nonsense. Pidgin closes this gap by providing a simple, modular cross-platform chat application. In terms of usability it still needs to catch up, however. We will tackle the most common problems here.

Unicode and font issues

Assuming you have Pidgin installed, the first thing you're going to notice will be the relatively small font which is used by default. The problem here is that you can increase the font size (Tools -> Conversation -> Font), but there is one thing Pidgin has in common with Trillian: it doesn't support dynamic font switching, i.e. if you use a font that was designed for Latin text only and you receive a message in Chinese, your chat app should normally switch to a fallback font that can display Chinese characters. Unlike the original MSN messenger, for example, Pidgin and Trillian don't do that. So what you have to do is choose a font that can display Latin text and characters in the language you need. Recommendations are: "Sim Sun" for Simplified Chinese, "MingLiU" for Traditional Chinese, "Mincho" for Japanese, "Batang" or "Guilim" for Korean.

Unfortunately, these fonts don't just fall from the sky. All are Microsoft TrueType fonts that come with one or another version of Windows. More are included in Microsoft's Office Proofing Tools, which among other things, also update your Windows IMEs. The central issue here is: what do you do when you need a font for more than one Unicode language? As for Chinese, Japanese and Korean (CJK), there are two fonts that I know of that can handle this task: "Batang" and "Arial Unicode MS". As far as I remember, Batang doesn't cover all Japanese and all Chinese characters, but it's more widely avaiable and probably okay for the occasional user. "Arial Unicode MS", on the other hand, looks more modern and blends in better on a Western system. To check which languages are supported by your font, copy the text below and paste it into Pidgin's chat window:

汉语 (Simplified Chinese), 漢語 (Traditional Chinese), 日本語 (Japanese), カタカナ (Japanese Katakana), ひらがな (Japanese Hiragana), 한국어 (Korean)

Unfortunately, you will continue to encounter problems occasionally when using a third party-client to exchange Unicode messages. Exchanging Unicode messages tends to work between third party-clients, e.g. Pidgin-Trillian, and between the original clients, e.g. ICQ-ICQ, but every now and then you'll run into trouble when sending a Unicode message from Pidgin to an original client. Especially Korean seems to cause problems frequently, but I wasn't able to single out a specific reason for this. Perhaps it's related to the system codepage in some way, or to messy character encoding on the client's side. Trillian used to cause problems when leaving Chinese offline messages for someone on ICQ. They came out right when the user "forced" Trillian to use UTF-8 encoding, but this somehow didn't go along too well with German umlauts for example.

Update (11-10-2009): Unicode problems will continue if the other side configured its client to use a non-unicode font. To get Pidgin to ignore font formats, open your Buddy List window and go to Tools -> Preferences -> Conversations and uncheck "Show formatting on incoming messages".

Update (14-02-2011): A more elegant solution to this problem was presented here on a blog called Cappella.blogspot.com. The idea is basically to manipulate the config files of the GTK framework used by Pidgin and to change the entries for the fallback fonts used for Unicode characters.

Conclusions

To close this post I would like to remind you that Pidgin is not for everybody. It will definitely require more "messing around" than MSN or even Trillian. It still does not support MSN offline messages and so far there is no support for audio/video chat, although the developers have profited heavily from a Google summer of code project that contributed a lot to the development of these new features. This has drawn a lot of criticism and the future of Pidgin looks a little cloudy. Still, it is a viable alternative for those who prefer open source software and need a lightweight chat client.

Links

• Pidgin Homepage (Pidgin download)
• Pidgin Facebook Plugin
• HanNom (free Vietnamese Unicode true type font that also works pretty well for Chinese and Japanese characters)

This post was originally published at http://blog.portblue.de/2008/06/making-pidgin-unicode-compatible/