Use Strings to Internationalize C Programs

Submitted by webmaster on Sun, 12/02/2007 - 03:27.

• Embedded C/C++

by Nigel Jones

Products destined for use in multiple
countries often require user interfaces that support several human
languages. Sloppy string management in your programs could result
in unintelligible babble.

A decade or two ago, most embedded systems had an extremely limited
user interface. In most cases, the interface was either non-existent,
or it consisted of a few LEDs and the odd jumper or push button.
As the cost of display technology has plummeted, alphanumeric
user interfaces have become increasingly common. Simultaneously,
a variety of technological, economic, and political pressures
have brought about the need for products to be sold in many countries.
As a result, the need for an embedded system to support multiple
languages has become apparent.

This problem is even more acute for the non-embedded computer
world. Part of the solution for that marketplace was the introduction
of Unicode, wide character types, and so on. Unfortunately, these
techniques require storage capabilities and display resolutions
rarely found in embedded systems. Instead, most embedded systems
with displays typically use a low-resolution LCD or vacuum fluorescent
display (VFD) with a built-in character generator. It's this type
of display that I'll be concentrating on.

Lessons learned

A few years ago, I worked on an industrial measurement system. The product
was to be sold in both North America and Europe. Consequently,
one of the essential design requirements was to support multiple
European languages. The product in question has a 240 x 128-pixel
LCD panel with a built-in character generator. The character generator
contains a subset of the ASCII 256-character set, including "specialized"
characters such as c, u, and e.

Anyway, as I pondered possible approaches to the design, I looked
back at my previous attempts to solve this problem. They weren't
pretty! At the risk of ruining what little reputation I may have
gained, I think it's instructive to look at these previous attempts.

Lesson 1

The first product that I designed that had an alphanumeric display
was implemented with the typical arrogance of a native English
speaker. Namely, it never occurred to me to even consider the
rest of the world. As a result, my code was littered with text
strings. That is, you'd see the assembly language (yes, it was
that long ago) equivalent of:

WrStr("Jet Black");

To make matters worse, this construct would be found in many
functions, split between several files.

The foolishness of this approach struck home when I was asked
to produce a version of the product for the German market. I quickly
realized that I had to edit source code files to implement the
translation. This had several ramifications:

• A separate make environment was needed for each language.
• Every time a change had to be made to the code, the same modification
  had to be made to each language's version of the file. In short,
  it was a maintenance nightmare.
• The source code had to be given to the translator. I think
  you can imagine the problems this caused.

Lesson 2

The next product was a big improvement, because I did what should
have been done in the first place, which was to place all the
text strings into one file. That is, there were alternative string
files called english.c, german.c, and so on for additional languages,
each containing all of the strings for a particular language.
Each string file contained a single array definition that looked
something like this:

char const * const strings[] = 







{







    "Jet Black",







    "Red",







    ......







};

Thus, to display a particular string, my code looked like this:

WrStr(strings[0]);

Now all I had to do to enable support for a new language was
to hand a copy of english.c to the translator, and have him produce
the equivalent strings for the new language. Unfortunately, it
didn't work out that way. It turns out that the English language
is extraordinarily terse when compared with certain other languages.
For example, the German equivalent of Jet Black is Rabenschwarz.

Working from english.c, the translator assumed there were just
nine characters into which to fit the translation. Thus, the translator
was forced into abbreviating the German. However, in many cases,
there was actually more space available on the display such that
the abbreviation looked awkward in the product. The only way to
find out was to execute the code and look at the results. This
is a lot harder than it sounds, because many strings are only
displayed in exceptional conditions. Thus one has to generate
all the external stimuli such that those exceptions occur. In
short, the translation effort remained a Herculean task.

Once it was complete, I still wasn't out of the woods, because
the different length strings caused the memory allocation to change
significantly. Although it did not happen, theoretically I could
have run out of memory.

Lesson 3

By the time I was working on my third product requiring multiple
language support, I was a lot wiser and memory capacities had
increased dramatically. As a result, I now ensured that every
string in my string file was padded with spaces out to the maximum
allowed field width. Furthermore, I had also learned the intricacies
of conditional compilation and passing command line arguments
to make, such that I included every language into the same text
file. Thus, strings.c looked something like Listing
1.

#if defined(ENGLISH)







    char const * const strings[] = 







    {







        "Jet Black ",







        ...







        "Good Bye ",







        ...







        "Evening"







    };







#elif defined(GERMAN)







    char const * const strings[] = 







    {







        "Rabenschwarz",







        ...







        "Auf Wiedersehen ",







        ...







        "Abend "







    };







#endif

Listing 1. Multiple languages in a single C module

This third solution worked well, except that at the same time,
the size of the alphanumeric displays and the complexity of the
user interface had increased. While my first product had just
30 or 40 strings, this latest product had around 500. Thus, the
bulk of the user interface code ended up looking like this:

WrStr(strings[27]);







WrStr(strings[47]);







WrStr(strings[108]);

This code doesn't make clear what string I was actually displaying.
So I was beginning to long for the original:

WrStr("Jet Black");

I ran into another major problem at this time. As the product
evolved, so did the strings. I found myself wanting to delete
certain strings that were no longer needed. But I couldn't do
that without destroying my indexing scheme. Thus, I was forced
into changing unwanted text into null strings, such that strings.c
now contained sequences like this:

char const * const strings[] = {







    "Jet Black  ",







    ...







    "", /* deleted */







    ...







    "Evening"







};

Although this saved the space consumed by the string, I was still
wasting ROM on the pointer. In addition, it looked ugly and had
"kludge" written all over it. I also ran into a more serious problem.
From a maintenance perspective, it would be very useful if related
strings were in contiguous locations. Thus if a particular field
could contain "Jet Black," "Red," or "Pale Grey," I would place
these together in the string file. However, two years later, when
marketing asked for "Yellow" to be added to the list of selections,
I was forced to place "Yellow" at the end of the string list,
well away from its partners. This pained me greatly.

There was one final problem with this implementation (and all
the others) and that was the fact that the strings array was a
global. I've become quite paranoid about globals in the last few
years, such that when I look back at the code now, I have to confess
that I cringe.

I also discovered a neat feature that was missing from all of
the previous disasters. A few years ago, I saw a product demonstration
in which the language was changeable on the fly. That is, without
changing the operating mode or cycling power, the entire user
interface could be changed to another language. The demonstration
was incredibly slick. (Consider the following scenario. Your product
is being introduced at a trade show. Some native French speakers
come to the booth to look at the product. With the push of a button,
you switch the user interface to French. You're already halfway
to a sale.)

In addition to its value as a sales tool, the ability to change
language on the fly is also a valuable tool for validating a new
translation. It's particularly useful when the correct translation
of a word depends heavily upon its context. When working through
the string file, the translator can't see the context, so having
the ability to operate the product and switch back and forth between
languages is invaluable.

An international approach

Being quite a few years wiser than when Lesson #3 was learned,
I was determined to come up with a scheme that would address as
many of the aforementioned problems as possible and add the ability
to switch languages on the fly. What follows is my solution. It's
not perfect-but it is a lot better than any of the previous attempts.

The first decision I made was to separate the string retrieval
mechanism and the string storage technique. There would be no
more global strings array. Instead, strings would be accessed
through a function call. This access function would take a string
number as an argument and return a pointer to the desired string.
Its provisional prototype is:

char const * strGet(int string_no);

This abstraction gave me freedom to implement the data storage
part of the strings in whatever manner I saw fit. In particular,
I realized that implementing the storage as an array of data structures
would have considerable benefit. My data structure looked like
this:

#define N_LANG4







               







typedef struct







{







    /*







     * Maximum length







     */







    int const len; 







    







    /*







     * Array of pointers to language-specific string







     */







    char const * const text[N_LANG]; 







 







} STRING;

This arrangement offered some serious benefits. First, the maximum
allowed string length for the field is stored with the text. Second,
the original string and all of the various translations of it
are together in one place. This makes life a lot easier for the
translator. The downside is, of course, that this uses a lot more
ROM than a preprocessor-based selection. In my case, I had the
ROM to spare. With this data structure in hand, the strings array
now looked like Listing
2.

static const STRING strings[] =







{







    { 







        15,







        {







            "Jet Black ", 		/* English */







            "Rabenschwarz	", 	/* German */







            ...







        }







    },







    {







        15,







        {







            "Red ",			/* English */







            "Rot ",			/* German */







            ...







        }







    },







    ...







};

Listing 2. A better string storage structure

The access function also had another valuable property. When
the application requested a string, the access function could
interrogate the relevant database to find out the current language,
and return the requisite pointer. Voila! Language-independent
code. The access function looks something like this:

char const * strGet(int str_no)







{







    return strings[str_no].text[GetLanguage()];







}

Further improvements

The previous approach certainly solved a few of my problems.
However, it did nothing for the problems of indecipherable string
numbers or adding and deleting strings in the middle of the array.
I needed a means of referring to the strings in a meaningful manner,
together with a way of automatically re-indexing the string table.
My solution was to use an enumerated type. The members of the
enumeration are given the same name as the strings to which they
refer. An example should help clarify this.

Assume the first four strings that appear in the strings array
are "Jet Black," "Red," "Pale Grey," and "Yellow." To display
"Red," I would have to call:

WrStr(strGet(1));

Instead, I define an enumerated list as follows:

typedef enum { JET_BLACK, RED, PALE_GREY, YELLOW, ... } STR;

I now change the prototype of strGet() to (with the changes
in red):

char const * strGet(STR str_no);

Thus, to display the string "Red," the code becomes:

WrStr(strGet(RED));

Furthermore, by defining a macro, Wrstr(X) as follows,

#define Wrstr(X) WrStr(strGet((X)))

we can write:

Wrstr(RED);

This is just as readable as the original WrStr("Red"), but without
any of the aforementioned problems. Furthermore, this technique
allows one to insert or delete strings at will. For instance,
I could insert "Pink" before "Red" in the strings array, do the
same in the enumeration list, and recompile-and none of the code
should be broken.

This was a major step forward, because I now had a system that
met most of my goals

• No global data
• Easy to add additional languages
• Language selection on the fly
• Code is meaningful
• Allows strings to be inserted and deleted at will

Gotchas of enumerated types

However, a couple of "gotchas" arise when using enumerated types
in this way. The first, and most important, is portability. ANSI
C requires only that the first 31 identifiers in a name be significant.
If you can guarantee that all of your strings are shorter than
this, there is no problem. If you cannot make that guarantee,
these are some of your options:

• Switch to a compiler that allows unlimited identifier lengths.
  Many compilers do have this feature.
• Ensure that all strings are unique within the first 31 characters.
  Note that if they aren't, the compiler should issue a re-declaration
  warning.

The next issue to watch out for occurs when you have a large
number of strings. ANSI allows the compiler writer to implement
enumerated types using an implementation-defined integer type.
Thus, technically speaking, a compiler could limit the number
of items in an enumeration to 127 (the largest positive number
that can fit into an 8-bit integer). Thus, if you have rigid portability
constraints, this technique may be problematic. However, practically
speaking, most compilers appear to implement enumerations either
as an int, or as the smallest integral type that can accommodate
the enumerated list.

The third problem I ran into concerns the limited number of legal
characters that can make up an identifier (that is, a-z, A-Z,
0-9, and _). For instance, it is impossible to exactly reproduce
the string "3 Score & 10!". In situations like this, I used _
wherever I couldn't make the exact substitution. Thus, the enumerated
value for "3 Score & 10!" would be _3_SCORE___10_, or possibly
_3_SCORE_AND_10_. This isn't perfect, but it's still better than
a meaningless identifier such as STRING_49.

The final issue was the absolute necessity of keeping the string
file and the enumerated list synchronized. This proved to be quite
difficult. To aid the process, I modified the string table slightly
to include the enumerated type name in the comment field. Next,
I ensured that the last entry in the enumerated list was always
LAST_STR. This allowed the string array to be changed from being
an incomplete declaration to a complete declaration. This means
that the compiler will complain if the number of elements in the
enumerated list does not exactly match the number of elements
in the string array. This proved to be valuable in keeping the
two files synchronized.

The winning design

The final enumerated list and string table declarations are
as shown in Listing
3.

typedef enum







{







    JET_BLACK, RED, PALE_GREY, YELLOW, ..., LAST_STR














} STR;







 







static const STRING strings[LAST_STR






] =







{







    { /* JET_BLACK */














        15,







        {







            "Jet Black ",		/* English */







            "Rabenschwarz ",	/* German */







            ...







        }







    },







    { /* RED */














        15,







        {







            "Red ",			/* English */







            "Rot ",			/* German */







            ...







        }







    },







    ...







};

Listing 3: Final string storage structure

I did all of this manually, but you could certainly develop a
script to automate the process of creating the enumerated list
in the header file (from the contents of the string file).

Having gone through this exercise, I realized that I could make
a bit more use of enumerated lists to make my code more readable
and more maintainable. When the string data structure was introduced,
a manifest constant N_LANG was used to specify the number of languages
supported. A better approach is as follows (with the changes in
red):

typedef enum 

{ ENGLISH, FRENCH, GERMAN, SPANISH, LAST_LANGUAGE }

    LANGUAGE;

Now, the STRING data structure is defined as:

typedef struct







{







    /*







     * Maximum length







     */







    int const len; 







    







    /*







     * Array of pointers to language-specific string







     */







    char const * const text[LAST_LANGUAGE]; 







    







} STRING;

This change may look minor, but it makes adding another language
more intuitive.

So far, I haven't mentioned the utility of storing the maximum
string length in the STRING data structure. The use of this field
arises when ROM is not plentiful, such that it is necessary to
store strings without padding out to the maximum allowed field
width. In cases like this, one has to be careful to clear the
entire field before writing the string. This may be accomplished
by using the string len parameter. If you can afford to pad all
strings out to the allowed field width, it is permissible to drop
the len parameter from the data structure.

Well, that's my fourth attempt at producing an international
product. After three disasters, I'm reasonably confident that
this latest attempt will at least make it into the "not bad" category.
If you have any refinements that you would care to share, please
e-mail me. In the meantime, I'm going to ponder how to elegantly
and robustly support languages such as Chinese, Arabic and Russian
in embedded systems. If I manage to find a reasonable solution,
I'll let you know.