Telling Your Typographer to SmartStuffIt

Oh, it wouldn’t be an issue if the rules weren’t constantly changing, would it? How many of you were taught to place a pair of spaces after a period? Indent a paragraph? How about quotes? Do you remember any specific rules about quotes? Anyone ever tell you about a ‘Smart Quote” as you blasted out the Great American Novel on your Remington portable? I’ll make it easier. How many of you ever used a typewriter? Do you recall that, for the longest time, there was no such thing (or a need) for a scalable font, the difference between an Em-Dash and an En-Dash was relegated to the silent musings of the typesetter, cursed to a life of reading little metal characters in reverse as he set up the page prior to working his frustrations out by squeezing ink onto paper with the sheer bulk of his output.

But now we’re in a world of broader rebellion. Everyone has become an author. Everyone has turned into a typography specialist and all the things you used to do with print are, well, wrong. That’s okay. Our world changes all the time and you’ve gotta keep up. Still, those poor folks trying to apply the new rules to your work often have a wonderful education in exposing the life in your writing. Too bad they often have no idea how to make your work viewable, properly, across platforms.

For instance, were you aware that Microsoft Word’s SmartQuotes really aren’t? Did you know that the way Microsoft produces these characters is not compliant with ASCII, ANSI, the Latin-1 unicode set? If you produce a Word document, save it as HTML and publish it, only Microsoft users will be able to properly render the page?

Recently, a request came into Dian asking her what the ASCII character code for left and right SmartQuotes are. The answer, of course, is there is no real ASCII representation of these characters. The ONLY way to represent them is to leave the base ASCII code behind and move into the High ASCII range...and Microsoft Word doesn’t even use these values for the job so the best answer we could provide was to use characters 147 and 148 from the High ASCII range.

Sheesh!! That’s nuts!! There has to be some order here someplace, some way to get the stylist satisfied while not completely mucking up your work as it’s read on a Linux workstation. If you’ve been working with web pages at all, you’re probably familiar with the concept of the Color Safety Palette (the 216 color codes which will render the exact same color across all platforms). Think of the rest of this article as the Type Safety Palette and you’ll probably find it to be an easier meal to digest.

Here are the universally acceptable ASCII codes which will be properly represented across platforms. (The following two tables are taken directly from Microsoft’s own VBA help files)

Character Set (0 – 127)   

* *Values 8, 9, 10, and 13 convert to backspace, tab, linefeed, and carriage return characters, respectively. They have no graphical representation but, depending on the application, can affect the visual display of text.

The following High ASCII values will NOT be universally represented. They are represented as values 128 through 255:

Character Set (128 – 255)

The values in the table are the Windows default. However, values in the ANSI character set above 127 are determined by the code page specific to your operating system.

Note that characters 147 and 148 in the table above are listed as unsupported in Windows. Sadly, these ARE the HIGH ASCII values for SmartQuotes . Words interpretation of these characters is mystifying as the values used by Word are not even compliant with Unicode standards (that I have been able to identify).

Also notice the little note at the end of the table. What’s this Code Page stuff all about? Well, it’s fairly simple. The code page is the interpreter between your keyboard and the operating system’s rendering of the particular keystroke you pressed. And each key on your keyboard returns an 8 bit value to the system when you press it. That 8 bit value is interpreted directly by the OS depending on whether it is a keystroke in the lower 128 (lower 4 bits) or the upper 128 (upper 4 bits). In fact, that table may be soooo painfully incompatible that my editor for this article may just banish me to the couch for a few days.

Holy CAPS LOCK, Batman!

Next time you’re sitting in front of your machine to compose your next Op-Ed piece for the NYTimes ezine (that would be the North Yeoman Times Online), remember that you’ll really look much less the idiot if you confine yourself, where possible, to the standard Low ASCII set (characters 0-127). Whatever you create using these characters will render properly in any browser on any operating system and on any processor made in the last 30 years. Your characters will also be HTML compliant right out of the box!

Oh!! And do yourself a favor—disable SmartQuotes and other bits of funkiness in Word if you’d like to appeal to a broader audience. Perhaps at that point there will be no further need for tools like the DeMoronizer script! (http://www.fourmilab.ch/webtools/demoroniser/)

If you really need to represent a special character from that 128-255 range, be sure to use the HTML equivalents as found at http://www.ramsch.org/martin/uni/fmi-hp/iso8859-1.html for the ISO8859-1 (Latin) codeset.

Hey, Big Mouth, How About Some Simple Guidance?

Sure!! When producing documentation in which you don’t know what Fonts are available or even what system will be used to view the information, use only characters in the ASCII 0-127 range. If you need special characters (and you will, if you’re doing any scientific publishing), refer carefully to the characters from 128-255. If you’re producing HTML, use the special character shortcuts or the raw ASCII code for that range. Don’t use Microsoft Office Autoformatting features. If your audience is multinational, use UNICODE (double byte) character sets appropriate to the viewer’s region (assuming you also know the language, too).

That should keep you out of trouble.