| <?xml version="1.0"?> |
| <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" |
| "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [ |
| <!ENTITY % local.common.attrib "xmlns:xi CDATA #FIXED 'http://www.w3.org/2003/XInclude'"> |
| <!ENTITY version SYSTEM "version.xml"> |
| ]> |
| <chapter id="what-is-harfbuzz"> |
| <title>What is HarfBuzz?</title> |
| <para> |
| HarfBuzz is a <emphasis>text-shaping engine</emphasis>. If you |
| give HarfBuzz a font and a string containing a sequence of Unicode |
| codepoints, HarfBuzz selects and positions the corresponding |
| glyphs from the font, applying all of the necessary layout rules |
| and font features. HarfBuzz then returns the string to you in the |
| form that is correctly arranged for the language and writing |
| system. |
| </para> |
| <para> |
| HarfBuzz can properly shape all of the world's major writing |
| systems. It runs on all major operating systems and software |
| platforms and it supports the major font formats in use |
| today. |
| </para> |
| <section id="what-is-text-shaping"> |
| <title>What is text shaping?</title> |
| <para> |
| Text shaping is the process of translating a string of character |
| codes (such as Unicode codepoints) into a properly arranged |
| sequence of glyphs that can be rendered onto a screen or into |
| final output form for inclusion in a document. |
| </para> |
| <para> |
| The shaping process is dependent on the input string, the active |
| font, the script (or writing system) that the string is in, and |
| the language that the string is in. |
| </para> |
| <para> |
| Modern software systems generally only deal with strings in the |
| Unicode encoding scheme (although legacy systems and documents may |
| involve other encodings). |
| </para> |
| <para> |
| There are several font formats that a program might |
| encounter, each of which has a set of standard text-shaping |
| rules. |
| </para> |
| <para>The dominant format is <ulink |
| url="http://www.microsoft.com/typography/otspec/">OpenType</ulink>. The |
| OpenType specification defines a series of <ulink url="https://github.com/n8willis/opentype-shaping-documents">shaping models</ulink> for |
| various scripts from around the world. These shaping models depend on |
| the font incorporating certain features as |
| <emphasis>lookups</emphasis> in its <literal>GSUB</literal> |
| and <literal>GPOS</literal> tables. |
| </para> |
| <para> |
| Alternatively, OpenType fonts can include shaping features for |
| the <ulink url="https://graphite.sil.org/">Graphite</ulink> shaping model. |
| </para> |
| <para> |
| TrueType fonts can also include OpenType shaping |
| features. Alternatively, TrueType fonts can also include <ulink url="https://developer.apple.com/fonts/TrueType-Reference-Manual/RM09/AppendixF.html">Apple |
| Advanced Typography</ulink> (AAT) tables to implement shaping |
| support. AAT fonts are generally only found on macOS and iOS systems. |
| </para> |
| <para> |
| Text strings will usually be tagged with a script and language |
| tag that provide the context needed to perform text shaping |
| correctly. The necessary <ulink |
| url="https://docs.microsoft.com/en-us/typography/opentype/spec/scripttags">script</ulink> |
| and <ulink |
| url="https://docs.microsoft.com/en-us/typography/opentype/spec/languagetags">language</ulink> |
| tags are defined by OpenType. |
| </para> |
| </section> |
| |
| <section id="why-do-i-need-a-shaping-engine"> |
| <title>Why do I need a shaping engine?</title> |
| <para> |
| Text shaping is an integral part of preparing text for |
| display. Before a Unicode sequence can be rendered, the |
| codepoints in the sequence must be mapped to the corresponding |
| glyphs provided in the font, and those glyphs must be positioned |
| correctly relative to each other. For many of the scripts |
| supported in Unicode, these steps involve script-specific layout |
| rules, including complex joining, reordering, and positioning |
| behavior. Implementing these rules is the job of the shaping engine. |
| </para> |
| <para> |
| Text shaping is a fairly low-level operation. HarfBuzz is |
| used directly by text-handling libraries like <ulink |
| url="https://www.pango.org/">Pango</ulink>, as well as by the layout |
| engines in Firefox, LibreOffice, and Chromium. Unless you are |
| <emphasis>writing</emphasis> one of these layout engines |
| yourself, you will probably not need to use HarfBuzz: normally, |
| a layout engine, toolkit, or other library will turn text into |
| glyphs for you. |
| </para> |
| <para> |
| However, if you <emphasis>are</emphasis> writing a layout engine |
| or graphics library yourself, then you will need to perform text |
| shaping, and this is where HarfBuzz can help you. |
| </para> |
| <para> |
| Here are some specific scenarios where a text-shaping engine |
| like HarfBuzz helps you: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| OpenType fonts contain a set of glyphs (that is, shapes |
| to represent the letters, numbers, punctuation marks, and |
| all other symbols), which are indexed by a <literal>glyph ID</literal>. |
| </para> |
| <para> |
| A particular glyph ID within the font does not necessarily |
| correlate to a predictable Unicode codepoint. For instance, |
| some fonts have the letter "a" as glyph ID 1, but |
| many others do not. In order to retrieve the right glyph |
| from the font to display "a", you need to consult |
| the table inside the font (the <literal>cmap</literal> |
| table) that maps Unicode codepoints to glyph IDs. In other |
| words, <emphasis>text shaping turns codepoints into glyph |
| IDs</emphasis>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Many OpenType fonts contain ligatures: combinations of |
| characters that are rendered as a single unit. For instance, |
| it is common for the "f, i" letter |
| sequence to appear in print as the single ligature glyph |
| "fi". |
| </para> |
| <para> |
| Whether you should render an "f, i" sequence |
| as <literal>fi</literal> or as "fi" does not |
| depend on the input text. Instead, it depends on the whether |
| or not the font includes an "fi" glyph and on the |
| level of ligature application you wish to perform. The font |
| and the amount of ligature application used are under your |
| control. In other words, <emphasis>text shaping involves |
| querying the font's ligature tables and determining what |
| substitutions should be made</emphasis>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| While ligatures like "fi" are optional typographic |
| refinements, some languages <emphasis>require</emphasis> certain |
| substitutions to be made in order to display text correctly. |
| </para> |
| <para> |
| For example, in Tamil, when the letter "TTA" (ட) |
| letter is followed by the vowel sign "U" (ு), the pair |
| must be replaced by the single glyph "டு". The |
| sequence of Unicode characters "ட,ு" needs to be |
| substituted with a single "டு" glyph from the |
| font. |
| </para> |
| <para> |
| But "டு" does not have a Unicode codepoint. To |
| find this glyph, you need to consult the table inside |
| the font (the <literal>GSUB</literal> table) that contains |
| substitution information. In other words, <emphasis>text shaping |
| chooses the correct glyph for a sequence of characters |
| provided</emphasis>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Similarly, each Arabic character has four different variants |
| corresponding to the different positions it might appear in |
| within a sequence. Inside a font, there will be separate |
| glyphs for the initial, medial, final, and isolated forms of |
| each letter, each at a different glyph ID. |
| </para> |
| <para> |
| Unicode only assigns one codepoint per character, so a |
| Unicode string will not tell you which glyph variant to use |
| for each character. To decide, you need to analyze the whole |
| string and determine the appropriate glyph for each character |
| based on its position. In other words, <emphasis>text |
| shaping chooses the correct form of the letter by its |
| position and returns the correct glyph from the font</emphasis>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Other languages involve marks and accents that need to be |
| rendered in specific positions relative a base character. For |
| instance, the Moldovan language includes the Cyrillic letter |
| "zhe" (ж) with a breve accent, like so: "ӂ". |
| </para> |
| <para> |
| Some fonts will provide this character as a single |
| zhe-with-breve glyph, but other fonts will not and, instead, |
| will expect the rendering engine to form the character by |
| superimposing the separate "ж" and "˘" |
| glyphs. |
| </para> |
| <para> |
| But exactly where you should draw the breve depends on the |
| height and width of the preceding zhe glyph. To find the |
| right position, you need to consult the table inside |
| the font (the <literal>GPOS</literal> table) that contains |
| positioning information. |
| In other words, <emphasis>text shaping tells you whether you |
| have a precomposed glyph within your font or if you need to |
| compose a glyph yourself out of combining marks—and, |
| if so, where to position those marks.</emphasis> |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| If tasks like these are something that you need to do, then you |
| need a text shaping engine. You could use Uniscribe if you are |
| writing Windows software; you could use CoreText on macOS; or |
| you could use HarfBuzz. |
| </para> |
| <note> |
| <para> |
| In the rest of this manual, the text will assume that the reader |
| is that implementor of a text-layout engine. |
| </para> |
| </note> |
| </section> |
| |
| |
| <section> |
| <title>What does HarfBuzz do?</title> |
| <para> |
| HarfBuzz provides text shaping through a cross-platform |
| C API that accepts sequences of Unicode codepoints as input. Currently, |
| the following OpenType shaping models are supported: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| Indic (covering Devanagari, Bengali, Gujarati, |
| Gurmukhi, Kannada, Malayalam, Oriya, Tamil, Telugu, and |
| Sinhala) |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Arabic (covering Arabic, N'Ko, Syriac, and Mongolian) |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Thai and Lao |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Khmer |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Myanmar |
| </para> |
| </listitem> |
| |
| <listitem> |
| <para> |
| Tibetan |
| </para> |
| </listitem> |
| |
| <listitem> |
| <para> |
| Hangul |
| </para> |
| </listitem> |
| |
| <listitem> |
| <para> |
| Hebrew |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| The Universal Shaping Engine or <emphasis>USE</emphasis> |
| (covering complex scripts not covered by the above shaping |
| models) |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A default shaping model for non-complex scripts |
| (covering Latin, Cyrillic, Greek, Armenian, Georgian, Tifinagh, |
| and many others) |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Emoji (including emoji modifier sequences, flag sequences, |
| and ZWJ sequences) |
| </para> |
| </listitem> |
| </itemizedlist> |
| |
| <para> |
| In addition to OpenType shaping, HarfBuzz supports the latest |
| version of Graphite shaping (the "Graphite 2" model) and AAT |
| shaping. |
| </para> |
| |
| <para> |
| HarfBuzz can read and understand TrueType fonts (.ttf), TrueType |
| collections (.ttc), and OpenType fonts (.otf, including those |
| fonts that contain TrueType-style outlines and those that |
| contain PostScript CFF or CFF2 outlines). |
| </para> |
| |
| <para> |
| HarfBuzz is designed and tested to run on top of the FreeType |
| font renderer. It can run on Linux, Android, Windows, macOS, and |
| iOS systems. |
| </para> |
| |
| <para> |
| In addition to its core shaping functionality, HarfBuzz provides |
| functions for accessing other font features, including optional |
| GSUB and GPOS OpenType features, as well as |
| all color-font formats (<literal>CBDT</literal>, |
| <literal>sbix</literal>, <literal>COLR/CPAL</literal>, and |
| <literal>SVG-OT</literal>) and OpenType variable fonts. HarfBuzz |
| also includes a font-subsetting feature. HarfBuzz can perform |
| some low-level math-shaping operations, although it does not |
| currently perform full shaping for mathematical typesetting. |
| </para> |
| |
| <para> |
| A suite of command-line utilities is also provided in the |
| source-code tree, designed to help users test and debug |
| HarfBuzz's features on real-world fonts and input. |
| </para> |
| </section> |
| |
| <section id="what-harfbuzz-doesnt-do"> |
| <title>What HarfBuzz doesn't do</title> |
| <para> |
| HarfBuzz will take a Unicode string, shape it, and give you the |
| information required to lay it out correctly on a single |
| horizontal (or vertical) line using the font provided. That is the |
| extent of HarfBuzz's responsibility. |
| </para> |
| <para> |
| It is important to note that if you are implementing a complete |
| text-layout engine you may have other responsibilities that |
| HarfBuzz will <emphasis>not</emphasis> help you with. For example: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| HarfBuzz won't help you with bidirectionality. If you want to |
| lay out text that includes a mix of Hebrew and English, you |
| will need to ensure that each buffer provided to HarfBuzz |
| has all of its characters in the same order and that the |
| directionality of the buffer is set correctly. This may mean |
| segmenting the text before it is placed into HarfBuzz buffers. In |
| other words, the user will hit the keys in the following |
| sequence: |
| </para> |
| <programlisting> |
| A B C [space] ג ב א [space] D E F |
| </programlisting> |
| <para> |
| but will expect to see in the output: |
| </para> |
| <programlisting> |
| ABC אבג DEF |
| </programlisting> |
| <para> |
| This reordering is called <emphasis>bidi processing</emphasis> |
| ("bidi" is short for bidirectional), and there's an |
| algorithm as an annex to the Unicode Standard which tells you how |
| to process a string of mixed directionality. |
| Before sending your string to HarfBuzz, you may need to apply the |
| bidi algorithm to it. Libraries such as <ulink |
| url="http://icu-project.org/">ICU</ulink> and <ulink |
| url="http://fribidi.org/">fribidi</ulink> can do this for you. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| HarfBuzz won't help you with text that contains different font |
| properties. For instance, if you have the string "a |
| <emphasis>huge</emphasis> breakfast", and you expect |
| "huge" to be italic, then you will need to send three |
| strings to HarfBuzz: <literal>a</literal>, in your Roman font; |
| <literal>huge</literal> using your italic font; and |
| <literal>breakfast</literal> using your Roman font again. |
| </para> |
| <para> |
| Similarly, if you change the font, font size, script, |
| language, or direction within your string, then you will |
| need to shape each run independently and output them |
| independently. HarfBuzz expects to shape a run of characters |
| that all share the same properties. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| HarfBuzz won't help you with line breaking, hyphenation, or |
| justification. As mentioned above, HarfBuzz lays out the string |
| along a <emphasis>single line</emphasis> of, notionally, |
| infinite length. If you want to find out where the potential |
| word, sentence and line break points are in your text, you |
| could use the ICU library's break iterator functions. |
| </para> |
| <para> |
| HarfBuzz can tell you how wide a shaped piece of text is, which is |
| useful input to a justification algorithm, but it knows nothing |
| about paragraphs, lines or line lengths. Nor will it adjust the |
| space between words to fit them proportionally into a line. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| As a layout-engine implementor, HarfBuzz will help you with the |
| interface between your text and your font, and that's something |
| that you'll need—what you then do with the glyphs that your font |
| returns is up to you. |
| </para> |
| </section> |
| |
| <section id="why-is-it-called-harfbuzz"> |
| <title>Why is it called HarfBuzz?</title> |
| <para> |
| HarfBuzz began its life as text-shaping code within the FreeType |
| project (and you will see references to the FreeType authors |
| within the source code copyright declarations), but was then |
| extracted out to its own project. This project is maintained by |
| Behdad Esfahbod, who named it HarfBuzz. Originally, it was a |
| shaping engine for OpenType fonts—"HarfBuzz" is |
| the Persian for "open type". |
| </para> |
| </section> |
| </chapter> |