mule.texi 77.3 KB
Newer Older
Glenn Morris's avatar
Glenn Morris committed
1 2
@c This is part of the Emacs manual.
@c Copyright (C) 1997, 1999, 2000, 2001, 2002, 2003, 2004,
Glenn Morris's avatar
Glenn Morris committed
@c   2005, 2006, 2007, 2008, 2009, 2010 Free Software Foundation, Inc.
Glenn Morris's avatar
Glenn Morris committed
4 5 6
@c See file emacs.texi for copying conditions.
@node International, Major Modes, Frames, Top
@chapter International Character Set Support
7 8
@c This node is referenced in the tutorial.  When renaming or deleting
@c it, the tutorial needs to be adjusted.  (
Glenn Morris's avatar
Glenn Morris committed
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
@cindex MULE
@cindex international scripts
@cindex multibyte characters
@cindex encoding of characters

@cindex Celtic
@cindex Chinese
@cindex Cyrillic
@cindex Czech
@cindex Devanagari
@cindex Hindi
@cindex Marathi
@cindex Ethiopic
@cindex German
@cindex Greek
@cindex Hebrew
@cindex IPA
@cindex Japanese
@cindex Korean
@cindex Lao
@cindex Latin
@cindex Polish
@cindex Romanian
@cindex Slovak
@cindex Slovenian
@cindex Thai
@cindex Tibetan
@cindex Turkish
@cindex Vietnamese
@cindex Dutch
@cindex Spanish
  Emacs supports a wide variety of international character sets,
including European and Vietnamese variants of the Latin alphabet, as
well as Cyrillic, Devanagari (for Hindi and Marathi), Ethiopic, Greek,
Han (for Chinese and Japanese), Hangul (for Korean), Hebrew, IPA,
Kannada, Lao, Malayalam, Tamil, Thai, Tibetan, and Vietnamese scripts.
Emacs also supports various encodings of these characters used by
other internationalized software, such as word processors and mailers.

  Emacs allows editing text with international characters by supporting
all the related activities:

@itemize @bullet
You can visit files with non-@acronym{ASCII} characters, save non-@acronym{ASCII} text, and
pass non-@acronym{ASCII} text between Emacs and programs it invokes (such as
compilers, spell-checkers, and mailers).  Setting your language
environment (@pxref{Language Environments}) takes care of setting up the
coding systems and other options for a specific language or culture.
Alternatively, you can specify how Emacs should encode or decode text
for each command; see @ref{Text Coding}.

You can display non-@acronym{ASCII} characters encoded by the various
scripts.  This works by using appropriate fonts on graphics displays
(@pxref{Defining Fontsets}), and by sending special codes to text-only
displays (@pxref{Terminal Coding}).  If some characters are displayed
incorrectly, refer to @ref{Undisplayable Characters}, which describes
possible problems and explains how to solve them.

69 70 71 72 73 74
Characters from scripts whose natural ordering of text is from right
to left are reordered for display (@pxref{Bidirectional Editing}).
These scripts include Arabic, Hebrew, Syriac, Thaana, and a few

Glenn Morris's avatar
Glenn Morris committed
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110
You can insert non-@acronym{ASCII} characters or search for them.  To do that,
you can specify an input method (@pxref{Select Input Method}) suitable
for your language, or use the default input method set up when you set
your language environment.  If
your keyboard can produce non-@acronym{ASCII} characters, you can select an
appropriate keyboard coding system (@pxref{Terminal Coding}), and Emacs
will accept those characters.  Latin-1 characters can also be input by
using the @kbd{C-x 8} prefix, see @ref{Unibyte Mode}.

On X Window systems, your locale should be set to an appropriate value
to make sure Emacs interprets keyboard input correctly; see
@ref{Language Environments, locales}.
@end itemize

  The rest of this chapter describes these issues in detail.

* International Chars::     Basic concepts of multibyte characters.
* Enabling Multibyte::      Controlling whether to use multibyte characters.
* Language Environments::   Setting things up for the language you use.
* Input Methods::           Entering text characters not on your keyboard.
* Select Input Method::     Specifying your choice of input methods.
* Coding Systems::          Character set conversion when you read and
                              write files, and so on.
* Recognize Coding::        How Emacs figures out which conversion to use.
* Specify Coding::          Specifying a file's coding system explicitly.
* Output Coding::           Choosing coding systems for output.
* Text Coding::             Choosing conversion to use for file text.
* Communication Coding::    Coding systems for interprocess communication.
* File Name Coding::        Coding systems for file @emph{names}.
* Terminal Coding::         Specifying coding systems for converting
                              terminal input and output.
* Fontsets::                Fontsets are collections of fonts
                              that cover the whole spectrum of characters.
* Defining Fontsets::       Defining a new fontset.
* Modifying Fontsets::      Modifying an existing fontset.
Glenn Morris's avatar
Glenn Morris committed
112 113 114 115
* Undisplayable Characters:: When characters don't display.
* Unibyte Mode::            You can pick one European character set
                              to use without multibyte characters.
* Charsets::                How Emacs groups its internal character codes.
* Bidirectional Editing::   Support for right-to-left scripts.
Glenn Morris's avatar
Glenn Morris committed
117 118 119 120 121 122 123
@end menu

@node International Chars
@section Introduction to International Character Sets

  The users of international character sets and scripts have
established many more-or-less standard coding systems for storing
124 125 126 127 128 129 130 131 132 133 134
files.  These coding systems are typically @dfn{multibyte}, meaning
that sequences of two or more bytes are used to represent individual
non-@acronym{ASCII} characters.

@cindex Unicode
  Internally, Emacs uses its own multibyte character encoding, which
is a superset of the @dfn{Unicode} standard.  This internal encoding
allows characters from almost every known script to be intermixed in a
single buffer or string.  Emacs translates between the multibyte
character encoding and various other coding systems when reading and
writing files, and when exchanging data with subprocesses.
Glenn Morris's avatar
Glenn Morris committed
135 136 137 138 139 140 141 142 143 144 145

@kindex C-h h
@findex view-hello-file
@cindex undisplayable characters
@cindex @samp{?} in display
  The command @kbd{C-h h} (@code{view-hello-file}) displays the file
@file{etc/HELLO}, which shows how to say ``hello'' in many languages.
This illustrates various scripts.  If some characters can't be
displayed on your terminal, they appear as @samp{?} or as hollow boxes
(@pxref{Undisplayable Characters}).

146 147 148 149 150 151 152 153
  Keyboards, even in the countries where these character sets are
used, generally don't have keys for all the characters in them.  You
can insert characters that your keyboard does not support, using
@kbd{C-q} (@code{quoted-insert}) or @kbd{C-x 8 @key{RET}}
(@code{ucs-insert}).  @xref{Inserting Text}.  Emacs also supports
various @dfn{input methods}, typically one for each script or
language, which make it easier to type characters in the script.
@xref{Input Methods}.
Glenn Morris's avatar
Glenn Morris committed
154 155 156 157 158

@kindex C-x RET
  The prefix key @kbd{C-x @key{RET}} is used for commands that pertain
to multibyte characters, coding systems, and input methods.

159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180
@kindex C-x =
@findex what-cursor-position
  The command @kbd{C-x =} (@code{what-cursor-position}) shows
information about the character at point.  In addition to the
character position, which was described in @ref{Position Info}, this
command displays how the character is encoded.  For instance, it
displays the following line in the echo area for the character

Char: c (99, #o143, #x63) point=28062 of 36168 (78%) column=53
@end smallexample

  The four values after @samp{Char:} describe the character that
follows point, first by showing it and then by giving its character
code in decimal, octal and hex.  For a non-@acronym{ASCII} multibyte
character, these are followed by @samp{file} and the character's
representation, in hex, in the buffer's coding system, if that coding
system encodes the character safely and with a single byte
(@pxref{Coding Systems}).  If the character's encoding is longer than
one byte, Emacs shows @samp{file ...}.

181 182 183 184 185 186
  As a special case, if the character lies in the range 128 (0200
octal) through 159 (0237 octal), it stands for a ``raw'' byte that
does not correspond to any specific displayable character.  Such a
``character'' lies within the @code{eight-bit-control} character set,
and is displayed as an escaped octal character code.  In this case,
@kbd{C-x =} shows @samp{part of display ...} instead of @samp{file}.
187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247

@cindex character set of character at point
@cindex font of character at point
@cindex text properties at point
@cindex face at point
  With a prefix argument (@kbd{C-u C-x =}), this command displays a
detailed description of the character in a window:

@itemize @bullet
The character set name, and the codes that identify the character
within that character set; @acronym{ASCII} characters are identified
as belonging to the @code{ascii} character set.

The character's syntax and categories.

The character's encodings, both internally in the buffer, and externally
if you were to save the file.

What keys to type to input the character in the current input method
(if it supports the character).

If you are running Emacs on a graphical display, the font name and
glyph code for the character.  If you are running Emacs on a text-only
terminal, the code(s) sent to the terminal.

The character's text properties (@pxref{Text Properties,,,
elisp, the Emacs Lisp Reference Manual}), including any non-default
faces used to display the character, and any overlays containing it
(@pxref{Overlays,,, elisp, the same manual}).
@end itemize

  Here's an example showing the Latin-1 character A with grave accent,
in a buffer whose coding system is @code{utf-8-unix}:

        character: @`A (192, #o300, #xc0)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0xC0
           syntax: w 	which means: word
         category: j:Japanese l:Latin v:Vietnamese
      buffer code: #xC3 #x80
        file code: not encodable by coding system undecided-unix
          display: by this font (glyph code)
    xft:-unknown-DejaVu Sans Mono-normal-normal-normal-*-13-*-*-*-m-0-iso10646-1 (#x82)

Character code properties: customize what to show
  general-category: Lu (Letter, Uppercase)
  decomposition: (65 768) ('A' '̀')

There are text properties here:
  auto-composed        t
@end smallexample

Glenn Morris's avatar
Glenn Morris committed
248 249 250
@node Enabling Multibyte
@section Enabling Multibyte Characters

251 252 253 254 255
  By default, Emacs starts in multibyte mode: it stores the contents
of buffers and strings using an internal encoding that represents
non-@acronym{ASCII} characters using multi-byte sequences.  Multibyte
mode allows you to use all the supported languages and scripts without
Glenn Morris's avatar
Glenn Morris committed
256 257

@cindex turn multibyte support on or off
  Under very special circumstances, you may want to disable multibyte
259 260
character support, for a specific buffer.
When multibyte characters are disabled in a buffer, we call
261 262 263 264 265
that @dfn{unibyte mode}.  In unibyte mode, each character in the
buffer has a character code ranging from 0 through 255 (0377 octal); 0
through 127 (0177 octal) represent @acronym{ASCII} characters, and 128
(0200 octal) through 255 (0377 octal) represent non-@acronym{ASCII}
Glenn Morris's avatar
Glenn Morris committed
266 267

  To edit a particular file in unibyte representation, visit it using
268 269 270 271 272 273 274 275 276
@code{find-file-literally}.  @xref{Visiting}.  You can convert a
multibyte buffer to unibyte by saving it to a file, killing the
buffer, and visiting the file again with @code{find-file-literally}.
Alternatively, you can use @kbd{C-x @key{RET} c}
(@code{universal-coding-system-argument}) and specify @samp{raw-text}
as the coding system with which to visit or save a file.  @xref{Text
Coding}.  Unlike @code{find-file-literally}, finding a file as
@samp{raw-text} doesn't disable format conversion, uncompression, or
auto mode selection.
Glenn Morris's avatar
Glenn Morris committed
277 278 279 280 281

@cindex Lisp files, and multibyte operation
@cindex multibyte operation, and Lisp files
@cindex unibyte operation, and Lisp files
@cindex init file, and non-@acronym{ASCII} characters
282 283
  Emacs normally loads Lisp files as multibyte.
This includes the Emacs initialization
284 285 286 287 288 289 290 291 292
file, @file{.emacs}, and the initialization files of Emacs packages
such as Gnus.  However, you can specify unibyte loading for a
particular Lisp file, by putting @w{@samp{-*-unibyte: t;-*-}} in a
comment on the first line (@pxref{File Variables}).  Then that file is
always loaded as unibyte text.  The motivation for these conventions
is that it is more reliable to always load any particular Lisp file in
the same way.  However, you can load a Lisp file as unibyte, on any
one occasion, by typing @kbd{C-x @key{RET} c raw-text @key{RET}}
immediately before loading it.
Glenn Morris's avatar
Glenn Morris committed
293 294 295 296 297 298 299 300 301

  The mode line indicates whether multibyte character support is
enabled in the current buffer.  If it is, there are two or more
characters (most often two dashes) near the beginning of the mode
line, before the indication of the visited file's end-of-line
convention (colon, backslash, etc.).  When multibyte characters
are not enabled, nothing precedes the colon except a single dash.
@xref{Mode Line}, for more details about this.

@findex toggle-enable-multibyte-characters
303 304
You can turn on multibyte support in a specific buffer by invoking the
command @code{toggle-enable-multibyte-characters} in that buffer.

Glenn Morris's avatar
Glenn Morris committed
306 307 308 309 310 311 312
@node Language Environments
@section Language Environments
@cindex language environments

  All supported character sets are supported in Emacs buffers whenever
multibyte characters are enabled; there is no need to select a
particular language in order to display its characters in an Emacs
313 314 315 316
buffer.  However, it is important to select a @dfn{language
environment} in order to set various defaults.  Roughly speaking, the
language environment represents a choice of preferred script rather
than a choice of language.
Glenn Morris's avatar
Glenn Morris committed
317 318 319

  The language environment controls which coding systems to recognize
when reading text (@pxref{Recognize Coding}).  This applies to files,
320 321 322
incoming mail, and any other text you read into Emacs.  It may also
specify the default coding system to use when you create a file.  Each
language environment also specifies a default input method.
Glenn Morris's avatar
Glenn Morris committed
323 324 325

@findex set-language-environment
@vindex current-language-environment
  To select a language environment, customize the variable
Glenn Morris's avatar
Glenn Morris committed
327 328
@code{current-language-environment} or use the command @kbd{M-x
set-language-environment}.  It makes no difference which buffer is
329 330
current when you use this command, because the effects apply globally
to the Emacs session.  The supported language environments include:
Glenn Morris's avatar
Glenn Morris committed
331 332 333 334

@cindex Euro sign
@cindex UTF-8
335 336 337 338 339 340 341 342 343 344 345 346 347
ASCII, Belarusian, Bengali, Brazilian Portuguese, Bulgarian,
Chinese-BIG5, Chinese-CNS, Chinese-EUC-TW, Chinese-GB, Chinese-GBK,
Chinese-GB18030, Croatian, Cyrillic-ALT, Cyrillic-ISO, Cyrillic-KOI8,
Czech, Devanagari, Dutch, English, Esperanto, Ethiopic, French,
Georgian, German, Greek, Gujarati, Hebrew, IPA, Italian, Japanese,
Kannada, Khmer, Korean, Lao, Latin-1, Latin-2, Latin-3, Latin-4,
Latin-5, Latin-6, Latin-7, Latin-8 (Celtic), Latin-9 (updated Latin-1
with the Euro sign), Latvian, Lithuanian, Malayalam, Oriya, Polish,
Punjabi, Romanian, Russian, Sinhala, Slovak, Slovenian, Spanish,
Swedish, TaiViet, Tajik, Tamil, Telugu, Thai, Tibetan, Turkish, UTF-8
(for a setup which prefers Unicode characters and files encoded in
UTF-8), Ukrainian, Vietnamese, Welsh, and Windows-1255 (for a setup
which prefers Cyrillic characters and files encoded in Windows-1255).
Glenn Morris's avatar
Glenn Morris committed
348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413
@end quotation

@cindex fonts for various scripts
@cindex Intlfonts package, installation
  To display the script(s) used by your language environment on a
graphical display, you need to have a suitable font.  If some of the
characters appear as empty boxes, you should install the GNU Intlfonts
package, which includes fonts for most supported scripts.@footnote{If
you run Emacs on X, you need to inform the X server about the location
of the newly installed fonts with the following commands:

 xset fp+ /usr/local/share/emacs/fonts
 xset fp rehash
@end example
@xref{Fontsets}, for more details about setting up your fonts.

@findex set-locale-environment
@vindex locale-language-names
@vindex locale-charset-language-names
@cindex locales
  Some operating systems let you specify the character-set locale you
are using by setting the locale environment variables @env{LC_ALL},
@env{LC_CTYPE}, or @env{LANG}.@footnote{If more than one of these is
set, the first one that is nonempty specifies your locale for this
purpose.}  During startup, Emacs looks up your character-set locale's
name in the system locale alias table, matches its canonical name
against entries in the value of the variables
@code{locale-charset-language-names} and @code{locale-language-names},
and selects the corresponding language environment if a match is found.
(The former variable overrides the latter.)  It also adjusts the display
table and terminal coding system, the locale coding system, the
preferred coding system as needed for the locale, and---last but not
least---the way Emacs decodes non-@acronym{ASCII} characters sent by your keyboard.

  If you modify the @env{LC_ALL}, @env{LC_CTYPE}, or @env{LANG}
environment variables while running Emacs, you may want to invoke the
@code{set-locale-environment} function afterwards to readjust the
language environment from the new locale.

@vindex locale-preferred-coding-systems
  The @code{set-locale-environment} function normally uses the preferred
coding system established by the language environment to decode system
messages.  But if your locale matches an entry in the variable
@code{locale-preferred-coding-systems}, Emacs uses the corresponding
coding system instead.  For example, if the locale @samp{ja_JP.PCK}
matches @code{japanese-shift-jis} in
@code{locale-preferred-coding-systems}, Emacs uses that encoding even
though it might normally use @code{japanese-iso-8bit}.

  You can override the language environment chosen at startup with
explicit use of the command @code{set-language-environment}, or with
customization of @code{current-language-environment} in your init

@kindex C-h L
@findex describe-language-environment
  To display information about the effects of a certain language
environment @var{lang-env}, use the command @kbd{C-h L @var{lang-env}
@key{RET}} (@code{describe-language-environment}).  This tells you
which languages this language environment is useful for, and lists the
character sets, coding systems, and input methods that go with it.  It
also shows some sample text to illustrate scripts used in this
language environment.  If you give an empty input for @var{lang-env},
this command describes the chosen language environment.
@anchor{Describe Language Environment}
Glenn Morris's avatar
Glenn Morris committed
415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618

@vindex set-language-environment-hook
  You can customize any language environment with the normal hook
@code{set-language-environment-hook}.  The command
@code{set-language-environment} runs that hook after setting up the new
language environment.  The hook functions can test for a specific
language environment by checking the variable
@code{current-language-environment}.  This hook is where you should
put non-default settings for specific language environment, such as
coding systems for keyboard input and terminal output, the default
input method, etc.

@vindex exit-language-environment-hook
  Before it starts to set up the new language environment,
@code{set-language-environment} first runs the hook
@code{exit-language-environment-hook}.  This hook is useful for undoing
customizations that were made with @code{set-language-environment-hook}.
For instance, if you set up a special key binding in a specific language
environment using @code{set-language-environment-hook}, you should set
up @code{exit-language-environment-hook} to restore the normal binding
for that key.

@node Input Methods
@section Input Methods

@cindex input methods
  An @dfn{input method} is a kind of character conversion designed
specifically for interactive input.  In Emacs, typically each language
has its own input method; sometimes several languages which use the same
characters can share one input method.  A few languages support several
input methods.

  The simplest kind of input method works by mapping @acronym{ASCII} letters
into another alphabet; this allows you to use one other alphabet
instead of @acronym{ASCII}.  The Greek and Russian input methods
work this way.

  A more powerful technique is composition: converting sequences of
characters into one letter.  Many European input methods use composition
to produce a single non-@acronym{ASCII} letter from a sequence that consists of a
letter followed by accent characters (or vice versa).  For example, some
methods convert the sequence @kbd{a'} into a single accented letter.
These input methods have no special commands of their own; all they do
is compose sequences of printing characters.

  The input methods for syllabic scripts typically use mapping followed
by composition.  The input methods for Thai and Korean work this way.
First, letters are mapped into symbols for particular sounds or tone
marks; then, sequences of these which make up a whole syllable are
mapped into one syllable sign.

  Chinese and Japanese require more complex methods.  In Chinese input
methods, first you enter the phonetic spelling of a Chinese word (in
input method @code{chinese-py}, among others), or a sequence of
portions of the character (input methods @code{chinese-4corner} and
@code{chinese-sw}, and others).  One input sequence typically
corresponds to many possible Chinese characters.  You select the one
you mean using keys such as @kbd{C-f}, @kbd{C-b}, @kbd{C-n},
@kbd{C-p}, and digits, which have special meanings in this situation.

  The possible characters are conceptually arranged in several rows,
with each row holding up to 10 alternatives.  Normally, Emacs displays
just one row at a time, in the echo area; @code{(@var{i}/@var{j})}
appears at the beginning, to indicate that this is the @var{i}th row
out of a total of @var{j} rows.  Type @kbd{C-n} or @kbd{C-p} to
display the next row or the previous row.

    Type @kbd{C-f} and @kbd{C-b} to move forward and backward among
the alternatives in the current row.  As you do this, Emacs highlights
the current alternative with a special color; type @code{C-@key{SPC}}
to select the current alternative and use it as input.  The
alternatives in the row are also numbered; the number appears before
the alternative.  Typing a digit @var{n} selects the @var{n}th
alternative of the current row and uses it as input.

  @key{TAB} in these Chinese input methods displays a buffer showing
all the possible characters at once; then clicking @kbd{Mouse-2} on
one of them selects that alternative.  The keys @kbd{C-f}, @kbd{C-b},
@kbd{C-n}, @kbd{C-p}, and digits continue to work as usual, but they
do the highlighting in the buffer showing the possible characters,
rather than in the echo area.

  In Japanese input methods, first you input a whole word using
phonetic spelling; then, after the word is in the buffer, Emacs
converts it into one or more characters using a large dictionary.  One
phonetic spelling corresponds to a number of different Japanese words;
to select one of them, use @kbd{C-n} and @kbd{C-p} to cycle through
the alternatives.

  Sometimes it is useful to cut off input method processing so that the
characters you have just entered will not combine with subsequent
characters.  For example, in input method @code{latin-1-postfix}, the
sequence @kbd{e '} combines to form an @samp{e} with an accent.  What if
you want to enter them as separate characters?

  One way is to type the accent twice; this is a special feature for
entering the separate letter and accent.  For example, @kbd{e ' '} gives
you the two characters @samp{e'}.  Another way is to type another letter
after the @kbd{e}---something that won't combine with that---and
immediately delete it.  For example, you could type @kbd{e e @key{DEL}
'} to get separate @samp{e} and @samp{'}.

  Another method, more general but not quite as easy to type, is to use
@kbd{C-\ C-\} between two characters to stop them from combining.  This
is the command @kbd{C-\} (@code{toggle-input-method}) used twice.
@xref{Select Input Method}.
@end ifnottex

@cindex incremental search, input method interference
  @kbd{C-\ C-\} is especially useful inside an incremental search,
because it stops waiting for more characters to combine, and starts
searching for what you have already entered.

  To find out how to input the character after point using the current
input method, type @kbd{C-u C-x =}.  @xref{Position Info}.

@vindex input-method-verbose-flag
@vindex input-method-highlight-flag
  The variables @code{input-method-highlight-flag} and
@code{input-method-verbose-flag} control how input methods explain
what is happening.  If @code{input-method-highlight-flag} is
non-@code{nil}, the partial sequence is highlighted in the buffer (for
most input methods---some disable this feature).  If
@code{input-method-verbose-flag} is non-@code{nil}, the list of
possible characters to type next is displayed in the echo area (but
not when you are in the minibuffer).

@node Select Input Method
@section Selecting an Input Method

@table @kbd
@item C-\
Enable or disable use of the selected input method.

@item C-x @key{RET} C-\ @var{method} @key{RET}
Select a new input method for the current buffer.

@item C-h I @var{method} @key{RET}
@itemx C-h C-\ @var{method} @key{RET}
@findex describe-input-method
@kindex C-h I
@kindex C-h C-\
Describe the input method @var{method} (@code{describe-input-method}).
By default, it describes the current input method (if any).  This
description should give you the full details of how to use any
particular input method.

@item M-x list-input-methods
Display a list of all the supported input methods.
@end table

@findex set-input-method
@vindex current-input-method
@kindex C-x RET C-\
  To choose an input method for the current buffer, use @kbd{C-x
@key{RET} C-\} (@code{set-input-method}).  This command reads the
input method name from the minibuffer; the name normally starts with the
language environment that it is meant to be used with.  The variable
@code{current-input-method} records which input method is selected.

@findex toggle-input-method
@kindex C-\
  Input methods use various sequences of @acronym{ASCII} characters to
stand for non-@acronym{ASCII} characters.  Sometimes it is useful to
turn off the input method temporarily.  To do this, type @kbd{C-\}
(@code{toggle-input-method}).  To reenable the input method, type
@kbd{C-\} again.

  If you type @kbd{C-\} and you have not yet selected an input method,
it prompts for you to specify one.  This has the same effect as using
@kbd{C-x @key{RET} C-\} to specify an input method.

  When invoked with a numeric argument, as in @kbd{C-u C-\},
@code{toggle-input-method} always prompts you for an input method,
suggesting the most recently selected one as the default.

@vindex default-input-method
  Selecting a language environment specifies a default input method for
use in various buffers.  When you have a default input method, you can
select it in the current buffer by typing @kbd{C-\}.  The variable
@code{default-input-method} specifies the default input method
(@code{nil} means there is none).

  In some language environments, which support several different input
methods, you might want to use an input method different from the
default chosen by @code{set-language-environment}.  You can instruct
Emacs to select a different default input method for a certain
language environment, if you wish, by using
@code{set-language-environment-hook} (@pxref{Language Environments,
set-language-environment-hook}).  For example:

(defun my-chinese-setup ()
  "Set up my private Chinese environment."
  (if (equal current-language-environment "Chinese-GB")
      (setq default-input-method "chinese-tonepy")))
(add-hook 'set-language-environment-hook 'my-chinese-setup)
@end lisp

This sets the default input method to be @code{chinese-tonepy}
whenever you choose a Chinese-GB language environment.

619 620 621 622 623 624 625 626 627
You can instruct Emacs to activate a certain input method
automatically.  For example:

(add-hook 'text-mode-hook
  (lambda () (set-input-method "german-prefix")))
@end lisp

This activates the input method ``german-prefix'' automatically in the
629 630
Text mode.

Glenn Morris's avatar
Glenn Morris committed
631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662
@findex quail-set-keyboard-layout
  Some input methods for alphabetic scripts work by (in effect)
remapping the keyboard to emulate various keyboard layouts commonly used
for those scripts.  How to do this remapping properly depends on your
actual keyboard layout.  To specify which layout your keyboard has, use
the command @kbd{M-x quail-set-keyboard-layout}.

@findex quail-show-key
  You can use the command @kbd{M-x quail-show-key} to show what key (or
key sequence) to type in order to input the character following point,
using the selected keyboard layout.  The command @kbd{C-u C-x =} also
shows that information in addition to the other information about the

@findex list-input-methods
  To see a list of all the supported input methods, type @kbd{M-x
list-input-methods}.  The list gives information about each input
method, including the string that stands for it in the mode line.

@node Coding Systems
@section Coding Systems
@cindex coding systems

  Users of various languages have established many more-or-less standard
coding systems for representing them.  Emacs does not use these coding
systems internally; instead, it converts from various coding systems to
its own system when reading data, and converts the internal coding
system to other coding systems when writing data.  Conversion is
possible in reading or writing files, in sending or receiving from the
terminal, and in exchanging data with subprocesses.

  Emacs assigns a name to each coding system.  Most coding systems are
663 664 665 666 667
used for one language, and the name of the coding system starts with
the language name.  Some coding systems are used for several
languages; their names usually start with @samp{iso}.  There are also
special coding systems, such as @code{no-conversion}, @code{raw-text},
and @code{emacs-internal}.
Glenn Morris's avatar
Glenn Morris committed
668 669 670 671 672 673 674 675 676

@cindex international files from DOS/Windows systems
  A special class of coding systems, collectively known as
@dfn{codepages}, is designed to support text encoded by MS-Windows and
MS-DOS software.  The names of these coding systems are
@code{cp@var{nnnn}}, where @var{nnnn} is a 3- or 4-digit number of the
codepage.  You can use these encodings just like any other coding
system; for example, to visit a file encoded in codepage 850, type
@kbd{C-x @key{RET} c cp850 @key{RET} C-x C-f @var{filename}
Glenn Morris's avatar
Glenn Morris committed
678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778

  In addition to converting various representations of non-@acronym{ASCII}
characters, a coding system can perform end-of-line conversion.  Emacs
handles three different conventions for how to separate lines in a file:
newline, carriage-return linefeed, and just carriage-return.

@table @kbd
@item C-h C @var{coding} @key{RET}
Describe coding system @var{coding}.

@item C-h C @key{RET}
Describe the coding systems currently in use.

@item M-x list-coding-systems
Display a list of all the supported coding systems.
@end table

@kindex C-h C
@findex describe-coding-system
  The command @kbd{C-h C} (@code{describe-coding-system}) displays
information about particular coding systems, including the end-of-line
conversion specified by those coding systems.  You can specify a coding
system name as the argument; alternatively, with an empty argument, it
describes the coding systems currently selected for various purposes,
both in the current buffer and as the defaults, and the priority list
for recognizing coding systems (@pxref{Recognize Coding}).

@findex list-coding-systems
  To display a list of all the supported coding systems, type @kbd{M-x
list-coding-systems}.  The list gives information about each coding
system, including the letter that stands for it in the mode line
(@pxref{Mode Line}).

@cindex end-of-line conversion
@cindex line endings
@cindex MS-DOS end-of-line conversion
@cindex Macintosh end-of-line conversion
  Each of the coding systems that appear in this list---except for
@code{no-conversion}, which means no conversion of any kind---specifies
how and whether to convert printing characters, but leaves the choice of
end-of-line conversion to be decided based on the contents of each file.
For example, if the file appears to use the sequence carriage-return
linefeed to separate lines, DOS end-of-line conversion will be used.

  Each of the listed coding systems has three variants which specify
exactly what to do for end-of-line conversion:

@table @code
@item @dots{}-unix
Don't do any end-of-line conversion; assume the file uses
newline to separate lines.  (This is the convention normally used
on Unix and GNU systems.)

@item @dots{}-dos
Assume the file uses carriage-return linefeed to separate lines, and do
the appropriate conversion.  (This is the convention normally used on
Microsoft systems.@footnote{It is also specified for MIME @samp{text/*}
bodies and in other network transport contexts.  It is different
from the SGML reference syntax record-start/record-end format which
Emacs doesn't support directly.})

@item @dots{}-mac
Assume the file uses carriage-return to separate lines, and do the
appropriate conversion.  (This is the convention normally used on the
Macintosh system.)
@end table

  These variant coding systems are omitted from the
@code{list-coding-systems} display for brevity, since they are entirely
predictable.  For example, the coding system @code{iso-latin-1} has
variants @code{iso-latin-1-unix}, @code{iso-latin-1-dos} and

@cindex @code{undecided}, coding system
  The coding systems @code{unix}, @code{dos}, and @code{mac} are
aliases for @code{undecided-unix}, @code{undecided-dos}, and
@code{undecided-mac}, respectively.  These coding systems specify only
the end-of-line conversion, and leave the character code conversion to
be deduced from the text itself.

  The coding system @code{raw-text} is good for a file which is mainly
@acronym{ASCII} text, but may contain byte values above 127 which are
not meant to encode non-@acronym{ASCII} characters.  With
@code{raw-text}, Emacs copies those byte values unchanged, and sets
@code{enable-multibyte-characters} to @code{nil} in the current buffer
so that they will be interpreted properly.  @code{raw-text} handles
end-of-line conversion in the usual way, based on the data
encountered, and has the usual three variants to specify the kind of
end-of-line conversion to use.

  In contrast, the coding system @code{no-conversion} specifies no
character code conversion at all---none for non-@acronym{ASCII} byte values and
none for end of line.  This is useful for reading or writing binary
files, tar files, and other files that must be examined verbatim.  It,
too, sets @code{enable-multibyte-characters} to @code{nil}.

  The easiest way to edit a file with no conversion of any kind is with
the @kbd{M-x find-file-literally} command.  This uses
@code{no-conversion}, and also suppresses other Emacs features that
might convert the file contents before you see them.  @xref{Visiting}.

779 780 781 782 783 784
  The coding system @code{emacs-internal} (or @code{utf-8-emacs},
which is equivalent) means that the file contains non-@acronym{ASCII}
characters stored with the internal Emacs encoding.  This coding
system handles end-of-line conversion based on the data encountered,
and has the usual three variants to specify the kind of end-of-line
Glenn Morris's avatar
Glenn Morris committed
785 786 787 788

@node Recognize Coding
@section Recognizing Coding Systems

789 790 791 792 793
  Whenever Emacs reads a given piece of text, it tries to recognize
which coding system to use.  This applies to files being read, output
from subprocesses, text from X selections, etc.  Emacs can select the
right coding system automatically most of the time---once you have
specified your preferences.
Glenn Morris's avatar
Glenn Morris committed
794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896

  Some coding systems can be recognized or distinguished by which byte
sequences appear in the data.  However, there are coding systems that
cannot be distinguished, not even potentially.  For example, there is no
way to distinguish between Latin-1 and Latin-2; they use the same byte
values with different meanings.

  Emacs handles this situation by means of a priority list of coding
systems.  Whenever Emacs reads a file, if you do not specify the coding
system to use, Emacs checks the data against each coding system,
starting with the first in priority and working down the list, until it
finds a coding system that fits the data.  Then it converts the file
contents assuming that they are represented in this coding system.

  The priority list of coding systems depends on the selected language
environment (@pxref{Language Environments}).  For example, if you use
French, you probably want Emacs to prefer Latin-1 to Latin-2; if you use
Czech, you probably want Latin-2 to be preferred.  This is one of the
reasons to specify a language environment.

@findex prefer-coding-system
  However, you can alter the coding system priority list in detail
with the command @kbd{M-x prefer-coding-system}.  This command reads
the name of a coding system from the minibuffer, and adds it to the
front of the priority list, so that it is preferred to all others.  If
you use this command several times, each use adds one element to the
front of the priority list.

  If you use a coding system that specifies the end-of-line conversion
type, such as @code{iso-8859-1-dos}, what this means is that Emacs
should attempt to recognize @code{iso-8859-1} with priority, and should
use DOS end-of-line conversion when it does recognize @code{iso-8859-1}.

@vindex file-coding-system-alist
  Sometimes a file name indicates which coding system to use for the
file.  The variable @code{file-coding-system-alist} specifies this
correspondence.  There is a special function
@code{modify-coding-system-alist} for adding elements to this list.  For
example, to read and write all @samp{.txt} files using the coding system
@code{chinese-iso-8bit}, you can execute this Lisp expression:

(modify-coding-system-alist 'file "\\.txt\\'" 'chinese-iso-8bit)
@end smallexample

The first argument should be @code{file}, the second argument should be
a regular expression that determines which files this applies to, and
the third argument says which coding system to use for these files.

@vindex inhibit-eol-conversion
@cindex DOS-style end-of-line display
  Emacs recognizes which kind of end-of-line conversion to use based on
the contents of the file: if it sees only carriage-returns, or only
carriage-return linefeed sequences, then it chooses the end-of-line
conversion accordingly.  You can inhibit the automatic use of
end-of-line conversion by setting the variable @code{inhibit-eol-conversion}
to non-@code{nil}.  If you do that, DOS-style files will be displayed
with the @samp{^M} characters visible in the buffer; some people
prefer this to the more subtle @samp{(DOS)} end-of-line type
indication near the left edge of the mode line (@pxref{Mode Line,

@vindex inhibit-iso-escape-detection
@cindex escape sequences in files
  By default, the automatic detection of coding system is sensitive to
escape sequences.  If Emacs sees a sequence of characters that begin
with an escape character, and the sequence is valid as an ISO-2022
code, that tells Emacs to use one of the ISO-2022 encodings to decode
the file.

  However, there may be cases that you want to read escape sequences
in a file as is.  In such a case, you can set the variable
@code{inhibit-iso-escape-detection} to non-@code{nil}.  Then the code
detection ignores any escape sequences, and never uses an ISO-2022
encoding.  The result is that all escape sequences become visible in
the buffer.

  The default value of @code{inhibit-iso-escape-detection} is
@code{nil}.  We recommend that you not change it permanently, only for
one specific operation.  That's because many Emacs Lisp source files
in the Emacs distribution contain non-@acronym{ASCII} characters encoded in the
coding system @code{iso-2022-7bit}, and they won't be
decoded correctly when you visit those files if you suppress the
escape sequence detection.

@vindex auto-coding-alist
@vindex auto-coding-regexp-alist
@vindex auto-coding-functions
  The variables @code{auto-coding-alist},
@code{auto-coding-regexp-alist} and @code{auto-coding-functions} are
the strongest way to specify the coding system for certain patterns of
file names, or for files containing certain patterns; these variables
even override @samp{-*-coding:-*-} tags in the file itself.  Emacs
uses @code{auto-coding-alist} for tar and archive files, to prevent it
from being confused by a @samp{-*-coding:-*-} tag in a member of the
archive and thinking it applies to the archive file as a whole.
Likewise, Emacs uses @code{auto-coding-regexp-alist} to ensure that
RMAIL files, whose names in general don't match any particular
pattern, are decoded correctly.  One of the builtin
@code{auto-coding-functions} detects the encoding for XML files.

@vindex rmail-decode-mime-charset
@vindex rmail-file-coding-system
Glenn Morris's avatar
Glenn Morris committed
898 899 900 901 902
  When you get new mail in Rmail, each message is translated
automatically from the coding system it is written in, as if it were a
separate file.  This uses the priority list of coding systems that you
have specified.  If a MIME message specifies a character set, Rmail
obeys that specification, unless @code{rmail-decode-mime-charset} is
903 904 905 906 907
@code{nil}.  For reading and saving Rmail files themselves, Emacs uses
the coding system specified by the variable
@code{rmail-file-coding-system}.  The default value is @code{nil},
which means that Rmail files are not translated (they are read and
written in the Emacs internal character code).
Glenn Morris's avatar
Glenn Morris committed
908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943

@node Specify Coding
@section Specifying a File's Coding System

  If Emacs recognizes the encoding of a file incorrectly, you can
reread the file using the correct coding system by typing @kbd{C-x
@key{RET} r @var{coding-system} @key{RET}}.  To see what coding system
Emacs actually used to decode the file, look at the coding system
mnemonic letter near the left edge of the mode line (@pxref{Mode
Line}), or type @kbd{C-h C @key{RET}}.

@vindex coding
  You can specify the coding system for a particular file in the file
itself, using the @w{@samp{-*-@dots{}-*-}} construct at the beginning,
or a local variables list at the end (@pxref{File Variables}).  You do
this by defining a value for the ``variable'' named @code{coding}.
Emacs does not really have a variable @code{coding}; instead of
setting a variable, this uses the specified coding system for the
file.  For example, @samp{-*-mode: C; coding: latin-1;-*-} specifies
use of the Latin-1 coding system, as well as C mode.  When you specify
the coding explicitly in the file, that overrides

@node Output Coding
@section Choosing Coding Systems for Output

@vindex buffer-file-coding-system
  Once Emacs has chosen a coding system for a buffer, it stores that
coding system in @code{buffer-file-coding-system}.  That makes it the
default for operations that write from this buffer into a file, such
as @code{save-buffer} and @code{write-region}.  You can specify a
different coding system for further file output from the buffer using
@code{set-buffer-file-coding-system} (@pxref{Text Coding}).

  You can insert any character Emacs supports into any Emacs buffer,
but most coding systems can only handle a subset of these characters.
944 945 946 947
Therefore, it's possible that the characters you insert cannot be
encoded with the coding system that will be used to save the buffer.
For example, you could visit a text file in Polish, encoded in
@code{iso-8859-2}, and add some Russian words to it.  When you save
Glenn Morris's avatar
Glenn Morris committed
948 949 950 951 952 953
that buffer, Emacs cannot use the current value of
@code{buffer-file-coding-system}, because the characters you added
cannot be encoded by that coding system.

  When that happens, Emacs tries the most-preferred coding system (set
by @kbd{M-x prefer-coding-system} or @kbd{M-x
954 955 956 957 958
set-language-environment}).  If that coding system can safely encode
all of the characters in the buffer, Emacs uses it, and stores its
value in @code{buffer-file-coding-system}.  Otherwise, Emacs displays
a list of coding systems suitable for encoding the buffer's contents,
and asks you to choose one of those coding systems.
Glenn Morris's avatar
Glenn Morris committed
959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049

  If you insert the unsuitable characters in a mail message, Emacs
behaves a bit differently.  It additionally checks whether the
most-preferred coding system is recommended for use in MIME messages;
if not, Emacs tells you that the most-preferred coding system is not
recommended and prompts you for another coding system.  This is so you
won't inadvertently send a message encoded in a way that your
recipient's mail software will have difficulty decoding.  (You can
still use an unsuitable coding system if you type its name in response
to the question.)

@vindex sendmail-coding-system
  When you send a message with Mail mode (@pxref{Sending Mail}), Emacs has
four different ways to determine the coding system to use for encoding
the message text.  It tries the buffer's own value of
@code{buffer-file-coding-system}, if that is non-@code{nil}.  Otherwise,
it uses the value of @code{sendmail-coding-system}, if that is
non-@code{nil}.  The third way is to use the default coding system for
new files, which is controlled by your choice of language environment,
if that is non-@code{nil}.  If all of these three values are @code{nil},
Emacs encodes outgoing mail using the Latin-1 coding system.

@node Text Coding
@section Specifying a Coding System for File Text

  In cases where Emacs does not automatically choose the right coding
system for a file's contents, you can use these commands to specify

@table @kbd
@item C-x @key{RET} f @var{coding} @key{RET}
Use coding system @var{coding} for saving or revisiting the visited
file in the current buffer.

@item C-x @key{RET} c @var{coding} @key{RET}
Specify coding system @var{coding} for the immediately following

@item C-x @key{RET} r @var{coding} @key{RET}
Revisit the current file using the coding system @var{coding}.

@item M-x recode-region @key{RET} @var{right} @key{RET} @var{wrong} @key{RET}
Convert a region that was decoded using coding system @var{wrong},
decoding it using coding system @var{right} instead.
@end table

@kindex C-x RET f
@findex set-buffer-file-coding-system
  The command @kbd{C-x @key{RET} f}
(@code{set-buffer-file-coding-system}) sets the file coding system for
the current buffer---in other words, it says which coding system to
use when saving or reverting the visited file.  You specify which
coding system using the minibuffer.  If you specify a coding system
that cannot handle all of the characters in the buffer, Emacs warns
you about the troublesome characters when you actually save the

@cindex specify end-of-line conversion
  You can also use this command to specify the end-of-line conversion
(@pxref{Coding Systems, end-of-line conversion}) for encoding the
current buffer.  For example, @kbd{C-x @key{RET} f dos @key{RET}} will
cause Emacs to save the current buffer's text with DOS-style CRLF line

@kindex C-x RET c
@findex universal-coding-system-argument
  Another way to specify the coding system for a file is when you visit
the file.  First use the command @kbd{C-x @key{RET} c}
(@code{universal-coding-system-argument}); this command uses the
minibuffer to read a coding system name.  After you exit the minibuffer,
the specified coding system is used for @emph{the immediately following

  So if the immediately following command is @kbd{C-x C-f}, for example,
it reads the file using that coding system (and records the coding
system for when you later save the file).  Or if the immediately following
command is @kbd{C-x C-w}, it writes the file using that coding system.
When you specify the coding system for saving in this way, instead
of with @kbd{C-x @key{RET} f}, there is no warning if the buffer
contains characters that the coding system cannot handle.

  Other file commands affected by a specified coding system include
@kbd{C-x i} and @kbd{C-x C-v}, as well as the other-window variants
of @kbd{C-x C-f}.  @kbd{C-x @key{RET} c} also affects commands that
start subprocesses, including @kbd{M-x shell} (@pxref{Shell}).  If the
immediately following command does not use the coding system, then
@kbd{C-x @key{RET} c} ultimately has no effect.

  An easy way to visit a file with no conversion is with the @kbd{M-x
find-file-literally} command.  @xref{Visiting}.

1050 1051 1052 1053 1054
  The default value of the variable @code{buffer-file-coding-system}
specifies the choice of coding system to use when you create a new file.
It applies when you find a new file, and when you create a buffer and
then save it in a file.  Selecting a language environment typically sets
this variable to a good choice of default coding system for that language
Glenn Morris's avatar
Glenn Morris committed
1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106

@kindex C-x RET r
@findex revert-buffer-with-coding-system
  If you visit a file with a wrong coding system, you can correct this
with @kbd{C-x @key{RET} r} (@code{revert-buffer-with-coding-system}).
This visits the current file again, using a coding system you specify.

@findex recode-region
  If a piece of text has already been inserted into a buffer using the
wrong coding system, you can redo the decoding of it using @kbd{M-x
recode-region}.  This prompts you for the proper coding system, then
for the wrong coding system that was actually used, and does the
conversion.  It first encodes the region using the wrong coding system,
then decodes it again using the proper coding system.

@node Communication Coding
@section Coding Systems for Interprocess Communication

  This section explains how to specify coding systems for use
in communication with other processes.

@table @kbd
@item C-x @key{RET} x @var{coding} @key{RET}
Use coding system @var{coding} for transferring selections to and from
other window-based applications.

@item C-x @key{RET} X @var{coding} @key{RET}
Use coding system @var{coding} for transferring @emph{one}
selection---the next one---to or from another window-based application.

@item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
Use coding systems @var{input-coding} and @var{output-coding} for
subprocess input and output in the current buffer.

@item C-x @key{RET} c @var{coding} @key{RET}
Specify coding system @var{coding} for the immediately following
@end table

@kindex C-x RET x
@kindex C-x RET X
@findex set-selection-coding-system
@findex set-next-selection-coding-system
  The command @kbd{C-x @key{RET} x} (@code{set-selection-coding-system})
specifies the coding system for sending selected text to other windowing
applications, and for receiving the text of selections made in other
applications.  This command applies to all subsequent selections, until
you override it by using the command again.  The command @kbd{C-x
@key{RET} X} (@code{set-next-selection-coding-system}) specifies the
coding system for the next selection made in Emacs or read by Emacs.

@vindex x-select-request-type
1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119
  The variable @code{x-select-request-type} specifies the data type to
request from the X Window System for receiving text selections from
other applications.  If the value is @code{nil} (the default), Emacs
tries @code{COMPOUND_TEXT} and @code{UTF8_STRING}, in this order, and
uses various heuristics to choose the more appropriate of the two
results; if none of these succeed, Emacs falls back on @code{STRING}.
If the value of @code{x-select-request-type} is one of the symbols
@code{COMPOUND_TEXT}, @code{UTF8_STRING}, @code{STRING}, or
@code{TEXT}, Emacs uses only that request type.  If the value is a
list of some of these symbols, Emacs tries only the request types in
the list, in order, until one of them succeeds, or until the list is

Glenn Morris's avatar
Glenn Morris committed
1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149
@kindex C-x RET p
@findex set-buffer-process-coding-system
  The command @kbd{C-x @key{RET} p} (@code{set-buffer-process-coding-system})
specifies the coding system for input and output to a subprocess.  This
command applies to the current buffer; normally, each subprocess has its
own buffer, and thus you can use this command to specify translation to
and from a particular subprocess by giving the command in the
corresponding buffer.

  You can also use @kbd{C-x @key{RET} c} just before the command that
runs or starts a subprocess, to specify the coding system to use for
communication with that subprocess.

  The default for translation of process input and output depends on the
current language environment.

@vindex locale-coding-system
@cindex decoding non-@acronym{ASCII} keyboard input on X
  The variable @code{locale-coding-system} specifies a coding system
to use when encoding and decoding system strings such as system error
messages and @code{format-time-string} formats and time stamps.  That
coding system is also used for decoding non-@acronym{ASCII} keyboard input on X
Window systems.  You should choose a coding system that is compatible
with the underlying system's text representation, which is normally
specified by one of the environment variables @env{LC_ALL},
@env{LC_CTYPE}, and @env{LANG}.  (The first one, in the order
specified above, whose value is nonempty is the one that determines
the text representation.)

1150 1151 1152 1153 1154 1155 1156 1157 1158
@vindex x-select-request-type
  The variable @code{x-select-request-type} specifies a selection data
type of selection to request from the X server.  The default value is
@code{nil}, which means Emacs tries @code{COMPOUND_TEXT} and
@code{UTF8_STRING}, and uses whichever result seems more appropriate.
You can explicitly specify the data type by setting the variable to
one of the symbols @code{COMPOUND_TEXT}, @code{UTF8_STRING},
@code{STRING} and @code{TEXT}.

Glenn Morris's avatar
Glenn Morris committed
1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185
@node File Name Coding
@section Coding Systems for File Names

@table @kbd
@item C-x @key{RET} F @var{coding} @key{RET}
Use coding system @var{coding} for encoding and decoding file
@end table

@vindex file-name-coding-system
@cindex file names with non-@acronym{ASCII} characters
  The variable @code{file-name-coding-system} specifies a coding
system to use for encoding file names.  It has no effect on reading
and writing the @emph{contents} of files.

@findex set-file-name-coding-system
@kindex C-x @key{RET} F
  If you set the variable to a coding system name (as a Lisp symbol or
a string), Emacs encodes file names using that coding system for all
file operations.  This makes it possible to use non-@acronym{ASCII}
characters in file names---or, at least, those non-@acronym{ASCII}
characters which the specified coding system can encode.  Use @kbd{C-x
@key{RET} F} (@code{set-file-name-coding-system}) to specify this

  If @code{file-name-coding-system} is @code{nil}, Emacs uses a
default coding system determined by the selected language environment.
1186 1187 1188
In the default language environment, non-@acronym{ASCII} characters in
file names are not encoded specially; they appear in the file system
using the internal Emacs representation.
Glenn Morris's avatar
Glenn Morris committed
1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254

  @strong{Warning:} if you change @code{file-name-coding-system} (or the
language environment) in the middle of an Emacs session, problems can
result if you have already visited files whose names were encoded using
the earlier coding system and cannot be encoded (or are encoded
differently) under the new coding system.  If you try to save one of
these buffers under the visited file name, saving may use the wrong file
name, or it may get an error.  If such a problem happens, use @kbd{C-x
C-w} to specify a new file name for that buffer.

@findex recode-file-name
  If a mistake occurs when encoding a file name, use the command
@kbd{M-x recode-file-name} to change the file name's coding
system.  This prompts for an existing file name, its old coding
system, and the coding system to which you wish to convert.

@node Terminal Coding
@section Coding Systems for Terminal I/O

@table @kbd
@item C-x @key{RET} k @var{coding} @key{RET}
Use coding system @var{coding} for keyboard input.

@item C-x @key{RET} t @var{coding} @key{RET}
Use coding system @var{coding} for terminal output.
@end table

@kindex C-x RET t
@findex set-terminal-coding-system
  The command @kbd{C-x @key{RET} t} (@code{set-terminal-coding-system})
specifies the coding system for terminal output.  If you specify a
character code for terminal output, all characters output to the
terminal are translated into that coding system.

  This feature is useful for certain character-only terminals built to
support specific languages or character sets---for example, European
terminals that support one of the ISO Latin character sets.  You need to
specify the terminal coding system when using multibyte text, so that
Emacs knows which characters the terminal can actually handle.

  By default, output to the terminal is not translated at all, unless
Emacs can deduce the proper coding system from your terminal type or
your locale specification (@pxref{Language Environments}).

@kindex C-x RET k
@findex set-keyboard-coding-system
@vindex keyboard-coding-system
  The command @kbd{C-x @key{RET} k} (@code{set-keyboard-coding-system})
or the variable @code{keyboard-coding-system} specifies the coding
system for keyboard input.  Character-code translation of keyboard
input is useful for terminals with keys that send non-@acronym{ASCII}
graphic characters---for example, some terminals designed for ISO
Latin-1 or subsets of it.

  By default, keyboard input is translated based on your system locale
setting.  If your terminal does not really support the encoding
implied by your locale (for example, if you find it inserts a
non-@acronym{ASCII} character if you type @kbd{M-i}), you will need to set
@code{keyboard-coding-system} to @code{nil} to turn off encoding.
You can do this by putting

(set-keyboard-coding-system nil)
@end lisp

in your init file.
Glenn Morris's avatar
Glenn Morris committed
1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271

  There is a similarity between using a coding system translation for
keyboard input, and using an input method: both define sequences of
keyboard input that translate into single characters.  However, input
methods are designed to be convenient for interactive use by humans, and
the sequences that are translated are typically sequences of @acronym{ASCII}
printing characters.  Coding systems typically translate sequences of
non-graphic characters.

@node Fontsets
@section Fontsets
@cindex fontsets

  A font typically defines shapes for a single alphabet or script.
Therefore, displaying the entire range of scripts that Emacs supports
requires a collection of many fonts.  In Emacs, such a collection is
1272 1273 1274 1275
called a @dfn{fontset}.  A fontset is defined by a list of font specs,
each assigned to handle a range of character codes, and may fall back
on another fontset for characters which are not covered by the fonts
it specifies.
Glenn Morris's avatar
Glenn Morris committed
1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287

  Each fontset has a name, like a font.  However, while fonts are
stored in the system and the available font names are defined by the
system, fontsets are defined within Emacs itself.  Once you have
defined a fontset, you can use it within Emacs by specifying its name,
anywhere that you could use a single font.  Of course, Emacs fontsets
can use only the fonts that the system supports; if certain characters
appear on the screen as hollow boxes, this means that the fontset in
use for them has no font for those characters.@footnote{The Emacs
installation instructions have information on additional font

1288 1289 1290 1291 1292 1293
  Emacs creates three fontsets automatically: the @dfn{standard
fontset}, the @dfn{startup fontset} and the @dfn{default fontset}.
The default fontset is most likely to have fonts for a wide variety of
non-@acronym{ASCII} characters and is the default fallback for the
other two fontsets, and if you set a default font rather than fontset.
However it does not specify font family names, so results can be
1294 1295
somewhat random if you use it directly.  You can specify use of a
specific fontset with the @samp{-fn} option.  For example,
Glenn Morris's avatar
Glenn Morris committed
1296 1297 1298 1299 1300 1301 1302 1303 1304

emacs -fn fontset-standard
@end example

You can also specify a fontset with the @samp{Font} resource (@pxref{X

1305 1306 1307 1308 1309
  If no fontset is specified for use, then Emacs uses an
@acronym{ASCII} font, with @samp{fontset-default} as a fallback for
characters the font does not cover.  The standard fontset is only used if
explicitly requested, despite its name.

Glenn Morris's avatar
Glenn Morris committed
1310 1311 1312 1313 1314 1315 1316 1317 1318 1319
  A fontset does not necessarily specify a font for every character
code.  If a fontset specifies no font for a certain character, or if it
specifies a font that does not exist on your system, then it cannot
display that character properly.  It will display that character as an
empty box instead.

@node Defining Fontsets
@section Defining fontsets

@vindex standard-fontset-spec
1320 1321
@vindex w32-standard-fontset-spec
@vindex ns-standard-fontset-spec
Glenn Morris's avatar
Glenn Morris committed
@cindex standard fontset
  When running on X, Emacs creates a standard fontset automatically according to the value
Glenn Morris's avatar
Glenn Morris committed
1324 1325 1326 1327 1328 1329 1330 1331 1332
of @code{standard-fontset-spec}.  This fontset's name is

@end example

or just @samp{fontset-standard} for short.

1333 1334 1335 1336
  On GNUstep and Mac, fontset-standard is created using the value of
@code{ns-standard-fontset-spec}, and on Windows it is
created using the value of @code{w32-standard-fontset-spec}.

Glenn Morris's avatar
Glenn Morris committed
1337 1338 1339 1340 1341
  Bold, italic, and bold-italic variants of the standard fontset are
created automatically.  Their names have @samp{bold} instead of
@samp{medium}, or @samp{i} instead of @samp{r}, or both.

@cindex startup fontset
1342 1343 1344 1345 1346 1347 1348 1349
  Emacs generates a fontset automatically, based on any default
@acronym{ASCII} font that you specify with the @samp{Font} resource or
the @samp{-fn} argument, or the default font that Emacs found when it
started.  This is the @dfn{startup fontset} and its name is
@code{fontset-startup}.  It does this by replacing the
@var{charset_registry} field with @samp{fontset}, and replacing
@var{charset_encoding} field with @samp{startup}, then using the
resulting string to specify a fontset.
Glenn Morris's avatar
Glenn Morris committed
1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361

  For instance, if you start Emacs this way,

emacs -fn "*courier-medium-r-normal--14-140-*-iso8859-1"
@end example

Emacs generates the following fontset and uses it for the initial X
window frame:

Glenn Morris's avatar
Glenn Morris committed
1363 1364
@end example

1365 1366 1367 1368 1369
  The startup fontset will use the font that you specify or a variant
with a different registry and encoding for all the characters which
are supported by that font, and fallback on @samp{fontset-default} for
other characters.

Glenn Morris's avatar
Glenn Morris committed
1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451
  With the X resource @samp{Emacs.Font}, you can specify a fontset name
just like an actual font name.  But be careful not to specify a fontset
name in a wildcard resource like @samp{Emacs*Font}---that wildcard
specification matches various other resources, such as for menus, and
menus cannot handle fontsets.

  You can specify additional fontsets using X resources named
@samp{Fontset-@var{n}}, where @var{n} is an integer starting from 0.
The resource value should have this form:

@var{fontpattern}, @r{[}@var{charset}:@var{font}@r{]@dots{}}
@end smallexample

@var{fontpattern} should have the form of a standard X font name, except
for the last two fields.  They should have the form

  The fontset has two names, one long and one short.  The long name is
@var{fontpattern}.  The short name is @samp{fontset-@var{alias}}.  You
can refer to the fontset by either name.

  The construct @samp{@var{charset}:@var{font}} specifies which font to
use (in this fontset) for one particular character set.  Here,
@var{charset} is the name of a character set, and @var{font} is the
font to use for that character set.  You can use this construct any
number of times in defining one fontset.

  For the other character sets, Emacs chooses a font based on
@var{fontpattern}.  It replaces @samp{fontset-@var{alias}} with values
that describe the character set.  For the @acronym{ASCII} character font,
@samp{fontset-@var{alias}} is replaced with @samp{ISO8859-1}.

  In addition, when several consecutive fields are wildcards, Emacs
collapses them into a single wildcard.  This is to prevent use of
auto-scaled fonts.  Fonts made by scaling larger fonts are not usable
for editing, and scaling a smaller font is not useful because it is
better to use the smaller font in its own size, which is what Emacs

  Thus if @var{fontpattern} is this,

@end example

the font specification for @acronym{ASCII} characters would be this:

@end example

and the font specification for Chinese GB2312 characters would be this:

@end example

  You may not have any Chinese font matching the above font
specification.  Most X distributions include only Chinese fonts that
have @samp{song ti} or @samp{fangsong ti} in @var{family} field.  In
such a case, @samp{Fontset-@var{n}} can be specified as below:

Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
@end smallexample

Then, the font specifications for all but Chinese GB2312 characters have
@samp{fixed} in the @var{family} field, and the font specification for
Chinese GB2312 characters has a wild card @samp{*} in the @var{family}

@findex create-fontset-from-fontset-spec
  The function that processes the fontset resource value to create the
fontset is called @code{create-fontset-from-fontset-spec}.  You can also
call this function explicitly to create a fontset.

Chong Yidong's avatar
Chong Yidong committed
  @xref{Fonts}, for more information about font naming.
Glenn Morris's avatar
Glenn Morris committed

1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488
@node Modifying Fontsets
@section Modifying Fontsets
@cindex fontsets, modifying
@findex set-fontset-font

  Fontsets do not always have to be created from scratch.  If only
minor changes are required it may be easier to modify an existing
fontset.  Modifying @samp{fontset-default} will also affect other
fontsets that use it as a fallback, so can be an effective way of
fixing problems with the fonts that Emacs chooses for a particular

Fontsets can be modified using the function @code{set-fontset-font},
specifying a character, a charset, a script, or a range of characters
to modify the font for, and a font-spec for the font to be used. Some
examples are:

;; Use Liberation Mono for latin-3 charset.
(set-fontset-font "fontset-default" 'iso-8859-3 "Liberation Mono")

;; Prefer a big5 font for han characters
(set-fontset-font "fontset-default" 'han (font-spec :registry "big5")
                  nil 'prepend)

;; Use DejaVu Sans Mono as a fallback in fontset-startup before
;; resorting to fontset-default.
(set-fontset-font "fontset-startup" nil "DejaVu Sans Mono" nil 'append)

;; Use MyPrivateFont for the Unicode private use area.
(set-fontset-font "fontset-default"  '(#xe000 . #xf8ff) "MyPrivateFont")

@end example

Glenn Morris's avatar
Glenn Morris committed
1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524
@node Undisplayable Characters
@section Undisplayable Characters

  There may be a some non-@acronym{ASCII} characters that your terminal cannot
display.  Most text-only terminals support just a single character
set (use the variable @code{default-terminal-coding-system}
(@pxref{Terminal Coding}) to tell Emacs which one); characters which
can't be encoded in that coding system are displayed as @samp{?} by

  Graphical displays can display a broader range of characters, but
you may not have fonts installed for all of them; characters that have
no font appear as a hollow box.

  If you use Latin-1 characters but your terminal can't display
Latin-1, you can arrange to display mnemonic @acronym{ASCII} sequences
instead, e.g.@: @samp{"o} for o-umlaut.  Load the library
@file{iso-ascii} to do this.

@vindex latin1-display
  If your terminal can display Latin-1, you can display characters
from other European character sets using a mixture of equivalent
Latin-1 characters and @acronym{ASCII} mnemonics.  Customize the variable
@code{latin1-display} to enable this.  The mnemonic @acronym{ASCII}
sequences mostly correspond to those of the prefix input methods.

@node Unibyte Mode
@section Unibyte Editing Mode

@cindex European character sets
@cindex accented characters
@cindex ISO Latin character sets
@cindex Unibyte operation
  The ISO 8859 Latin-@var{n} character sets define character codes in
the range 0240 to 0377 octal (160 to 255 decimal) to handle the
accented letters and punctuation needed by various European languages
1525 1526 1527 1528 1529 1530
(and some non-European ones).  Note that Emacs considers bytes with
codes in this range as raw bytes, not as characters, even in a unibyte
session, i.e.@: if you disable multibyte characters.  However, Emacs
can still handle these character codes as if they belonged to
@emph{one} of the single-byte character sets at a time.  To specify
@emph{which} of these codes to use, invoke @kbd{M-x
Glenn Morris's avatar
Glenn Morris committed
1531 1532 1533 1534 1535 1536 1537 1538 1539
set-language-environment} and specify a suitable language environment
such as @samp{Latin-@var{n}}.

  For more information about unibyte operation, see @ref{Enabling
Multibyte}.  Note particularly that you probably want to ensure that
your initialization files are read as unibyte if they contain
non-@acronym{ASCII} characters.

@vindex unibyte-display-via-language-environment
1540 1541 1542 1543 1544 1545 1546 1547 1548 1549
  Emacs can also display bytes in the range 160 to 255 as readable
characters, provided the terminal or font in use supports them.  This
works automatically.  On a graphical display, Emacs can also display
single-byte characters through fontsets, in effect by displaying the
equivalent multibyte characters according to the current language
environment.  To request this, set the variable
@code{unibyte-display-via-language-environment} to a non-@code{nil}
value.  Note that setting this only affects how these bytes are
displayed, but does not change the fundamental fact that Emacs treats
them as raw bytes, not as characters.
Glenn Morris's avatar
Glenn Morris committed
1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617

@cindex @code{iso-ascii} library
  If your terminal does not support display of the Latin-1 character
set, Emacs can display these characters as @acronym{ASCII} sequences which at
least give you a clear idea of what the characters are.  To do this,
load the library @code{iso-ascii}.  Similar libraries for other
Latin-@var{n} character sets could be implemented, but we don't have
them yet.

@findex standard-display-8bit
@cindex 8-bit display
  Normally non-ISO-8859 characters (decimal codes between 128 and 159
inclusive) are displayed as octal escapes.  You can change this for
non-standard ``extended'' versions of ISO-8859 character sets by using the
function @code{standard-display-8bit} in the @code{disp-table} library.

  There are two ways to input single-byte non-@acronym{ASCII}

@itemize @bullet
@cindex 8-bit input
You can use an input method for the selected language environment.
@xref{Input Methods}.  When you use an input method in a unibyte buffer,
the non-@acronym{ASCII} character you specify with it is converted to unibyte.

If your keyboard can generate character codes 128 (decimal) and up,
representing non-@acronym{ASCII} characters, you can type those character codes

On a graphical display, you should not need to do anything special to use
these keys; they should simply work.  On a text-only terminal, you
should use the command @code{M-x set-keyboard-coding-system} or the
variable @code{keyboard-coding-system} to specify which coding system
your keyboard uses (@pxref{Terminal Coding}).  Enabling this feature
will probably require you to use @kbd{ESC} to type Meta characters;
however, on a console terminal or in @code{xterm}, you can arrange for
Meta to be converted to @kbd{ESC} and still be able type 8-bit
characters present directly on the keyboard or using @kbd{Compose} or
@kbd{AltGr} keys.  @xref{User Input}.

@kindex C-x 8
@cindex @code{iso-transl} library
@cindex compose character
@cindex dead character
For Latin-1 only, you can use the key @kbd{C-x 8} as a ``compose
character'' prefix for entry of non-@acronym{ASCII} Latin-1 printing
characters.  @kbd{C-x 8} is good for insertion (in the minibuffer as
well as other buffers), for searching, and in any other context where
a key sequence is allowed.

@kbd{C-x 8} works by loading the @code{iso-transl} library.  Once that
library is loaded, the @key{ALT} modifier key, if the keyboard has
one, serves the same purpose as @kbd{C-x 8}: use @key{ALT} together
with an accent character to modify the following letter.  In addition,
if the keyboard has keys for the Latin-1 ``dead accent characters,''
they too are defined to compose with the following character, once
@code{iso-transl} is loaded.

Use @kbd{C-x 8 C-h} to list all the available @kbd{C-x 8} translations.
@end itemize

@node Charsets
@section Charsets
@cindex charsets

1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628
  In Emacs, @dfn{charset} is short for ``character set''.  Emacs
supports most popular charsets (such as @code{ascii},
@code{iso-8859-1}, @code{cp1250}, @code{big5}, and @code{unicode}), in
addition to some charsets of its own (such as @code{emacs},
@code{unicode-bmp}, and @code{eight-bit}).  All supported characters
belong to one or more charsets.

  Emacs normally ``does the right thing'' with respect to charsets, so
that you don't have to worry about them.  However, it is sometimes
helpful to know some of the underlying details about charsets.

Chong Yidong's avatar
Chong Yidong committed
  One example is font selection (@pxref{Fonts}).  Each language
1630 1631 1632 1633 1634 1635 1636
environment (@pxref{Language Environments}) defines a ``priority
list'' for the various charsets.  When searching for a font, Emacs
initially attempts to find one that can display the highest-priority
charsets.  For instance, in the Japanese language environment, the
charset @code{japanese-jisx0208} has the highest priority, so Emacs
tries to use a font whose @code{registry} property is
Glenn Morris's avatar
Glenn Morris committed
1637 1638 1639 1640

@findex list-charset-chars
@cindex characters in a certain charset
@findex describe-character-set
  There are two commands that can be used to obtain information about
1642 1643 1644
charsets.  The command @kbd{M-x list-charset-chars} prompts for a
charset name, and displays all the characters in that character set.
The command @kbd{M-x describe-character-set} prompts for a charset
name, and displays information about that charset, including its
1646 1647 1648
internal representation within Emacs.

@findex list-character-sets
  To display a list of all supported charsets, type @kbd{M-x
list-character-sets}.  The list gives the names of charsets and
1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661
additional information to identity each charset (see
@url{} for details).  In this list,
charsets are divided into two categories: @dfn{normal charsets} are
listed first, followed by @dfn{supplementary charsets}.  A
supplementary charset is one that is used to define another charset
(as a parent or a subset), or to provide backward-compatibility for
older Emacs versions.

  To find out which charset a character in the buffer belongs to, put
point before it and type @kbd{C-u C-x =} (@pxref{International
Glenn Morris's avatar
Glenn Morris committed

1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740
@node Bidirectional Editing
@section Bidirectional Editing
@cindex bidirectional editing
@cindex right-to-left text

  Emacs supports editing text written in scripts, such as Arabic and
Hebrew, whose natural ordering of horizontal text for display is from
right to left.  However, digits and Latin text embedded in these
scripts are still displayed left to right.  It is also not uncommon to
have small portions of text in Arabic or Hebrew embedded in otherwise
Latin document, e.g., as comments and strings in a program source
file.  For these reasons, text that uses these scripts is actually
@dfn{bidirectional}: a mixture of runs of left-to-right and
right-to-left characters.

  This section describes the facilities and options provided by Emacs
for editing bidirectional text.

@cindex logical order
@cindex visual order
  Emacs stores right-to-left and bidirectional text in the so-called
@dfn{logical} (or @dfn{reading}) order: the buffer or string position
of the first character you read precedes that of the next character.
Reordering of bidirectional text into the @dfn{visual} order happens
at display time.  As result, character positions no longer increase
monotonically with their positions on display.  Emacs implements the
Unicode Bidirectional Algorithm described in the Unicode Standard
Annex #9, for reordering of bidirectional text for display.

@vindex bidi-display-reordering
  The buffer-local variable @code{bidi-display-reordering} controls
whether text in the buffer is reordered for display.  If its value is
non-@code{nil}, Emacs reorders characters that have right-to-left
directionality when they are displayed.  The default value is

  Each paragraph of bidirectional text can have its own @dfn{base
direction}, either right-to-left or left-to-right.  (Paragraph
boundaries are defined by the regular expressions
@code{paragraph-start} and @code{paragraph-separate}, see
@ref{Paragraphs}.)  Text in left-to-right paragraphs begins at the
left margin of the window and is truncated or continued when it
reaches the right margin.  By contrast, text in right-to-left
paragraphs begins at the right margin and is continued or truncated at
the left margin.

@vindex bidi-paragraph-direction
  Emacs determines the base direction of each paragraph dynamically,
based on the text at the beginning of the paragraph.  However,
sometimes a buffer may need to force a certain base direction for its
paragraphs.  The variable @code{bidi-paragraph-direction}, if
non-@code{nil}, disables the dynamic determination of the base
direction, and instead forces all paragraphs in the buffer to have the
direction specified by its buffer-local value.  The value can be either
@code{right-to-left} or @code{left-to-right}.  Any other value is
interpreted as @code{nil}.

@cindex LRM
@cindex RLM
  Alternatively, you can control the base direction of a paragraph by
inserting special formatting characters in front of the paragraph.
The special character @code{RIGHT-TO-LEFT MARK}, or @sc{rlm}, forces
the right-to-left direction on the following paragraph, while
@code{LEFT-TO-RIGHT MARK}, or @sc{lrm} forces the left-to-right
direction.  (You can use @kbd{C-x 8 RET} to insert these characters.)
In a GUI session, the @sc{lrm} and @sc{rlm} characters display as

  Because characters are reordered for display, Emacs commands that
operate in the logical order or on stretches of buffer positions may
produce unusual effects.  For example, @kbd{C-f} and @kbd{C-b}
commands move point in the logical order, so the cursor will sometimes
jump when point traverses reordered bidirectional text.  Similarly, a
highlighted region covering a contiguous range of character positions
may look discontinuous if the region spans reordered text.  This is
normal and similar to behavior of other programs that support
bidirectional text.

Glenn Morris's avatar
Glenn Morris committed
1741 1742 1743
   arch-tag: 310ba60d-31ef-4ce7-91f1-f282dd57b6b3
@end ignore