mule.texi 85 KB
Newer Older
1
@c -*- coding: utf-8 -*-
Glenn Morris's avatar
Glenn Morris committed
2
@c This is part of the Emacs manual.
Paul Eggert's avatar
Paul Eggert committed
3
@c Copyright (C) 1997, 1999--2020 Free Software Foundation, Inc.
Glenn Morris's avatar
Glenn Morris committed
4
@c See file emacs.texi for copying conditions.
5
@node International
Glenn Morris's avatar
Glenn Morris committed
6
@chapter International Character Set Support
7 8
@c This node is referenced in the tutorial.  When renaming or deleting
@c it, the tutorial needs to be adjusted.  (TUTORIAL.de)
Glenn Morris's avatar
Glenn Morris committed
9 10 11 12
@cindex international scripts
@cindex multibyte characters
@cindex encoding of characters

13
@cindex Han
Glenn Morris's avatar
Glenn Morris committed
14
@cindex Hindi
15
@cindex Hangul
Glenn Morris's avatar
Glenn Morris committed
16 17
  Emacs supports a wide variety of international character sets,
including European and Vietnamese variants of the Latin alphabet, as
18 19 20
well as Arabic scripts, Brahmic scripts (for languages such as
Bengali, Hindi, and Thai), Cyrillic, Ethiopic, Georgian, Greek, Han
(for Chinese and Japanese), Hangul (for Korean), Hebrew and IPA@.
21
Emacs also supports various encodings of these characters that are used by
Glenn Morris's avatar
Glenn Morris committed
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
other internationalized software, such as word processors and mailers.

  Emacs allows editing text with international characters by supporting
all the related activities:

@itemize @bullet
@item
You can visit files with non-@acronym{ASCII} characters, save non-@acronym{ASCII} text, and
pass non-@acronym{ASCII} text between Emacs and programs it invokes (such as
compilers, spell-checkers, and mailers).  Setting your language
environment (@pxref{Language Environments}) takes care of setting up the
coding systems and other options for a specific language or culture.
Alternatively, you can specify how Emacs should encode or decode text
for each command; see @ref{Text Coding}.

@item
You can display non-@acronym{ASCII} characters encoded by the various
scripts.  This works by using appropriate fonts on graphics displays
40
(@pxref{Defining Fontsets}), and by sending special codes to text
Glenn Morris's avatar
Glenn Morris committed
41 42 43 44
displays (@pxref{Terminal Coding}).  If some characters are displayed
incorrectly, refer to @ref{Undisplayable Characters}, which describes
possible problems and explains how to solve them.

45 46 47 48 49 50
@item
Characters from scripts whose natural ordering of text is from right
to left are reordered for display (@pxref{Bidirectional Editing}).
These scripts include Arabic, Hebrew, Syriac, Thaana, and a few
others.

Glenn Morris's avatar
Glenn Morris committed
51 52 53
@item
You can insert non-@acronym{ASCII} characters or search for them.  To do that,
you can specify an input method (@pxref{Select Input Method}) suitable
Eli Zaretskii's avatar
Eli Zaretskii committed
54
for your language, or use the default input method set up when you choose
Glenn Morris's avatar
Glenn Morris committed
55 56 57 58 59 60
your language environment.  If
your keyboard can produce non-@acronym{ASCII} characters, you can select an
appropriate keyboard coding system (@pxref{Terminal Coding}), and Emacs
will accept those characters.  Latin-1 characters can also be input by
using the @kbd{C-x 8} prefix, see @ref{Unibyte Mode}.

61
With the X Window System, your locale should be set to an appropriate
62
value to make sure Emacs interprets keyboard input correctly; see
Glenn Morris's avatar
Glenn Morris committed
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85
@ref{Language Environments, locales}.
@end itemize

  The rest of this chapter describes these issues in detail.

@menu
* International Chars::     Basic concepts of multibyte characters.
* Language Environments::   Setting things up for the language you use.
* Input Methods::           Entering text characters not on your keyboard.
* Select Input Method::     Specifying your choice of input methods.
* Coding Systems::          Character set conversion when you read and
                              write files, and so on.
* Recognize Coding::        How Emacs figures out which conversion to use.
* Specify Coding::          Specifying a file's coding system explicitly.
* Output Coding::           Choosing coding systems for output.
* Text Coding::             Choosing conversion to use for file text.
* Communication Coding::    Coding systems for interprocess communication.
* File Name Coding::        Coding systems for file @emph{names}.
* Terminal Coding::         Specifying coding systems for converting
                              terminal input and output.
* Fontsets::                Fontsets are collections of fonts
                              that cover the whole spectrum of characters.
* Defining Fontsets::       Defining a new fontset.
86
* Modifying Fontsets::      Modifying an existing fontset.
Glenn Morris's avatar
Glenn Morris committed
87 88 89 90
* Undisplayable Characters:: When characters don't display.
* Unibyte Mode::            You can pick one European character set
                              to use without multibyte characters.
* Charsets::                How Emacs groups its internal character codes.
91
* Bidirectional Editing::   Support for right-to-left scripts.
Glenn Morris's avatar
Glenn Morris committed
92 93 94 95 96 97 98
@end menu

@node International Chars
@section Introduction to International Character Sets

  The users of international character sets and scripts have
established many more-or-less standard coding systems for storing
99 100 101 102 103 104 105 106 107 108 109
files.  These coding systems are typically @dfn{multibyte}, meaning
that sequences of two or more bytes are used to represent individual
non-@acronym{ASCII} characters.

@cindex Unicode
  Internally, Emacs uses its own multibyte character encoding, which
is a superset of the @dfn{Unicode} standard.  This internal encoding
allows characters from almost every known script to be intermixed in a
single buffer or string.  Emacs translates between the multibyte
character encoding and various other coding systems when reading and
writing files, and when exchanging data with subprocesses.
Glenn Morris's avatar
Glenn Morris committed
110 111 112 113 114 115

@kindex C-h h
@findex view-hello-file
@cindex undisplayable characters
@cindex @samp{?} in display
  The command @kbd{C-h h} (@code{view-hello-file}) displays the file
116 117
@file{etc/HELLO}, which illustrates various scripts by showing
how to say ``hello'' in many languages.  If some characters can't be
Glenn Morris's avatar
Glenn Morris committed
118 119 120
displayed on your terminal, they appear as @samp{?} or as hollow boxes
(@pxref{Undisplayable Characters}).

121 122 123
  Keyboards, even in the countries where these character sets are
used, generally don't have keys for all the characters in them.  You
can insert characters that your keyboard does not support, using
124 125 126
@kbd{C-x 8 @key{RET}} (@code{insert-char}).  @xref{Inserting Text}.
Shorthands are available for some common characters; for example, you
can insert a left single quotation mark @t{‘} by typing @kbd{C-x 8
127
[}, or in Electric Quote mode, usually by simply typing @kbd{`}.
128
@xref{Quotation Marks}.  Emacs also supports
129 130 131
various @dfn{input methods}, typically one for each script or
language, which make it easier to type characters in the script.
@xref{Input Methods}.
Glenn Morris's avatar
Glenn Morris committed
132

133
@kindex C-x RET
Glenn Morris's avatar
Glenn Morris committed
134 135 136
  The prefix key @kbd{C-x @key{RET}} is used for commands that pertain
to multibyte characters, coding systems, and input methods.

137 138
@kindex C-x =@r{, and international characters}
@findex what-cursor-position@r{, and international characters}
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158
  The command @kbd{C-x =} (@code{what-cursor-position}) shows
information about the character at point.  In addition to the
character position, which was described in @ref{Position Info}, this
command displays how the character is encoded.  For instance, it
displays the following line in the echo area for the character
@samp{c}:

@smallexample
Char: c (99, #o143, #x63) point=28062 of 36168 (78%) column=53
@end smallexample

  The four values after @samp{Char:} describe the character that
follows point, first by showing it and then by giving its character
code in decimal, octal and hex.  For a non-@acronym{ASCII} multibyte
character, these are followed by @samp{file} and the character's
representation, in hex, in the buffer's coding system, if that coding
system encodes the character safely and with a single byte
(@pxref{Coding Systems}).  If the character's encoding is longer than
one byte, Emacs shows @samp{file ...}.

159 160 161 162 163 164 165 166
@cindex eight-bit character set
@cindex raw bytes
  On rare occasions, Emacs encounters @dfn{raw bytes}: single bytes
whose values are in the range 128 (0200 octal) through 255 (0377
octal), which Emacs cannot interpret as part of a known encoding of
some non-ASCII character.  Such raw bytes are treated as if they
belonged to a special character set @code{eight-bit}; Emacs displays
them as escaped octal codes (this can be customized; @pxref{Display
167
Custom}).  In this case, @kbd{C-x =} shows @samp{raw-byte} instead of
168 169 170 171
@samp{file}.  In addition, @kbd{C-x =} shows the character codes of
raw bytes as if they were in the range @code{#x3FFF80..#x3FFFFF},
which is where Emacs maps them to distinguish them from Unicode
characters in the range @code{#x0080..#x00FF}.
172 173 174 175 176 177 178 179 180 181 182 183 184 185 186

@cindex character set of character at point
@cindex font of character at point
@cindex text properties at point
@cindex face at point
  With a prefix argument (@kbd{C-u C-x =}), this command displays a
detailed description of the character in a window:

@itemize @bullet
@item
The character set name, and the codes that identify the character
within that character set; @acronym{ASCII} characters are identified
as belonging to the @code{ascii} character set.

@item
187
The character's script, syntax and categories.
188 189 190 191 192

@item
What keys to type to input the character in the current input method
(if it supports the character).

193 194
@item
The character's encodings, both internally in the buffer, and externally
195
if you were to save the buffer to a file.
196

197 198
@item
If you are running Emacs on a graphical display, the font name and
199
glyph code for the character.  If you are running Emacs on a text
200 201
terminal, the code(s) sent to the terminal.

202 203 204 205 206 207
@item
If the character was composed on display with any following characters
to form one or more grapheme clusters, the composition information:
the font glyphs if the frame is on a graphical display, else the
characters that were composed.

208 209 210 211 212 213 214
@item
The character's text properties (@pxref{Text Properties,,,
elisp, the Emacs Lisp Reference Manual}), including any non-default
faces used to display the character, and any overlays containing it
(@pxref{Overlays,,, elisp, the same manual}).
@end itemize

215
  Here's an example, with some lines folded to fit into this manual:
216 217

@smallexample
218
             position: 1 of 1 (0%), column: 0
219
            character: ê (displayed as ê) (codepoint 234, #o352, #xea)
220
    preferred charset: unicode (Unicode (ISO10646))
221 222 223 224
code point in charset: 0xEA
               script: latin
               syntax: w        which means: word
             category: .:Base, L:Left-to-right (strong), c:Chinese,
225
                       j:Japanese, l:Latin, v:Viet
Paul Eggert's avatar
Paul Eggert committed
226 227
             to input: type "C-x 8 RET ea" or
                       "C-x 8 RET LATIN SMALL LETTER E WITH CIRCUMFLEX"
228 229
          buffer code: #xC3 #xAA
            file code: #xC3 #xAA (encoded by coding system utf-8-unix)
230
              display: by this font (glyph code)
Paul Eggert's avatar
Paul Eggert committed
231
    xft:-PfEd-DejaVu Sans Mono-normal-normal-
232
        normal-*-15-*-*-*-m-0-iso10646-1 (#xAC)
233 234

Character code properties: customize what to show
235 236 237 238
  name: LATIN SMALL LETTER E WITH CIRCUMFLEX
  old-name: LATIN SMALL LETTER E CIRCUMFLEX
  general-category: Ll (Letter, Lowercase)
  decomposition: (101 770) ('e' '^')
239 240
@end smallexample

Glenn Morris's avatar
Glenn Morris committed
241 242 243 244 245 246
@node Language Environments
@section Language Environments
@cindex language environments

  All supported character sets are supported in Emacs buffers whenever
multibyte characters are enabled; there is no need to select a
247 248
particular language in order to display its characters.
However, it is important to select a @dfn{language
249 250 251
environment} in order to set various defaults.  Roughly speaking, the
language environment represents a choice of preferred script rather
than a choice of language.
Glenn Morris's avatar
Glenn Morris committed
252 253 254

  The language environment controls which coding systems to recognize
when reading text (@pxref{Recognize Coding}).  This applies to files,
255 256 257
incoming mail, and any other text you read into Emacs.  It may also
specify the default coding system to use when you create a file.  Each
language environment also specifies a default input method.
Glenn Morris's avatar
Glenn Morris committed
258 259 260

@findex set-language-environment
@vindex current-language-environment
261
  To select a language environment, customize
Glenn Morris's avatar
Glenn Morris committed
262 263
@code{current-language-environment} or use the command @kbd{M-x
set-language-environment}.  It makes no difference which buffer is
264
current when you use this command, because the effects apply globally
265 266 267 268 269
to the Emacs session.  See the variable @code{language-info-alist} for
the list of supported language environments, and use the command
@kbd{C-h L @var{lang-env} @key{RET}} (@code{describe-language-environment})
for more information about the language environment @var{lang-env}.
Supported language environments include:
Glenn Morris's avatar
Glenn Morris committed
270

271
@c @cindex entries below are split between portions of the list to
Paul Eggert's avatar
Paul Eggert committed
272
@c make them more accurate, i.e., land on the line that mentions the
273 274 275 276
@c language.  However, makeinfo 4.x doesn't fill inside @quotation
@c lines that follow a @cindex entry and whose text has no whitespace.
@c To work around, we group the language environments together, so
@c that the blank that separates them triggers refill.
Glenn Morris's avatar
Glenn Morris committed
277
@quotation
278
@cindex ASCII (language environment)
279
@cindex Arabic
280
ASCII, Arabic,
281 282
@cindex Belarusian
@cindex Bengali
283
Belarusian, Bengali,
284 285
@cindex Brazilian Portuguese
@cindex Bulgarian
286
Brazilian Portuguese, Bulgarian,
287 288
@cindex Burmese
@cindex Cham
289
Burmese, Cham,
290 291 292 293 294
@cindex Chinese
Chinese-BIG5, Chinese-CNS, Chinese-EUC-TW, Chinese-GB,
Chinese-GB18030, Chinese-GBK,
@cindex Croatian
@cindex Cyrillic
295
Croatian, Cyrillic-ALT, Cyrillic-ISO, Cyrillic-KOI8,
296 297
@cindex Czech
@cindex Devanagari
298
Czech, Devanagari,
299 300
@cindex Dutch
@cindex English
301
Dutch, English,
302 303
@cindex Esperanto
@cindex Ethiopic
304
Esperanto, Ethiopic,
305 306
@cindex French
@cindex Georgian
307
French, Georgian,
308 309 310
@cindex German
@cindex Greek
@cindex Gujarati
311
German, Greek, Gujarati,
312 313
@cindex Hebrew
@cindex IPA
314
Hebrew, IPA,
315 316 317 318
@cindex Italian
Italian,
@cindex Japanese
@cindex Kannada
319
Japanese, Kannada,
320 321 322
@cindex Khmer
@cindex Korean
@cindex Lao
323
Khmer, Korean, Lao,
324 325 326 327 328
@cindex Latin
Latin-1, Latin-2, Latin-3, Latin-4, Latin-5, Latin-6, Latin-7,
Latin-8, Latin-9,
@cindex Latvian
@cindex Lithuanian
329
Latvian, Lithuanian,
330 331
@cindex Malayalam
@cindex Oriya
332
Malayalam, Oriya,
333 334
@cindex Persian
@cindex Polish
335
Persian, Polish,
336 337
@cindex Punjabi
@cindex Romanian
338
Punjabi, Romanian,
339 340
@cindex Russian
@cindex Sinhala
341
Russian, Sinhala,
342 343 344
@cindex Slovak
@cindex Slovenian
@cindex Spanish
345
Slovak, Slovenian, Spanish,
346 347
@cindex Swedish
@cindex TaiViet
348
Swedish, TaiViet,
349 350
@cindex Tajik
@cindex Tamil
351
Tajik, Tamil,
352 353
@cindex Telugu
@cindex Thai
354
Telugu, Thai,
355 356
@cindex Tibetan
@cindex Turkish
357
Tibetan, Turkish,
358 359
@cindex UTF-8
@cindex Ukrainian
360
UTF-8, Ukrainian,
361 362
@cindex Vietnamese
@cindex Welsh
363
Vietnamese, Welsh,
364
@cindex Windows-1255
365
and Windows-1255.
Glenn Morris's avatar
Glenn Morris committed
366 367 368
@end quotation

  To display the script(s) used by your language environment on a
369
graphical display, you need to have suitable fonts.
Glenn Morris's avatar
Glenn Morris committed
370 371 372 373 374 375 376 377
@xref{Fontsets}, for more details about setting up your fonts.

@findex set-locale-environment
@vindex locale-language-names
@vindex locale-charset-language-names
@cindex locales
  Some operating systems let you specify the character-set locale you
are using by setting the locale environment variables @env{LC_ALL},
378
@env{LC_CTYPE}, or @env{LANG}.  (If more than one of these is
Glenn Morris's avatar
Glenn Morris committed
379
set, the first one that is nonempty specifies your locale for this
380
purpose.)  During startup, Emacs looks up your character-set locale's
Glenn Morris's avatar
Glenn Morris committed
381 382
name in the system locale alias table, matches its canonical name
against entries in the value of the variables
383 384
@code{locale-charset-language-names} and @code{locale-language-names}
(the former overrides the latter),
Glenn Morris's avatar
Glenn Morris committed
385
and selects the corresponding language environment if a match is found.
386
It also adjusts the display
Glenn Morris's avatar
Glenn Morris committed
387 388 389 390
table and terminal coding system, the locale coding system, the
preferred coding system as needed for the locale, and---last but not
least---the way Emacs decodes non-@acronym{ASCII} characters sent by your keyboard.

391
@c This seems unlikely, doesn't it?
Glenn Morris's avatar
Glenn Morris committed
392
  If you modify the @env{LC_ALL}, @env{LC_CTYPE}, or @env{LANG}
393 394
environment variables while running Emacs (by using @kbd{M-x setenv}),
you may want to invoke the @code{set-locale-environment}
395
command afterwards to readjust the language environment from the new
396
locale.
Glenn Morris's avatar
Glenn Morris committed
397 398 399 400 401 402 403 404 405

@vindex locale-preferred-coding-systems
  The @code{set-locale-environment} function normally uses the preferred
coding system established by the language environment to decode system
messages.  But if your locale matches an entry in the variable
@code{locale-preferred-coding-systems}, Emacs uses the corresponding
coding system instead.  For example, if the locale @samp{ja_JP.PCK}
matches @code{japanese-shift-jis} in
@code{locale-preferred-coding-systems}, Emacs uses that encoding even
406
though it might normally use @code{utf-8}.
Glenn Morris's avatar
Glenn Morris committed
407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430

  You can override the language environment chosen at startup with
explicit use of the command @code{set-language-environment}, or with
customization of @code{current-language-environment} in your init
file.

@kindex C-h L
@findex describe-language-environment
  To display information about the effects of a certain language
environment @var{lang-env}, use the command @kbd{C-h L @var{lang-env}
@key{RET}} (@code{describe-language-environment}).  This tells you
which languages this language environment is useful for, and lists the
character sets, coding systems, and input methods that go with it.  It
also shows some sample text to illustrate scripts used in this
language environment.  If you give an empty input for @var{lang-env},
this command describes the chosen language environment.

@vindex set-language-environment-hook
  You can customize any language environment with the normal hook
@code{set-language-environment-hook}.  The command
@code{set-language-environment} runs that hook after setting up the new
language environment.  The hook functions can test for a specific
language environment by checking the variable
@code{current-language-environment}.  This hook is where you should
431
put non-default settings for specific language environments, such as
Glenn Morris's avatar
Glenn Morris committed
432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450
coding systems for keyboard input and terminal output, the default
input method, etc.

@vindex exit-language-environment-hook
  Before it starts to set up the new language environment,
@code{set-language-environment} first runs the hook
@code{exit-language-environment-hook}.  This hook is useful for undoing
customizations that were made with @code{set-language-environment-hook}.
For instance, if you set up a special key binding in a specific language
environment using @code{set-language-environment-hook}, you should set
up @code{exit-language-environment-hook} to restore the normal binding
for that key.

@node Input Methods
@section Input Methods

@cindex input methods
  An @dfn{input method} is a kind of character conversion designed
specifically for interactive input.  In Emacs, typically each language
451
has its own input method; sometimes several languages that use the same
Glenn Morris's avatar
Glenn Morris committed
452 453 454 455 456 457 458 459 460 461 462 463
characters can share one input method.  A few languages support several
input methods.

  The simplest kind of input method works by mapping @acronym{ASCII} letters
into another alphabet; this allows you to use one other alphabet
instead of @acronym{ASCII}.  The Greek and Russian input methods
work this way.

  A more powerful technique is composition: converting sequences of
characters into one letter.  Many European input methods use composition
to produce a single non-@acronym{ASCII} letter from a sequence that consists of a
letter followed by accent characters (or vice versa).  For example, some
464
methods convert the sequence @kbd{o ^} into a single accented letter.
Glenn Morris's avatar
Glenn Morris committed
465 466 467 468 469 470
These input methods have no special commands of their own; all they do
is compose sequences of printing characters.

  The input methods for syllabic scripts typically use mapping followed
by composition.  The input methods for Thai and Korean work this way.
First, letters are mapped into symbols for particular sounds or tone
471
marks; then, sequences of these that make up a whole syllable are
Glenn Morris's avatar
Glenn Morris committed
472 473 474 475 476 477 478 479 480
mapped into one syllable sign.

  Chinese and Japanese require more complex methods.  In Chinese input
methods, first you enter the phonetic spelling of a Chinese word (in
input method @code{chinese-py}, among others), or a sequence of
portions of the character (input methods @code{chinese-4corner} and
@code{chinese-sw}, and others).  One input sequence typically
corresponds to many possible Chinese characters.  You select the one
you mean using keys such as @kbd{C-f}, @kbd{C-b}, @kbd{C-n},
481 482
@kbd{C-p} (or the arrow keys), and digits, which have special meanings
in this situation.
Glenn Morris's avatar
Glenn Morris committed
483 484 485 486 487 488 489 490 491 492

  The possible characters are conceptually arranged in several rows,
with each row holding up to 10 alternatives.  Normally, Emacs displays
just one row at a time, in the echo area; @code{(@var{i}/@var{j})}
appears at the beginning, to indicate that this is the @var{i}th row
out of a total of @var{j} rows.  Type @kbd{C-n} or @kbd{C-p} to
display the next row or the previous row.

    Type @kbd{C-f} and @kbd{C-b} to move forward and backward among
the alternatives in the current row.  As you do this, Emacs highlights
493
the current alternative with a special color; type @kbd{C-@key{SPC}}
Glenn Morris's avatar
Glenn Morris committed
494 495
to select the current alternative and use it as input.  The
alternatives in the row are also numbered; the number appears before
496 497
the alternative.  Typing a number selects the associated alternative
of the current row and uses it as input.
Glenn Morris's avatar
Glenn Morris committed
498 499

  @key{TAB} in these Chinese input methods displays a buffer showing
500
all the possible characters at once; then clicking @kbd{mouse-2} on
Glenn Morris's avatar
Glenn Morris committed
501 502 503 504 505
one of them selects that alternative.  The keys @kbd{C-f}, @kbd{C-b},
@kbd{C-n}, @kbd{C-p}, and digits continue to work as usual, but they
do the highlighting in the buffer showing the possible characters,
rather than in the echo area.

506 507 508 509
  To enter characters according to the @dfn{pīnyīn} transliteration
method instead, use the @code{chinese-sisheng} input method.  This is
a composition based method, where e.g. @kbd{pi1} results in @samp{pī}.

Glenn Morris's avatar
Glenn Morris committed
510 511 512 513 514 515 516 517 518 519
  In Japanese input methods, first you input a whole word using
phonetic spelling; then, after the word is in the buffer, Emacs
converts it into one or more characters using a large dictionary.  One
phonetic spelling corresponds to a number of different Japanese words;
to select one of them, use @kbd{C-n} and @kbd{C-p} to cycle through
the alternatives.

  Sometimes it is useful to cut off input method processing so that the
characters you have just entered will not combine with subsequent
characters.  For example, in input method @code{latin-1-postfix}, the
520
sequence @kbd{o ^} combines to form an @samp{o} with an accent.  What if
Glenn Morris's avatar
Glenn Morris committed
521 522 523
you want to enter them as separate characters?

  One way is to type the accent twice; this is a special feature for
524 525 526 527
entering the separate letter and accent.  For example, @kbd{o ^ ^} gives
you the two characters @samp{o^}.  Another way is to type another letter
after the @kbd{o}---something that won't combine with that---and
immediately delete it.  For example, you could type @kbd{o o @key{DEL}
528 529 530 531
^} to get separate @samp{o} and @samp{^}.  Another method, more
general but not quite as easy to type, is to use @kbd{C-\ C-\} between
two characters to stop them from combining.  This is the command
@kbd{C-\} (@code{toggle-input-method}) used twice.
Glenn Morris's avatar
Glenn Morris committed
532 533 534 535 536 537 538 539 540 541 542 543
@ifnottex
@xref{Select Input Method}.
@end ifnottex

@cindex incremental search, input method interference
  @kbd{C-\ C-\} is especially useful inside an incremental search,
because it stops waiting for more characters to combine, and starts
searching for what you have already entered.

  To find out how to input the character after point using the current
input method, type @kbd{C-u C-x =}.  @xref{Position Info}.

544 545
@c TODO: document complex-only/default/t of
@c @code{input-method-verbose-flag}
Glenn Morris's avatar
Glenn Morris committed
546 547 548 549 550 551 552 553 554 555 556
@vindex input-method-verbose-flag
@vindex input-method-highlight-flag
  The variables @code{input-method-highlight-flag} and
@code{input-method-verbose-flag} control how input methods explain
what is happening.  If @code{input-method-highlight-flag} is
non-@code{nil}, the partial sequence is highlighted in the buffer (for
most input methods---some disable this feature).  If
@code{input-method-verbose-flag} is non-@code{nil}, the list of
possible characters to type next is displayed in the echo area (but
not when you are in the minibuffer).

557 558 559 560 561 562 563 564 565
@vindex quail-activate-hook
@findex quail-translation-keymap
  You can modify how an input method works by making your changes in a
function that you add to the hook variable @code{quail-activate-hook}.
@xref{Hooks}.  For example, you can redefine some of the input
method's keys by defining key bindings in the keymap returned by the
function @code{quail-translation-keymap}, using @code{define-key}.
@xref{Init Rebinding}.

566
  Another facility for typing characters not on your keyboard is by
Chong Yidong's avatar
Chong Yidong committed
567
using @kbd{C-x 8 @key{RET}} (@code{insert-char}) to insert a single
568 569 570
character based on its Unicode name or code-point; see @ref{Inserting
Text}.

Glenn Morris's avatar
Glenn Morris committed
571 572 573 574 575
@node Select Input Method
@section Selecting an Input Method

@table @kbd
@item C-\
576
Enable or disable use of the selected input method (@code{toggle-input-method}).
Glenn Morris's avatar
Glenn Morris committed
577 578

@item C-x @key{RET} C-\ @var{method} @key{RET}
579
Select a new input method for the current buffer (@code{set-input-method}).
Glenn Morris's avatar
Glenn Morris committed
580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596

@item C-h I @var{method} @key{RET}
@itemx C-h C-\ @var{method} @key{RET}
@findex describe-input-method
@kindex C-h I
@kindex C-h C-\
Describe the input method @var{method} (@code{describe-input-method}).
By default, it describes the current input method (if any).  This
description should give you the full details of how to use any
particular input method.

@item M-x list-input-methods
Display a list of all the supported input methods.
@end table

@findex set-input-method
@vindex current-input-method
597
@kindex C-x RET C-\
Glenn Morris's avatar
Glenn Morris committed
598 599 600 601 602 603 604 605 606 607 608 609 610 611 612
  To choose an input method for the current buffer, use @kbd{C-x
@key{RET} C-\} (@code{set-input-method}).  This command reads the
input method name from the minibuffer; the name normally starts with the
language environment that it is meant to be used with.  The variable
@code{current-input-method} records which input method is selected.

@findex toggle-input-method
@kindex C-\
  Input methods use various sequences of @acronym{ASCII} characters to
stand for non-@acronym{ASCII} characters.  Sometimes it is useful to
turn off the input method temporarily.  To do this, type @kbd{C-\}
(@code{toggle-input-method}).  To reenable the input method, type
@kbd{C-\} again.

  If you type @kbd{C-\} and you have not yet selected an input method,
613
it prompts you to specify one.  This has the same effect as using
Glenn Morris's avatar
Glenn Morris committed
614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646
@kbd{C-x @key{RET} C-\} to specify an input method.

  When invoked with a numeric argument, as in @kbd{C-u C-\},
@code{toggle-input-method} always prompts you for an input method,
suggesting the most recently selected one as the default.

@vindex default-input-method
  Selecting a language environment specifies a default input method for
use in various buffers.  When you have a default input method, you can
select it in the current buffer by typing @kbd{C-\}.  The variable
@code{default-input-method} specifies the default input method
(@code{nil} means there is none).

  In some language environments, which support several different input
methods, you might want to use an input method different from the
default chosen by @code{set-language-environment}.  You can instruct
Emacs to select a different default input method for a certain
language environment, if you wish, by using
@code{set-language-environment-hook} (@pxref{Language Environments,
set-language-environment-hook}).  For example:

@lisp
(defun my-chinese-setup ()
  "Set up my private Chinese environment."
  (if (equal current-language-environment "Chinese-GB")
      (setq default-input-method "chinese-tonepy")))
(add-hook 'set-language-environment-hook 'my-chinese-setup)
@end lisp

@noindent
This sets the default input method to be @code{chinese-tonepy}
whenever you choose a Chinese-GB language environment.

647 648 649 650 651 652 653 654 655
You can instruct Emacs to activate a certain input method
automatically.  For example:

@lisp
(add-hook 'text-mode-hook
  (lambda () (set-input-method "german-prefix")))
@end lisp

@noindent
Paul Eggert's avatar
Paul Eggert committed
656
This automatically activates the input method @code{german-prefix} in
657 658
Text mode.

Glenn Morris's avatar
Glenn Morris committed
659 660 661 662 663 664 665 666 667 668 669
@findex quail-set-keyboard-layout
  Some input methods for alphabetic scripts work by (in effect)
remapping the keyboard to emulate various keyboard layouts commonly used
for those scripts.  How to do this remapping properly depends on your
actual keyboard layout.  To specify which layout your keyboard has, use
the command @kbd{M-x quail-set-keyboard-layout}.

@findex quail-show-key
  You can use the command @kbd{M-x quail-show-key} to show what key (or
key sequence) to type in order to input the character following point,
using the selected keyboard layout.  The command @kbd{C-u C-x =} also
670
shows that information, in addition to other information about the
Glenn Morris's avatar
Glenn Morris committed
671 672 673
character.

@findex list-input-methods
674 675 676
  @kbd{M-x list-input-methods} displays a list of all the supported
input methods.  The list gives information about each input method,
including the string that stands for it in the mode line.
Glenn Morris's avatar
Glenn Morris committed
677 678 679 680 681 682 683 684 685 686 687 688 689 690

@node Coding Systems
@section Coding Systems
@cindex coding systems

  Users of various languages have established many more-or-less standard
coding systems for representing them.  Emacs does not use these coding
systems internally; instead, it converts from various coding systems to
its own system when reading data, and converts the internal coding
system to other coding systems when writing data.  Conversion is
possible in reading or writing files, in sending or receiving from the
terminal, and in exchanging data with subprocesses.

  Emacs assigns a name to each coding system.  Most coding systems are
691 692 693 694 695
used for one language, and the name of the coding system starts with
the language name.  Some coding systems are used for several
languages; their names usually start with @samp{iso}.  There are also
special coding systems, such as @code{no-conversion}, @code{raw-text},
and @code{emacs-internal}.
Glenn Morris's avatar
Glenn Morris committed
696 697 698 699 700 701 702 703 704

@cindex international files from DOS/Windows systems
  A special class of coding systems, collectively known as
@dfn{codepages}, is designed to support text encoded by MS-Windows and
MS-DOS software.  The names of these coding systems are
@code{cp@var{nnnn}}, where @var{nnnn} is a 3- or 4-digit number of the
codepage.  You can use these encodings just like any other coding
system; for example, to visit a file encoded in codepage 850, type
@kbd{C-x @key{RET} c cp850 @key{RET} C-x C-f @var{filename}
705
@key{RET}}.
Glenn Morris's avatar
Glenn Morris committed
706 707 708 709

  In addition to converting various representations of non-@acronym{ASCII}
characters, a coding system can perform end-of-line conversion.  Emacs
handles three different conventions for how to separate lines in a file:
710 711
newline (Unix), carriage return followed by linefeed (DOS), and just
carriage return (Mac).
Glenn Morris's avatar
Glenn Morris committed
712 713 714

@table @kbd
@item C-h C @var{coding} @key{RET}
715
Describe coding system @var{coding} (@code{describe-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
716 717

@item C-h C @key{RET}
Eli Zaretskii's avatar
Eli Zaretskii committed
718
Describe the coding systems currently in use (@code{describe-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747

@item M-x list-coding-systems
Display a list of all the supported coding systems.
@end table

@kindex C-h C
@findex describe-coding-system
  The command @kbd{C-h C} (@code{describe-coding-system}) displays
information about particular coding systems, including the end-of-line
conversion specified by those coding systems.  You can specify a coding
system name as the argument; alternatively, with an empty argument, it
describes the coding systems currently selected for various purposes,
both in the current buffer and as the defaults, and the priority list
for recognizing coding systems (@pxref{Recognize Coding}).

@findex list-coding-systems
  To display a list of all the supported coding systems, type @kbd{M-x
list-coding-systems}.  The list gives information about each coding
system, including the letter that stands for it in the mode line
(@pxref{Mode Line}).

@cindex end-of-line conversion
@cindex line endings
@cindex MS-DOS end-of-line conversion
@cindex Macintosh end-of-line conversion
  Each of the coding systems that appear in this list---except for
@code{no-conversion}, which means no conversion of any kind---specifies
how and whether to convert printing characters, but leaves the choice of
end-of-line conversion to be decided based on the contents of each file.
748 749
For example, if the file appears to use the sequence carriage return
and linefeed to separate lines, DOS end-of-line conversion will be used.
Glenn Morris's avatar
Glenn Morris committed
750

751
  Each of the listed coding systems has three variants, which specify
Glenn Morris's avatar
Glenn Morris committed
752 753 754 755 756 757
exactly what to do for end-of-line conversion:

@table @code
@item @dots{}-unix
Don't do any end-of-line conversion; assume the file uses
newline to separate lines.  (This is the convention normally used
758
on Unix and GNU systems, and macOS.)
Glenn Morris's avatar
Glenn Morris committed
759 760

@item @dots{}-dos
761 762 763 764 765 766
Assume the file uses carriage return followed by linefeed to separate
lines, and do the appropriate conversion.  (This is the convention
normally used on Microsoft systems.@footnote{It is also specified for
MIME @samp{text/*} bodies and in other network transport contexts.  It
is different from the SGML reference syntax record-start/record-end
format, which Emacs doesn't support directly.})
Glenn Morris's avatar
Glenn Morris committed
767 768

@item @dots{}-mac
769
Assume the file uses carriage return to separate lines, and do the
770 771
appropriate conversion.  (This was the convention used in Classic Mac
OS.)
Glenn Morris's avatar
Glenn Morris committed
772 773 774 775 776 777 778 779 780 781 782 783 784 785 786
@end table

  These variant coding systems are omitted from the
@code{list-coding-systems} display for brevity, since they are entirely
predictable.  For example, the coding system @code{iso-latin-1} has
variants @code{iso-latin-1-unix}, @code{iso-latin-1-dos} and
@code{iso-latin-1-mac}.

@cindex @code{undecided}, coding system
  The coding systems @code{unix}, @code{dos}, and @code{mac} are
aliases for @code{undecided-unix}, @code{undecided-dos}, and
@code{undecided-mac}, respectively.  These coding systems specify only
the end-of-line conversion, and leave the character code conversion to
be deduced from the text itself.

787
@cindex @code{raw-text}, coding system
Glenn Morris's avatar
Glenn Morris committed
788
  The coding system @code{raw-text} is good for a file which is mainly
789
@acronym{ASCII} text, but may contain byte values above 127 that are
Glenn Morris's avatar
Glenn Morris committed
790 791 792 793 794 795 796 797
not meant to encode non-@acronym{ASCII} characters.  With
@code{raw-text}, Emacs copies those byte values unchanged, and sets
@code{enable-multibyte-characters} to @code{nil} in the current buffer
so that they will be interpreted properly.  @code{raw-text} handles
end-of-line conversion in the usual way, based on the data
encountered, and has the usual three variants to specify the kind of
end-of-line conversion to use.

798
@cindex @code{no-conversion}, coding system
Glenn Morris's avatar
Glenn Morris committed
799 800 801 802 803 804 805 806 807 808 809
  In contrast, the coding system @code{no-conversion} specifies no
character code conversion at all---none for non-@acronym{ASCII} byte values and
none for end of line.  This is useful for reading or writing binary
files, tar files, and other files that must be examined verbatim.  It,
too, sets @code{enable-multibyte-characters} to @code{nil}.

  The easiest way to edit a file with no conversion of any kind is with
the @kbd{M-x find-file-literally} command.  This uses
@code{no-conversion}, and also suppresses other Emacs features that
might convert the file contents before you see them.  @xref{Visiting}.

810
@cindex @code{emacs-internal}, coding system
811 812 813 814 815 816
  The coding system @code{emacs-internal} (or @code{utf-8-emacs},
which is equivalent) means that the file contains non-@acronym{ASCII}
characters stored with the internal Emacs encoding.  This coding
system handles end-of-line conversion based on the data encountered,
and has the usual three variants to specify the kind of end-of-line
conversion.
Glenn Morris's avatar
Glenn Morris committed
817 818 819 820

@node Recognize Coding
@section Recognizing Coding Systems

821 822 823 824 825
  Whenever Emacs reads a given piece of text, it tries to recognize
which coding system to use.  This applies to files being read, output
from subprocesses, text from X selections, etc.  Emacs can select the
right coding system automatically most of the time---once you have
specified your preferences.
Glenn Morris's avatar
Glenn Morris committed
826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878

  Some coding systems can be recognized or distinguished by which byte
sequences appear in the data.  However, there are coding systems that
cannot be distinguished, not even potentially.  For example, there is no
way to distinguish between Latin-1 and Latin-2; they use the same byte
values with different meanings.

  Emacs handles this situation by means of a priority list of coding
systems.  Whenever Emacs reads a file, if you do not specify the coding
system to use, Emacs checks the data against each coding system,
starting with the first in priority and working down the list, until it
finds a coding system that fits the data.  Then it converts the file
contents assuming that they are represented in this coding system.

  The priority list of coding systems depends on the selected language
environment (@pxref{Language Environments}).  For example, if you use
French, you probably want Emacs to prefer Latin-1 to Latin-2; if you use
Czech, you probably want Latin-2 to be preferred.  This is one of the
reasons to specify a language environment.

@findex prefer-coding-system
  However, you can alter the coding system priority list in detail
with the command @kbd{M-x prefer-coding-system}.  This command reads
the name of a coding system from the minibuffer, and adds it to the
front of the priority list, so that it is preferred to all others.  If
you use this command several times, each use adds one element to the
front of the priority list.

  If you use a coding system that specifies the end-of-line conversion
type, such as @code{iso-8859-1-dos}, what this means is that Emacs
should attempt to recognize @code{iso-8859-1} with priority, and should
use DOS end-of-line conversion when it does recognize @code{iso-8859-1}.

@vindex file-coding-system-alist
  Sometimes a file name indicates which coding system to use for the
file.  The variable @code{file-coding-system-alist} specifies this
correspondence.  There is a special function
@code{modify-coding-system-alist} for adding elements to this list.  For
example, to read and write all @samp{.txt} files using the coding system
@code{chinese-iso-8bit}, you can execute this Lisp expression:

@smallexample
(modify-coding-system-alist 'file "\\.txt\\'" 'chinese-iso-8bit)
@end smallexample

@noindent
The first argument should be @code{file}, the second argument should be
a regular expression that determines which files this applies to, and
the third argument says which coding system to use for these files.

@vindex inhibit-eol-conversion
@cindex DOS-style end-of-line display
  Emacs recognizes which kind of end-of-line conversion to use based on
879 880 881 882 883 884 885 886 887
the contents of the file: if it sees only carriage returns, or only
carriage return followed by linefeed sequences, then it chooses the
end-of-line conversion accordingly.  You can inhibit the automatic use
of end-of-line conversion by setting the variable
@code{inhibit-eol-conversion} to non-@code{nil}.  If you do that,
DOS-style files will be displayed with the @samp{^M} characters
visible in the buffer; some people prefer this to the more subtle
@samp{(DOS)} end-of-line type indication near the left edge of the
mode line (@pxref{Mode Line, eol-mnemonic}).
Glenn Morris's avatar
Glenn Morris committed
888 889 890

@vindex inhibit-iso-escape-detection
@cindex escape sequences in files
Eli Zaretskii's avatar
Eli Zaretskii committed
891
  By default, the automatic detection of the coding system is sensitive to
Glenn Morris's avatar
Glenn Morris committed
892 893 894 895 896 897 898 899 900 901 902 903 904 905
escape sequences.  If Emacs sees a sequence of characters that begin
with an escape character, and the sequence is valid as an ISO-2022
code, that tells Emacs to use one of the ISO-2022 encodings to decode
the file.

  However, there may be cases that you want to read escape sequences
in a file as is.  In such a case, you can set the variable
@code{inhibit-iso-escape-detection} to non-@code{nil}.  Then the code
detection ignores any escape sequences, and never uses an ISO-2022
encoding.  The result is that all escape sequences become visible in
the buffer.

  The default value of @code{inhibit-iso-escape-detection} is
@code{nil}.  We recommend that you not change it permanently, only for
906
one specific operation.  That's because some Emacs Lisp source files
Glenn Morris's avatar
Glenn Morris committed
907 908 909 910
in the Emacs distribution contain non-@acronym{ASCII} characters encoded in the
coding system @code{iso-2022-7bit}, and they won't be
decoded correctly when you visit those files if you suppress the
escape sequence detection.
911
@c I count a grand total of 3 such files, so is the above really true?
Glenn Morris's avatar
Glenn Morris committed
912 913 914

@vindex auto-coding-alist
@vindex auto-coding-regexp-alist
915 916
  The variables @code{auto-coding-alist} and
@code{auto-coding-regexp-alist} are
Glenn Morris's avatar
Glenn Morris committed
917
the strongest way to specify the coding system for certain patterns of
918 919
file names, or for files containing certain patterns, respectively.
These variables even override @samp{-*-coding:-*-} tags in the file
920
itself (@pxref{Specify Coding}).  For example, Emacs
Glenn Morris's avatar
Glenn Morris committed
921 922 923
uses @code{auto-coding-alist} for tar and archive files, to prevent it
from being confused by a @samp{-*-coding:-*-} tag in a member of the
archive and thinking it applies to the archive file as a whole.
924 925
@ignore
@c This describes old-style BABYL files, which are no longer relevant.
Glenn Morris's avatar
Glenn Morris committed
926 927
Likewise, Emacs uses @code{auto-coding-regexp-alist} to ensure that
RMAIL files, whose names in general don't match any particular
928 929 930 931 932 933
pattern, are decoded correctly.
@end ignore

@vindex auto-coding-functions
  Another way to specify a coding system is with the variable
@code{auto-coding-functions}.  For example, one of the builtin
Glenn Morris's avatar
Glenn Morris committed
934
@code{auto-coding-functions} detects the encoding for XML files.
935 936
Unlike the previous two, this variable does not override any
@samp{-*-coding:-*-} tag.
Glenn Morris's avatar
Glenn Morris committed
937 938 939 940 941

@node Specify Coding
@section Specifying a File's Coding System

  If Emacs recognizes the encoding of a file incorrectly, you can
Chong Yidong's avatar
Chong Yidong committed
942 943 944 945 946 947
reread the file using the correct coding system with @kbd{C-x
@key{RET} r} (@code{revert-buffer-with-coding-system}).  This command
prompts for the coding system to use.  To see what coding system Emacs
actually used to decode the file, look at the coding system mnemonic
letter near the left edge of the mode line (@pxref{Mode Line}), or
type @kbd{C-h C} (@code{describe-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
948 949 950 951 952 953 954 955

@vindex coding
  You can specify the coding system for a particular file in the file
itself, using the @w{@samp{-*-@dots{}-*-}} construct at the beginning,
or a local variables list at the end (@pxref{File Variables}).  You do
this by defining a value for the ``variable'' named @code{coding}.
Emacs does not really have a variable @code{coding}; instead of
setting a variable, this uses the specified coding system for the
Eli Zaretskii's avatar
Eli Zaretskii committed
956
file.  For example, @w{@samp{-*-mode: C; coding: latin-1; -*-}} specifies
Glenn Morris's avatar
Glenn Morris committed
957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973
use of the Latin-1 coding system, as well as C mode.  When you specify
the coding explicitly in the file, that overrides
@code{file-coding-system-alist}.

@node Output Coding
@section Choosing Coding Systems for Output

@vindex buffer-file-coding-system
  Once Emacs has chosen a coding system for a buffer, it stores that
coding system in @code{buffer-file-coding-system}.  That makes it the
default for operations that write from this buffer into a file, such
as @code{save-buffer} and @code{write-region}.  You can specify a
different coding system for further file output from the buffer using
@code{set-buffer-file-coding-system} (@pxref{Text Coding}).

  You can insert any character Emacs supports into any Emacs buffer,
but most coding systems can only handle a subset of these characters.
974 975 976 977
Therefore, it's possible that the characters you insert cannot be
encoded with the coding system that will be used to save the buffer.
For example, you could visit a text file in Polish, encoded in
@code{iso-8859-2}, and add some Russian words to it.  When you save
Glenn Morris's avatar
Glenn Morris committed
978 979 980 981 982 983
that buffer, Emacs cannot use the current value of
@code{buffer-file-coding-system}, because the characters you added
cannot be encoded by that coding system.

  When that happens, Emacs tries the most-preferred coding system (set
by @kbd{M-x prefer-coding-system} or @kbd{M-x
984 985 986 987 988
set-language-environment}).  If that coding system can safely encode
all of the characters in the buffer, Emacs uses it, and stores its
value in @code{buffer-file-coding-system}.  Otherwise, Emacs displays
a list of coding systems suitable for encoding the buffer's contents,
and asks you to choose one of those coding systems.
Glenn Morris's avatar
Glenn Morris committed
989 990 991

  If you insert the unsuitable characters in a mail message, Emacs
behaves a bit differently.  It additionally checks whether the
992
@c What determines this?
Glenn Morris's avatar
Glenn Morris committed
993
most-preferred coding system is recommended for use in MIME messages;
994 995 996 997 998
if not, it informs you of this fact and prompts you for another coding
system.  This is so you won't inadvertently send a message encoded in
a way that your recipient's mail software will have difficulty
decoding.  (You can still use an unsuitable coding system if you enter
its name at the prompt.)
Glenn Morris's avatar
Glenn Morris committed
999

1000
@c It seems that select-message-coding-system does this.
1001
@c Both sendmail.el and smptmail.el call it; i.e., smtpmail.el still
1002
@c obeys sendmail-coding-system.
Glenn Morris's avatar
Glenn Morris committed
1003
@vindex sendmail-coding-system
1004
  When you send a mail message (@pxref{Sending Mail}),
1005
Emacs has four different ways to determine the coding system to use
1006
for encoding the message text.  It first tries the buffer's own value of
1007 1008
@code{buffer-file-coding-system}, if that is non-@code{nil}.
Otherwise, it uses the value of @code{sendmail-coding-system}, if that
1009 1010 1011 1012 1013 1014
is non-@code{nil}.  Thirdly, it uses the value of
@code{default-sendmail-coding-system}.
If all of these three values are @code{nil}, Emacs encodes outgoing
mail using the default coding system for new files (i.e., the
default value of @code{buffer-file-coding-system}), which is
controlled by your choice of language environment.
Glenn Morris's avatar
Glenn Morris committed
1015 1016 1017 1018 1019 1020 1021 1022 1023 1024

@node Text Coding
@section Specifying a Coding System for File Text

  In cases where Emacs does not automatically choose the right coding
system for a file's contents, you can use these commands to specify
one:

@table @kbd
@item C-x @key{RET} f @var{coding} @key{RET}
1025 1026
Use coding system @var{coding} to save or revisit the file in
the current buffer (@code{set-buffer-file-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1027 1028 1029

@item C-x @key{RET} c @var{coding} @key{RET}
Specify coding system @var{coding} for the immediately following
Chong Yidong's avatar
Chong Yidong committed
1030
command (@code{universal-coding-system-argument}).
Glenn Morris's avatar
Glenn Morris committed
1031 1032

@item C-x @key{RET} r @var{coding} @key{RET}
Chong Yidong's avatar
Chong Yidong committed
1033 1034
Revisit the current file using the coding system @var{coding}
(@code{revert-buffer-with-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1035 1036 1037 1038 1039 1040

@item M-x recode-region @key{RET} @var{right} @key{RET} @var{wrong} @key{RET}
Convert a region that was decoded using coding system @var{wrong},
decoding it using coding system @var{right} instead.
@end table

1041
@kindex C-x RET f
Glenn Morris's avatar
Glenn Morris committed
1042 1043 1044
@findex set-buffer-file-coding-system
  The command @kbd{C-x @key{RET} f}
(@code{set-buffer-file-coding-system}) sets the file coding system for
1045
the current buffer (i.e., the coding system to use when saving or
1046 1047
reverting the file).  You specify which coding system using the
minibuffer.  You can also invoke this command by clicking with
1048
@kbd{mouse-3} on the coding system indicator in the mode line
1049 1050 1051 1052 1053 1054
(@pxref{Mode Line}).

  If you specify a coding system that cannot handle all the characters
in the buffer, Emacs will warn you about the troublesome characters,
and ask you to choose another coding system, when you try to save the
buffer (@pxref{Output Coding}).
Glenn Morris's avatar
Glenn Morris committed
1055 1056 1057 1058 1059

@cindex specify end-of-line conversion
  You can also use this command to specify the end-of-line conversion
(@pxref{Coding Systems, end-of-line conversion}) for encoding the
current buffer.  For example, @kbd{C-x @key{RET} f dos @key{RET}} will
1060
cause Emacs to save the current buffer's text with DOS-style
1061
carriage return followed by linefeed line endings.
Glenn Morris's avatar
Glenn Morris committed
1062

1063
@kindex C-x RET c
Glenn Morris's avatar
Glenn Morris committed
1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089
@findex universal-coding-system-argument
  Another way to specify the coding system for a file is when you visit
the file.  First use the command @kbd{C-x @key{RET} c}
(@code{universal-coding-system-argument}); this command uses the
minibuffer to read a coding system name.  After you exit the minibuffer,
the specified coding system is used for @emph{the immediately following
command}.

  So if the immediately following command is @kbd{C-x C-f}, for example,
it reads the file using that coding system (and records the coding
system for when you later save the file).  Or if the immediately following
command is @kbd{C-x C-w}, it writes the file using that coding system.
When you specify the coding system for saving in this way, instead
of with @kbd{C-x @key{RET} f}, there is no warning if the buffer
contains characters that the coding system cannot handle.

  Other file commands affected by a specified coding system include
@kbd{C-x i} and @kbd{C-x C-v}, as well as the other-window variants
of @kbd{C-x C-f}.  @kbd{C-x @key{RET} c} also affects commands that
start subprocesses, including @kbd{M-x shell} (@pxref{Shell}).  If the
immediately following command does not use the coding system, then
@kbd{C-x @key{RET} c} ultimately has no effect.

  An easy way to visit a file with no conversion is with the @kbd{M-x
find-file-literally} command.  @xref{Visiting}.

1090 1091 1092 1093 1094
  The default value of the variable @code{buffer-file-coding-system}
specifies the choice of coding system to use when you create a new file.
It applies when you find a new file, and when you create a buffer and
then save it in a file.  Selecting a language environment typically sets
this variable to a good choice of default coding system for that language
Glenn Morris's avatar
Glenn Morris committed
1095 1096
environment.

1097
@kindex C-x RET r
Glenn Morris's avatar
Glenn Morris committed
1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119
@findex revert-buffer-with-coding-system
  If you visit a file with a wrong coding system, you can correct this
with @kbd{C-x @key{RET} r} (@code{revert-buffer-with-coding-system}).
This visits the current file again, using a coding system you specify.

@findex recode-region
  If a piece of text has already been inserted into a buffer using the
wrong coding system, you can redo the decoding of it using @kbd{M-x
recode-region}.  This prompts you for the proper coding system, then
for the wrong coding system that was actually used, and does the
conversion.  It first encodes the region using the wrong coding system,
then decodes it again using the proper coding system.

@node Communication Coding
@section Coding Systems for Interprocess Communication

  This section explains how to specify coding systems for use
in communication with other processes.

@table @kbd
@item C-x @key{RET} x @var{coding} @key{RET}
Use coding system @var{coding} for transferring selections to and from
1120
other graphical applications (@code{set-selection-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1121 1122 1123

@item C-x @key{RET} X @var{coding} @key{RET}
Use coding system @var{coding} for transferring @emph{one}
1124
selection---the next one---to or from another graphical application
Chong Yidong's avatar
Chong Yidong committed
1125
(@code{set-next-selection-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1126 1127 1128

@item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
Use coding systems @var{input-coding} and @var{output-coding} for
Chong Yidong's avatar
Chong Yidong committed
1129 1130
subprocess input and output in the current buffer
(@code{set-buffer-process-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1131 1132
@end table

1133 1134
@kindex C-x RET x
@kindex C-x RET X
Glenn Morris's avatar
Glenn Morris committed
1135 1136 1137 1138 1139 1140 1141 1142 1143 1144
@findex set-selection-coding-system
@findex set-next-selection-coding-system
  The command @kbd{C-x @key{RET} x} (@code{set-selection-coding-system})
specifies the coding system for sending selected text to other windowing
applications, and for receiving the text of selections made in other
applications.  This command applies to all subsequent selections, until
you override it by using the command again.  The command @kbd{C-x
@key{RET} X} (@code{set-next-selection-coding-system}) specifies the
coding system for the next selection made in Emacs or read by Emacs.

1145
@vindex x-select-request-type
1146 1147 1148
  The variable @code{x-select-request-type} specifies the data type to
request from the X Window System for receiving text selections from
other applications.  If the value is @code{nil} (the default), Emacs
1149
tries @code{UTF8_STRING} and @code{COMPOUND_TEXT}, in this order, and
1150 1151 1152 1153 1154 1155 1156 1157
uses various heuristics to choose the more appropriate of the two
results; if none of these succeed, Emacs falls back on @code{STRING}.
If the value of @code{x-select-request-type} is one of the symbols
@code{COMPOUND_TEXT}, @code{UTF8_STRING}, @code{STRING}, or
@code{TEXT}, Emacs uses only that request type.  If the value is a
list of some of these symbols, Emacs tries only the request types in
the list, in order, until one of them succeeds, or until the list is
exhausted.
1158

1159
@kindex C-x RET p
Glenn Morris's avatar
Glenn Morris committed
1160 1161 1162 1163 1164 1165 1166 1167
@findex set-buffer-process-coding-system
  The command @kbd{C-x @key{RET} p} (@code{set-buffer-process-coding-system})
specifies the coding system for input and output to a subprocess.  This
command applies to the current buffer; normally, each subprocess has its
own buffer, and thus you can use this command to specify translation to
and from a particular subprocess by giving the command in the
corresponding buffer.

Chong Yidong's avatar
Chong Yidong committed
1168 1169 1170 1171
  You can also use @kbd{C-x @key{RET} c}
(@code{universal-coding-system-argument}) just before the command that
runs or starts a subprocess, to specify the coding system for
communicating with that subprocess.  @xref{Text Coding}.
Glenn Morris's avatar
Glenn Morris committed
1172 1173 1174 1175 1176 1177 1178 1179 1180

  The default for translation of process input and output depends on the
current language environment.

@vindex locale-coding-system
@cindex decoding non-@acronym{ASCII} keyboard input on X
  The variable @code{locale-coding-system} specifies a coding system
to use when encoding and decoding system strings such as system error
messages and @code{format-time-string} formats and time stamps.  That
1181
coding system is also used for decoding non-@acronym{ASCII} keyboard
1182 1183 1184
input on the X Window System and for encoding text sent to the
standard output and error streams when in batch mode.  You should
choose a coding system that is compatible
Glenn Morris's avatar
Glenn Morris committed
1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196
with the underlying system's text representation, which is normally
specified by one of the environment variables @env{LC_ALL},
@env{LC_CTYPE}, and @env{LANG}.  (The first one, in the order
specified above, whose value is nonempty is the one that determines
the text representation.)

@node File Name Coding
@section Coding Systems for File Names

@table @kbd
@item C-x @key{RET} F @var{coding} @key{RET}
Use coding system @var{coding} for encoding and decoding file
1197
names (@code{set-file-name-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1198 1199 1200
@end table

@findex set-file-name-coding-system
1201
@kindex C-x RET F
1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214
@cindex file names with non-@acronym{ASCII} characters
  The command @kbd{C-x @key{RET} F} (@code{set-file-name-coding-system})
specifies a coding system to use for encoding file @emph{names}.  It
has no effect on reading and writing the @emph{contents} of files.

@vindex file-name-coding-system
  In fact, all this command does is set the value of the variable
@code{file-name-coding-system}.  If you set the variable to a coding
system name (as a Lisp symbol or a string), Emacs encodes file names
using that coding system for all file operations.  This makes it
possible to use non-@acronym{ASCII} characters in file names---or, at
least, those non-@acronym{ASCII} characters that the specified coding
system can encode.
Glenn Morris's avatar
Glenn Morris committed
1215 1216

  If @code{file-name-coding-system} is @code{nil}, Emacs uses a
1217 1218 1219
default coding system determined by the selected language environment,
and stored in the @code{default-file-name-coding-system} variable.
@c FIXME?  Is this correct?  What is the "default language environment"?
1220 1221 1222
In the default language environment, non-@acronym{ASCII} characters in
file names are not encoded specially; they appear in the file system
using the internal Emacs representation.
Glenn Morris's avatar
Glenn Morris committed
1223

Eli Zaretskii's avatar
Eli Zaretskii committed
1224 1225 1226
@cindex file-name encoding, MS-Windows
@vindex w32-unicode-filenames
  When Emacs runs on MS-Windows versions that are descendants of the
Eli Zaretskii's avatar
Eli Zaretskii committed
1227 1228 1229 1230 1231 1232 1233
NT family (Windows 2000, XP, and all the later versions), the value of
@code{file-name-coding-system} is largely ignored, as Emacs by default
uses APIs that allow passing Unicode file names directly.  By
contrast, on Windows 9X, file names are encoded using
@code{file-name-coding-system}, which should be set to the codepage
(@pxref{Coding Systems, codepage}) pertinent for the current system
locale.  The value of the variable @code{w32-unicode-filenames}
Eli Zaretskii's avatar
Eli Zaretskii committed
1234 1235 1236 1237 1238
controls whether Emacs uses the Unicode APIs when it calls OS
functions that accept file names.  This variable is set by the startup
code to @code{nil} on Windows 9X, and to @code{t} on newer versions of
MS-Windows.

Glenn Morris's avatar
Glenn Morris committed
1239 1240 1241 1242 1243 1244
  @strong{Warning:} if you change @code{file-name-coding-system} (or the
language environment) in the middle of an Emacs session, problems can
result if you have already visited files whose names were encoded using
the earlier coding system and cannot be encoded (or are encoded
differently) under the new coding system.  If you try to save one of
these buffers under the visited file name, saving may use the wrong file
1245
name, or it may encounter an error.  If such a problem happens, use @kbd{C-x
Glenn Morris's avatar
Glenn Morris committed
1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258
C-w} to specify a new file name for that buffer.

@findex recode-file-name
  If a mistake occurs when encoding a file name, use the command
@kbd{M-x recode-file-name} to change the file name's coding
system.  This prompts for an existing file name, its old coding
system, and the coding system to which you wish to convert.

@node Terminal Coding
@section Coding Systems for Terminal I/O

@table @kbd
@item C-x @key{RET} t @var{coding} @key{RET}
Chong Yidong's avatar
Chong Yidong committed
1259 1260
Use coding system @var{coding} for terminal output
(@code{set-terminal-coding-system}).
1261 1262 1263 1264

@item C-x @key{RET} k @var{coding} @key{RET}
Use coding system @var{coding} for keyboard input
(@code{set-keyboard-coding-system}).
Glenn Morris's avatar
Glenn Morris committed
1265 1266
@end table

1267