mail-extr.el 67.1 KB
Newer Older
Eric S. Raymond's avatar
Eric S. Raymond committed
1 2
;;; mail-extr.el --- extract full name and address from RFC 822 mail header.

3
;; Copyright (C) 1991, 1992, 1993, 1994 Free Software Foundation, Inc.
Eric S. Raymond's avatar
Eric S. Raymond committed
4

Eric S. Raymond's avatar
Eric S. Raymond committed
5
;; Author: Joe Wells <jbw@cs.bu.edu>
6 7
;; Maintainer: Jamie Zawinski <jwz@lucid.com>
;; Version: 1.8
Eric S. Raymond's avatar
Eric S. Raymond committed
8 9 10 11 12 13
;; Keywords: mail

;; This file is part of GNU Emacs.

;; GNU Emacs is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
14
;; the Free Software Foundation; either version 2, or (at your option)
Eric S. Raymond's avatar
Eric S. Raymond committed
15 16 17 18 19 20 21 22 23 24 25 26 27
;; any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to
;; the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.

;;; Commentary:

28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
;; The entry point of this code is
;;
;;    mail-extract-address-components: (address)
;;  
;;    Given an RFC-822 ADDRESS, extract full name and canonical address.
;;    Returns a list of the form (FULL-NAME CANONICAL-ADDRESS).
;;    If no name can be extracted, FULL-NAME will be nil.
;;    ADDRESS may be a string or a buffer.  If it is a buffer, the visible 
;;     (narrowed) portion of the buffer will be interpreted as the address.
;;     (This feature exists so that the clever caller might be able to avoid
;;     consing a string.)
;;    If ADDRESS contains more than one RFC-822 address, only the first is
;;     returned.
;;
;; This code is more correct (and more heuristic) parser than the code in
;; rfc822.el.  And despite its size, it's fairly fast.
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
45
;; There are two main benefits:
46
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
47
;; 1. Higher probability of getting the correct full name for a human than
48
;;    any other package we know of.  (On the other hand, it will cheerfully
Eric S. Raymond's avatar
Eric S. Raymond committed
49 50
;;    mangle non-human names/comments.)
;; 2. Address part is put in a canonical form.
51 52 53 54
;;
;; The interface is not yet carved in stone; please give us suggestions.
;;
;; We have an extensive test-case collection of funny addresses if you want to
Eric S. Raymond's avatar
Eric S. Raymond committed
55
;; work with the code.  Developing this code requires frequent testing to
56 57 58 59 60 61 62 63 64
;; make sure you're not breaking functionality.  The test cases aren't included
;; because they are over 100K.
;;
;; If you find an address that mail-extr fails on, please send it to the 
;; maintainer along with what you think the correct results should be.  We do
;; not consider it a bug if mail-extr mangles a comment that does not
;; correspond to a real human full name, although we would prefer that 
;; mail-extr would return the comment as-is.
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
65
;; Features:
66
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
67
;; * Full name handling:
68
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
69 70 71 72 73
;;   * knows where full names can be found in an address.
;;   * avoids using empty comments and quoted text.
;;   * extracts full names from mailbox names.
;;   * recognizes common formats for comments after a full name.
;;   * puts a period and a space after each initial.
74 75
;;   * understands & referring to the mailbox name, capitalized.
;;   * strips name prefixes like "Prof.", etc.
Eric S. Raymond's avatar
Eric S. Raymond committed
76 77 78 79 80
;;   * understands what characters can occur in names (not just letters).
;;   * figures out middle initial from mailbox name.
;;   * removes funny nicknames.
;;   * keeps suffixes such as Jr., Sr., III, etc.
;;   * reorders "Last, First" type names.
81
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
82
;; * Address handling:
83
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
84 85 86 87 88 89 90 91 92
;;   * parses rfc822 quoted text, comments, and domain literals.
;;   * parses rfc822 multi-line headers.
;;   * does something reasonable with rfc822 GROUP addresses.
;;   * handles many rfc822 noncompliant and garbage addresses.
;;   * canonicalizes addresses (after stripping comments/phrases outside <>).
;;     * converts ! addresses into .UUCP and %-style addresses.
;;     * converts rfc822 ROUTE addresses to %-style addresses.
;;     * truncates %-style addresses at leftmost fully qualified domain name.
;;     * handles local relative precedence of ! vs. % and @ (untested).
93
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
94 95 96
;; It does almost no string creation.  It primarily uses the built-in
;; parsing routines with the appropriate syntax tables.  This should
;; result in greater speed.
97
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
98
;; TODO:
99
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120
;; * handle all test cases.  (This will take forever.)
;; * software to pick the correct header to use (eg., "Senders-Name:").
;; * multiple addresses in the "From:" header (almost all of the necessary
;;   code is there).
;; * flag to not treat `,' as an address separator.  (This is useful when
;;   there is a "From:" header but no "Sender:" header, because then there
;;   is only allowed to be one address.)
;; * mailbox name does not necessarily contain full name.
;; * fixing capitalization when it's all upper or lowercase.  (Hard!)
;; * some of the domain literal handling is missing.  (But I've never even
;;   seen one of these in a mail address, so maybe no big deal.)
;; * arrange to have syntax tables byte-compiled.
;; * speed hacks.
;; * delete unused variables.
;; * arrange for testing with different relative precedences of ! vs. @
;;   and %.
;; * insert documentation strings!
;; * handle X.400-gatewayed addresses according to RFC 1148.

;;; Change Log: 
;; 
121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183
;; Thu Feb 17 17:57:33 1994  Jamie Zawinski (jwz@lucid.com)
;;
;;	* merged with jbw's latest version
;;
;; Wed Feb  9 21:56:27 1994  Jamie Zawinski (jwz@lucid.com)
;;
;;      * high-bit chars in comments weren't treated as word syntax
;;
;; Sat Feb  5 03:13:40 1994  Jamie Zawinski (jwz@lucid.com)
;;
;;      * call replace-match with fixed-case arg
;;
;; Thu Dec 16 21:56:45 1993  Jamie Zawinski (jwz@lucid.com)
;;
;;      * some more cleanup, doc, added provide
;;
;; Tue Mar 23 21:23:18 1993  Joe Wells  (jbw at csd.bu.edu)
;; 
;; 	* Made mail-full-name-prefixes a user-customizable variable.
;;        Allow passing the address as a buffer as well as as a string.
;;        Allow [ and ] as name characters (Finnish character set).
;; 
;; Mon Mar 22 21:20:56 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Handle "null" addresses.  Handle = used for spacing in mailbox
;; 	  name.  Fix bug in handling of ROUTE-ADDR-type addresses that are
;; 	  missing their brackets.  Handle uppercase "JR".  Extract full
;; 	  names from X.400 addresses encoded in RFC-822.  Fix bug in
;;        handling of multiple addresses where first has trailing comment.
;;        Handle more kinds of telephone extension lead-ins.
;; 
;; Mon Mar 22 20:16:57 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Handle HZ encoding for embedding GB encoded chinese characters.
;; 
;; Mon Mar 22 00:46:12 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Fixed too broad matching of ham radio call signs.  Fixed bug in
;; 	  handling an unmatched ' in a name string.  Enhanced recognition
;; 	  of when . in the mailbox name terminates the name portion.
;; 	  Narrowed conversion of . to space to only the necessary
;; 	  situation.  Deal with VMS's stupid date stamps.  Handle a unique
;; 	  way of introducing an alternate address.  Fixed spacing bug I
;; 	  introduced in switching last name order.  Fixed bug in handling
;; 	  address with ! and % but no @.  Narrowed the cases in which
;; 	  certain trailing words are discarded.
;; 
;; Sun Mar 21 21:41:06 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Fixed bugs in handling GROUP addresses.  Certain words in the
;; 	  middle of a name no longer terminate it.  Handle LISTSERV list
;;        names.  Ignore comment field containing mailbox name.
;; 
;; Sun Mar 21 14:39:38 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Moved variant-method code back into main function.  Handle
;; 	underscores as spaces in comments.  Handle leading nickname.  Add
;; 	flag to ignore single-word names.  Other changes.
;; 
;; Mon Feb  1 22:23:31 1993  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Added in changes by Rod Whitby and Jamie Zawinski.  This
;;        includes the flag mail-extr-guess-middle-initial and the fix for
Richard M. Stallman's avatar
Richard M. Stallman committed
184 185
;;        handling multiple addresses correctly.  (Whitby just changed
;;	  a > to a <.)
186
;; 
Eric S. Raymond's avatar
Eric S. Raymond committed
187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206
;; Mon Apr  6 23:59:09 1992  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Cleaned up some more.  Release version 1.0 to world.
;; 
;; Sun Apr  5 19:39:08 1992  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Cleaned up full name extraction extensively.
;; 
;; Sun Feb  2 14:45:24 1992  Joe Wells  (jbw at bigbird.bu.edu)
;; 
;; 	* Total rewrite.  Integrated mail-canonicalize-address into
;; 	mail-extract-address-components.  Now handles GROUP addresses more
;; 	or less correctly.  Better handling of lots of different cases.
;; 
;; Fri Jun 14 19:39:50 1991
;;	* Created.

;;; Code:


207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; User configuration variable definitions.
;;

(defvar mail-extr-guess-middle-initial nil
  "*Whether to try to guess middle initial from mail address.
If true, then when we see an address like \"John Smith <jqs@host.com>\"
we will assume that \"John Q. Smith\" is the fellow's name.")

(defvar mail-extr-ignore-single-names t
  "*Whether to ignore a name that is just a single word.
If true, then when we see an address like \"Idiot <dumb@stupid.com>\"
we will act as though we couldn't find a full name in the address.")

;; Matches a leading title that is not part of the name (does not
;; contribute to uniquely identifying the person).
(defvar mail-extr-full-name-prefixes
  (purecopy
   "\\(Prof\\|D[Rr]\\|Mrs?\\|Rev\\|Rabbi\\|SysOp\\|LCDR\\)\\.?[ \t\n]")
  "*Matches prefixes to the full name that identify a person's position.
These are stripped from the full name because they do not contribute to
uniquely identifying the person.")

(defvar mail-extr-@-binds-tighter-than-! nil
  "*Whether the local mail transport agent looks at ! before @.")

(defvar mail-extr-mangle-uucp nil
  "*Whether to throw away information in UUCP addresses
by translating things like \"foo!bar!baz@host\" into \"baz@bar.UUCP\".")
Eric S. Raymond's avatar
Eric S. Raymond committed
237 238 239 240 241 242 243 244 245 246 247 248

;;----------------------------------------------------------------------
;; what orderings are meaningful?????
;;(defvar mail-operator-precedence-list '(?! ?% ?@))
;; Right operand of a % or a @ must be a domain name, period.  No other
;; operators allowed.  Left operand of a @ is an address relative to that
;; site.

;; Left operand of a ! must be a domain name.  Right operand is an
;; arbitrary address.
;;----------------------------------------------------------------------

249

Eric S. Raymond's avatar
Eric S. Raymond committed
250

251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Constant definitions.
;;

;;           Codes in
;; Names in  ISO 8859-1 Name
;; ISO 10XXX ISO 8859-2 in
;; ISO 6937  ISO 10646  RFC            Swedish
;; etc.      Hex Oct    1345 TeX Split ASCII Description
;; --------- ---------- ---- --- ----- ----- -------------------------------
;; %a        E4  344    a:   \"a ae    {     latin small   a + diaeresis   d
;; %o        F6  366    o:   \"o oe    |     latin small   o + diaeresis   v
;; @a        E5  345    aa   \oa aa    }     latin small   a + ring above  e
;; %u        FC  374    u:   \"u ue    ~     latin small   u + diaeresis   |
;; /e        E9  351    e'   \'e       `     latin small   e + acute       i
;; %A        C4  304    A:   \"A AE    [     latin capital a + diaeresis   D
;; %O        D6  326    O:   \"O OE    \     latin capital o + diaeresis   V
;; @A        C5  305    AA   \oA AA    ]     latin capital a + ring above  E
;; %U        DC  334    U:   \"U UE    ^     latin capital u + diaeresis   \
;; /E        C9  311    E'   \'E       @     latin capital e + acute       I

;; NOTE: @a and @A are not in ISO 8859-2 (the codes mentioned above invoke
;; /l and /L).  Some of this data was retrieved from
;; listserv@jhuvm.hcf.jhu.edu.
Eric S. Raymond's avatar
Eric S. Raymond committed
276 277

;; Any character that can occur in a name, not counting characters that
278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294
;; separate parts of a multipart name (hyphen and period).
;; Yes, there are weird people with digits in their names.
;; You will also notice the consideration for the
;; Swedish/Finnish/Norwegian character set.
;; #### (go to \376 instead of \377 to work around bug in search.c...)
(defconst mail-extr-all-letters-but-separators
  (purecopy "][A-Za-z{|}'~0-9`\200-\376"))

;; Any character that can occur in a name in an RFC822 address including
;; the separator (hyphen and possibly period) for multipart names.
;; #### should . be in here?
(defconst mail-extr-all-letters
  (purecopy (concat mail-extr-all-letters-but-separators "---")))

;; Any character that can start a name.
;; Keep this set as minimal as possible.
(defconst mail-extr-first-letters (purecopy "A-Za-z"))
Eric S. Raymond's avatar
Eric S. Raymond committed
295 296

;; Any character that can end a name.
297 298
;; Keep this set as minimal as possible.
(defconst mail-extr-last-letters (purecopy "[A-Za-z`'."))
Eric S. Raymond's avatar
Eric S. Raymond committed
299

300 301
(defconst mail-extr-leading-garbage
  (purecopy (format "[^%s]+" mail-extr-first-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
302

303 304 305 306 307 308
;; (defconst mail-extr-non-name-chars 
;;   (purecopy (concat "^" mail-extr-all-letters ".")))
;; (defconst mail-extr-non-begin-name-chars
;;   (purecopy (concat "^" mail-extr-first-letters)))
;; (defconst mail-extr-non-end-name-chars
;;   (purecopy (concat "^" mail-extr-last-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
309

310 311 312 313 314
;; Matches an initial not followed by both a period and a space. 
;; (defconst mail-extr-bad-initials-pattern
;;   (purecopy 
;;    (format "\\(\\([^%s]\\|\\`\\)[%s]\\)\\(\\.\\([^ ]\\)\\| \\|\\([^%s .]\\)\\|\\'\\)"
;;            mail-extr-all-letters mail-extr-first-letters mail-extr-all-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
315 316 317

;; Matches periods used instead of spaces.  Must not match the period
;; following an initial.
318 319 320 321 322 323
(defconst mail-extr-bad-dot-pattern
  (purecopy
   (format "\\([%s][%s]\\)\\.+\\([%s]\\)"
	   mail-extr-all-letters
	   mail-extr-last-letters
	   mail-extr-first-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
324 325

;; Matches an embedded or leading nickname that should be removed.
326 327 328 329
;; (defconst mail-extr-nickname-pattern
;;   (purecopy
;;    (format "\\([ .]\\|\\`\\)[\"'`\[\(]\\([ .%s]+\\)[\]\"'\)] "
;;            mail-extr-all-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
330 331

;; Matches the occurrence of a generational name suffix, and the last
332 333 334 335 336 337 338 339 340 341 342
;; character of the preceding name.  This is important because we want to
;; keep such suffixes: they help to uniquely identify the person.
;; *** Perhaps this should be a user-customizable variable.  However, the
;; *** regular expression is fairly tricky to alter, so maybe not.
(defconst mail-extr-full-name-suffix-pattern
  (purecopy
   (format
    "\\(,? ?\\([JjSs][Rr]\\.?\\|V?I+V?\\)\\)\\([^%s]\\([^%s]\\|\\'\\)\\|\\'\\)"
    mail-extr-all-letters mail-extr-all-letters)))

(defconst mail-extr-roman-numeral-pattern (purecopy "V?I+V?\\b"))
Eric S. Raymond's avatar
Eric S. Raymond committed
343 344 345

;; Matches a trailing uppercase (with other characters possible) acronym.
;; Must not match a trailing uppercase last name or trailing initial
346 347
(defconst mail-extr-weird-acronym-pattern
  (purecopy "\\([A-Z]+[-_/]\\|[A-Z][A-Z][A-Z]?\\b\\)"))
Eric S. Raymond's avatar
Eric S. Raymond committed
348 349
      
;; Matches a mixed-case or lowercase name (not an initial).
350 351 352 353 354 355 356 357
;; #### Match Latin1 lower case letters here too?
;; (defconst mail-extr-mixed-case-name-pattern
;;   (purecopy
;;    (format
;;     "\\b\\([a-z][%s]*[%s]\\|[%s][%s]*[a-z][%s]*[%s]\\|[%s][%s]*[a-z]\\)"
;;     mail-extr-all-letters mail-extr-last-letters
;;     mail-extr-first-letters mail-extr-all-letters mail-extr-all-letters
;;     mail-extr-last-letters mail-extr-first-letters mail-extr-all-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
358 359

;; Matches a trailing alternative address.
360 361 362 363
;; #### Match Latin1 letters here too?
;; #### Match _ before @ here too?  
(defconst mail-extr-alternative-address-pattern
  (purecopy "\\(aka *\\)?[a-zA-Z.]+[!@][a-zA-Z.]"))
Eric S. Raymond's avatar
Eric S. Raymond committed
364 365 366

;; Matches a variety of trailing comments not including comma-delimited
;; comments.
367 368
(defconst mail-extr-trailing-comment-start-pattern
  (purecopy " [-{]\\|--\\|[+@#></\;]"))
Eric S. Raymond's avatar
Eric S. Raymond committed
369 370 371 372

;; Matches a name (not an initial).
;; This doesn't force a word boundary at the end because sometimes a
;; comment is separated by a `-' with no preceding space.
373 374 375 376 377
(defconst mail-extr-name-pattern
  (purecopy (format "\\b[%s][%s]*[%s]"
		    mail-extr-first-letters
		    mail-extr-all-letters
		    mail-extr-last-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
378

379 380
(defconst mail-extr-initial-pattern
  (purecopy (format "\\b[%s]\\([. ]\\|\\b\\)" mail-extr-first-letters)))
Eric S. Raymond's avatar
Eric S. Raymond committed
381 382

;; Matches a single name before a comma.
383 384
;; (defconst mail-extr-last-name-first-pattern
;;   (purecopy (concat "\\`" mail-extr-name-pattern ",")))
Eric S. Raymond's avatar
Eric S. Raymond committed
385 386

;; Matches telephone extensions.
387 388 389
(defconst mail-extr-telephone-extension-pattern
  (purecopy
   "\\(\\([Ee]xt\\|\\|[Tt]ph\\|[Tt]el\\|[Xx]\\).?\\)? *\\+?[0-9][- 0-9]+"))
Eric S. Raymond's avatar
Eric S. Raymond committed
390 391

;; Matches ham radio call signs.
392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408
;; Help from: Mat Maessen N2NJZ <maessm@rpi.edu>, Mark Feit
;; <mark@era.com>, Michael Covington <mcovingt@ai.uga.edu>.
;; Examples: DX504 DX515 K5MRU K8DHK KA9WGN KA9WGN KD3FU KD6EUI KD6HBW
;; KE9TV KF0NV N1API N3FU N3GZE N3IGS N4KCC N7IKQ N9HHU W4YHF W6ANK WA2SUH
;; WB7VZI N2NJZ NR3G KJ4KK AB4UM AL7NI KH6OH WN3KBT N4TMI W1A N0NZO
(defconst mail-extr-ham-call-sign-pattern
  (purecopy "\\b\\(DX[0-9]+\\|[AKNW][A-Z]?[0-9][A-Z][A-Z]?[A-Z]?\\)"))

;; Possible trailing suffixes: "\\(/\\(KT\\|A[AEG]\\|[R0-9]\\)\\)?"
;; /KT == Temporary Technician (has CSC but not "real" license)
;; /AA == Temporary Advanced
;; /AE == Temporary Extra
;; /AG == Temporary General
;; /R  == repeater
;; /#  == stations operating out of home district
;; I don't include these in the regexp above because I can't imagine
;; anyone putting them with their name in an e-mail address.
Eric S. Raymond's avatar
Eric S. Raymond committed
409 410

;; Matches normal single-part name
411 412 413 414 415 416 417 418 419 420
(defconst mail-extr-normal-name-pattern
  (purecopy (format "\\b[%s][%s]+[%s]"
		    mail-extr-first-letters
		    mail-extr-all-letters-but-separators
		    mail-extr-last-letters)))

;; Matches a single word name.
;; (defconst mail-extr-one-name-pattern
;;   (purecopy (concat "\\`" mail-extr-normal-name-pattern "\\'")))
  
Eric S. Raymond's avatar
Eric S. Raymond committed
421
;; Matches normal two names with missing middle initial
422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495
;; The first name is not allowed to have a hyphen because this can cause
;; false matches where the "middle initial" is actually the first letter
;; of the second part of the first name.
(defconst mail-extr-two-name-pattern
  (purecopy
   (concat "\\`\\(" mail-extr-normal-name-pattern
	   "\\|" mail-extr-initial-pattern
	   "\\) +\\(" mail-extr-name-pattern "\\)\\(,\\|\\'\\)")))

(defconst mail-extr-listserv-list-name-pattern
  (purecopy "Multiple recipients of list \\([-A-Z]+\\)"))

(defconst mail-extr-stupid-vms-date-stamp-pattern
  (purecopy
   "[0-9][0-9]-[JFMASOND][aepuco][nbrylgptvc]-[0-9][0-9][0-9][0-9] [0-9]+ *"))

;;; HZ -- GB (PRC Chinese character encoding) in ASCII embedding protocol
;;
;; In ASCII mode, a byte is interpreted as an ASCII character, unless a '~' is
;; encountered. The character '~' is an escape character. By convention, it
;; must be immediately followed ONLY by '~', '{' or '\n' (<LF>), with the
;; following special meaning.
;; 
;; o The escape sequence '~~' is interpreted as a '~'.
;; o The escape-to-GB sequence '~{' switches the mode from ASCII to GB.
;; o The escape sequence '~\n' is a line-continuation marker to be consumed
;;   with no output produced.
;; 
;; In GB mode, characters are interpreted two bytes at a time as (pure) GB
;; codes until the escape-from-GB code '~}' is read. This code switches the
;; mode from GB back to ASCII.  (Note that the escape-from-GB code '~}'
;; ($7E7D) is outside the defined GB range.)
(defconst mail-extr-hz-embedded-gb-encoded-chinese-pattern
  (purecopy "~{\\([^~].\\|~[^\}]\\)+~}"))

;; The leading optional lowercase letters are for a bastardized version of
;; the encoding, as is the optional nature of the final slash.
(defconst mail-extr-x400-encoded-address-pattern
  (purecopy "[a-z]?[a-z]?\\(/[A-Za-z]+\\(\\.[A-Za-z]+\\)?=[^/]+\\)+/?\\'"))

(defconst mail-extr-x400-encoded-address-field-pattern-format
  (purecopy "/%s=\\([^/]+\\)\\(/\\|\\'\\)"))

(defconst mail-extr-x400-encoded-address-surname-pattern
  ;; S stands for Surname (family name).
  (purecopy
   (format mail-extr-x400-encoded-address-field-pattern-format "[Ss]")))

(defconst mail-extr-x400-encoded-address-given-name-pattern
  ;; G stands for Given name.
  (purecopy
   (format mail-extr-x400-encoded-address-field-pattern-format "[Gg]")))

(defconst mail-extr-x400-encoded-address-full-name-pattern
  ;; PN stands for Personal Name.  When used it represents the combination
  ;; of the G and S fields.
  ;; "The one system I used having this field asked it with the prompt
  ;; `Personal Name'.  But they mapped it into G and S on outgoing real
  ;; X.400 addresses.  As they mapped G and S into PN on incoming..."
  (purecopy
   (format mail-extr-x400-encoded-address-field-pattern-format "[Pp][Nn]")))



;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Syntax tables used for quick parsing.
;;

(defconst mail-extr-address-syntax-table (make-syntax-table))
(defconst mail-extr-address-comment-syntax-table (make-syntax-table))
(defconst mail-extr-address-domain-literal-syntax-table (make-syntax-table))
(defconst mail-extr-address-text-comment-syntax-table (make-syntax-table))
(defconst mail-extr-address-text-syntax-table (make-syntax-table))
Eric S. Raymond's avatar
Eric S. Raymond committed
496 497 498 499 500 501 502 503
(mapcar
 (function
  (lambda (pair)
    (let ((syntax-table (symbol-value (car pair))))
      (mapcar
       (function
	(lambda (item)
	  (if (eq 2 (length item))
504
	      ;; modifying syntax of a single character
Eric S. Raymond's avatar
Eric S. Raymond committed
505
	      (modify-syntax-entry (car item) (car (cdr item)) syntax-table)
506 507 508 509
	    ;; modifying syntax of a range of characters
	    (let ((char (nth 0 item))
		  (bound (nth 1 item))
		  (syntax (nth 2 item)))
Eric S. Raymond's avatar
Eric S. Raymond committed
510 511 512 513
	      (while (<= char bound)
		(modify-syntax-entry char syntax syntax-table)
		(setq char (1+ char)))))))
       (cdr pair)))))
514 515 516 517 518 519 520
 '((mail-extr-address-syntax-table
    (?\000 ?\037 "w")			;control characters
    (?\040	 " ")			;SPC
    (?! ?~	 "w")			;printable characters
    (?\177	 "w")			;DEL
    (?\200 ?\377 "w")			;high-bit-on characters
    (?\240	 " ")			;nobreakspace
Eric S. Raymond's avatar
Eric S. Raymond committed
521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538
    (?\t " ")
    (?\r " ")
    (?\n " ")
    (?\( ".")
    (?\) ".")
    (?<  ".")
    (?>  ".")
    (?@  ".")
    (?,  ".")
    (?\; ".")
    (?:  ".")
    (?\\ "\\")
    (?\" "\"")
    (?.  ".")
    (?\[ ".")
    (?\] ".")
    ;; % and ! aren't RFC822 characters, but it is convenient to pretend
    (?%  ".")
539
    (?!  ".") ;; this needs to be word-constituent when not in .UUCP mode
Eric S. Raymond's avatar
Eric S. Raymond committed
540
    )
541 542 543 544 545 546 547
   (mail-extr-address-comment-syntax-table
    (?\000 ?\377 "w")
    (?\040 " ")
    (?\240 " ")
    (?\t " ")
    (?\r " ")
    (?\n " ")
Eric S. Raymond's avatar
Eric S. Raymond committed
548 549 550
    (?\( "\(\)")
    (?\) "\)\(")
    (?\\ "\\"))
551 552 553 554 555 556 557
   (mail-extr-address-domain-literal-syntax-table
    (?\000 ?\377 "w")
    (?\040 " ")
    (?\240 " ")
    (?\t " ")
    (?\r " ")
    (?\n " ")
Eric S. Raymond's avatar
Eric S. Raymond committed
558 559 560
    (?\[ "\(\]")			;??????
    (?\] "\)\[")			;??????
    (?\\ "\\"))
561 562 563 564 565 566 567
   (mail-extr-address-text-comment-syntax-table
    (?\000 ?\377 "w")
    (?\040 " ")
    (?\240 " ")
    (?\t " ")
    (?\r " ")
    (?\n " ")
Eric S. Raymond's avatar
Eric S. Raymond committed
568 569 570 571 572 573 574 575 576 577 578
    (?\( "\(\)")
    (?\) "\)\(")
    (?\[ "\(\]")
    (?\] "\)\[")
    (?\{ "\(\}")
    (?\} "\)\{")
    (?\\ "\\")
    (?\" "\"")
    ;; (?\' "\)\`")
    ;; (?\` "\(\'")
    )
579 580 581 582 583 584 585
   (mail-extr-address-text-syntax-table
    (?\000 ?\177 ".")
    (?\200 ?\377 "w")
    (?\040 " ")
    (?\t " ")
    (?\r " ")
    (?\n " ")
Eric S. Raymond's avatar
Eric S. Raymond committed
586 587 588 589 590 591 592 593 594 595 596 597
    (?A ?Z "w")
    (?a ?z "w")
    (?-    "w")
    (?\}   "w")
    (?\{   "w")
    (?|    "w")
    (?\'   "w")
    (?~    "w")
    (?0 ?9 "w"))
   ))


598 599
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
600
;; Utility functions and macros.
601 602 603 604 605 606 607 608 609 610 611 612 613 614 615
;;

(defmacro mail-extr-delete-char (n)
  ;; in v19, delete-char is compiled as a function call, but delete-region
  ;; is byte-coded, so it's much much faster.
  (list 'delete-region '(point) (list '+ '(point) n)))

(defmacro mail-extr-skip-whitespace-forward ()
  ;; v19 fn skip-syntax-forward is more tasteful, but not byte-coded.
  '(skip-chars-forward " \t\n\r\240"))

(defmacro mail-extr-skip-whitespace-backward ()
  ;; v19 fn skip-syntax-backward is more tasteful, but not byte-coded.
  '(skip-chars-backward " \t\n\r\240"))

Eric S. Raymond's avatar
Eric S. Raymond committed
616

617
(defmacro mail-extr-undo-backslash-quoting (beg end)
Eric S. Raymond's avatar
Eric S. Raymond committed
618 619 620 621 622
  (`(save-excursion
      (save-restriction
	(narrow-to-region (, beg) (, end))
	(goto-char (point-min))
	;; undo \ quoting
623 624 625 626
	(while (search-forward "\\" nil t)
	  (mail-extr-delete-char -1)
	  (or (eobp)
	      (forward-char 1))
Eric S. Raymond's avatar
Eric S. Raymond committed
627 628
	  )))))

629
(defmacro mail-extr-nuke-char-at (pos)
Eric S. Raymond's avatar
Eric S. Raymond committed
630 631
  (` (save-excursion
       (goto-char (, pos))
632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651
       (mail-extr-delete-char 1)
       (insert ?\ ))))

(put 'mail-extr-nuke-outside-range
     'edebug-form-spec '(symbolp &optional form form atom))

(defmacro mail-extr-nuke-outside-range (list-symbol
					beg-symbol end-symbol
					&optional no-replace)
  ;; LIST-SYMBOL names a variable holding a list of buffer positions
  ;; BEG-SYMBOL and END-SYMBOL name variables delimiting a range
  ;; Each element of LIST-SYMBOL which lies outside of the range is
  ;;  deleted from the list.
  ;; Unless NO-REPLACE is true, at each of the positions in LIST-SYMBOL
  ;;  which lie outside of the range, one character at that position is
  ;;  replaced with a SPC.
  (or (memq no-replace '(t nil))
      (error "no-replace must be t or nil, evalable at macroexpand-time."))
  (` (let ((temp (, list-symbol))
	   ch)
Eric S. Raymond's avatar
Eric S. Raymond committed
652
       (while temp
653 654 655 656 657 658
	 (setq ch (car temp))
	 (cond ((or (> ch (, end-symbol))
		    (< ch (, beg-symbol)))
		(,@ (if no-replace
			nil
		      (` ((mail-extr-nuke-char-at ch)))))
Eric S. Raymond's avatar
Eric S. Raymond committed
659 660 661 662
		(setcar temp nil)))
	 (setq temp (cdr temp)))
       (setq (, list-symbol) (delq nil (, list-symbol))))))

663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680
(defun mail-extr-demarkerize (marker)
  ;; if arg is a marker, destroys the marker, then returns the old value.
  ;; otherwise returns the arg.
  (if (markerp marker)
      (let ((temp (marker-position marker)))
	(set-marker marker nil)
	temp)
    marker))

(defun mail-extr-markerize (pos)
  ;; coerces pos to a marker if non-nil.
  (if (or (markerp pos) (null pos))
      pos
    (copy-marker pos)))

(defmacro mail-extr-last (list)
  ;; Returns last element of LIST.
  ;; Could be a subst.
Eric S. Raymond's avatar
Eric S. Raymond committed
681 682 683 684 685
  (` (let ((list (, list)))
       (while (not (null (cdr list)))
	 (setq list (cdr list)))
       (car list))))
  
686 687
(defmacro mail-extr-safe-move-sexp (arg)
  ;; Safely skip over one balanced sexp, if there is one.  Return t if success.
Eric S. Raymond's avatar
Eric S. Raymond committed
688 689 690 691 692
  (` (condition-case error
	 (progn
	   (goto-char (scan-sexps (point) (, arg)))
	   t)
       (error
693
	;; #### kludge kludge kludge kludge kludge kludge kludge !!!
Eric S. Raymond's avatar
Eric S. Raymond committed
694 695 696 697 698
	(if (string-equal (nth 1 error) "Unbalanced parentheses")
	    nil
	  (while t
	    (signal (car error) (cdr error))))))))

699 700
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
Eric S. Raymond's avatar
Eric S. Raymond committed
701
;; The main function to grind addresses
702 703 704 705 706
;;

(defvar disable-initial-guessing-flag)	; dynamic assignment
(defvar cbeg)				; dynamic assignment
(defvar cend)				; dynamic assignment
Eric S. Raymond's avatar
Eric S. Raymond committed
707

708
;;;###autoload
Eric S. Raymond's avatar
Eric S. Raymond committed
709
(defun mail-extract-address-components (address)
710 711 712 713 714 715 716 717 718 719 720 721
  "Given an RFC-822 ADDRESS, extract full name and canonical address.
Returns a list of the form (FULL-NAME CANONICAL-ADDRESS).
If no name can be extracted, FULL-NAME will be nil.
ADDRESS may be a string or a buffer.  If it is a buffer, the visible 
 (narrowed) portion of the buffer will be interpreted as the address.
 (This feature exists so that the clever caller might be able to avoid
 consing a string.)
If ADDRESS contains more than one RFC-822 address, only the first is
 returned.  Some day this function may be extended to extract multiple
 addresses, or perhaps return the position at which parsing stopped."
  (let ((canonicalization-buffer (get-buffer-create " *canonical address*"))
	(extraction-buffer (get-buffer-create " *extract address components*"))
Eric S. Raymond's avatar
Eric S. Raymond committed
722
	char
723
;;	multiple-addresses
Eric S. Raymond's avatar
Eric S. Raymond committed
724 725 726 727 728
	<-pos >-pos @-pos :-pos ,-pos !-pos %-pos \;-pos
	group-:-pos group-\;-pos route-addr-:-pos
	record-pos-symbol
	first-real-pos last-real-pos
	phrase-beg phrase-end
729
	cbeg cend			; dynamically set from -voodoo
Eric S. Raymond's avatar
Eric S. Raymond committed
730 731 732 733 734
	quote-beg quote-end
	atom-beg atom-end
	mbox-beg mbox-end
	\.-ends-name
	temp
735 736
;;	name-suffix
	fi mi li			; first, middle, last initial
Eric S. Raymond's avatar
Eric S. Raymond committed
737
	saved-%-pos saved-!-pos saved-@-pos
738 739 740 741
	domain-pos \.-pos insert-point
;;	mailbox-name-processed-flag
	disable-initial-guessing-flag	; dynamically set from -voodoo
	)
Eric S. Raymond's avatar
Eric S. Raymond committed
742 743 744
    
    (save-excursion
      (set-buffer extraction-buffer)
745 746
      (fundamental-mode)
      (kill-all-local-variables)
Jim Blandy's avatar
Jim Blandy committed
747
      (buffer-disable-undo extraction-buffer)
748
      (set-syntax-table mail-extr-address-syntax-table)
Eric S. Raymond's avatar
Eric S. Raymond committed
749 750 751 752 753 754
      (widen)
      (erase-buffer)
      (setq case-fold-search nil)
      
      ;; Insert extra space at beginning to allow later replacement with <
      ;; without having to move markers.
755 756 757 758 759 760 761 762 763
      (insert ?\ )

      ;; Insert the address itself.
      (cond ((stringp address)
	     (insert address))
	    ((bufferp address)
	     (insert-buffer-substring address))
	    (t
	     (error "Illegal address: %s" address)))
Eric S. Raymond's avatar
Eric S. Raymond committed
764 765 766 767 768 769 770 771 772 773
      
      ;; stolen from rfc822.el
      ;; Unfold multiple lines.
      (goto-char (point-min))
      (while (re-search-forward "\\([^\\]\\(\\\\\\\\\\)*\\)\n[ \t]" nil t)
	(replace-match "\\1 " t))
      
      ;; first pass grabs useful information about address
      (goto-char (point-min))
      (while (progn
774
	       (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
775 776 777 778 779 780 781 782
	       (not (eobp)))
	(setq char (char-after (point)))
	(or first-real-pos
	    (if (not (eq char ?\())
		(setq first-real-pos (point))))
	(cond
	 ;; comment
	 ((eq char ?\()
783
	  (set-syntax-table mail-extr-address-comment-syntax-table)
Eric S. Raymond's avatar
Eric S. Raymond committed
784
	  ;; only record the first non-empty comment's position
785
	  (if (and (not cbeg)
Eric S. Raymond's avatar
Eric S. Raymond committed
786 787
		   (save-excursion
		     (forward-char 1)
788
		     (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
789
		     (not (eq ?\) (char-after (point))))))
790
	      (setq cbeg (point)))
Eric S. Raymond's avatar
Eric S. Raymond committed
791
	  ;; TODO: don't record if unbalanced
792
	  (or (mail-extr-safe-move-sexp 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
793
	      (forward-char 1))
794 795 796 797
	  (set-syntax-table mail-extr-address-syntax-table)
	  (if (and cbeg
		   (not cend))
	      (setq cend (point))))
Eric S. Raymond's avatar
Eric S. Raymond committed
798 799 800 801 802 803
	 ;; quoted text
	 ((eq char ?\")
	  ;; only record the first non-empty quote's position
	  (if (and (not quote-beg)
		   (save-excursion
		     (forward-char 1)
804
		     (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
805 806 807
		     (not (eq ?\" (char-after (point))))))
	      (setq quote-beg (point)))
	  ;; TODO: don't record if unbalanced
808
	  (or (mail-extr-safe-move-sexp 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
809 810 811 812 813 814
	      (forward-char 1))
	  (if (and quote-beg
		   (not quote-end))
	      (setq quote-end (point))))
	 ;; domain literals
	 ((eq char ?\[)
815 816
	  (set-syntax-table mail-extr-address-domain-literal-syntax-table)
	  (or (mail-extr-safe-move-sexp 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
817
	      (forward-char 1))
818
	  (set-syntax-table mail-extr-address-syntax-table))
Eric S. Raymond's avatar
Eric S. Raymond committed
819 820
	 ;; commas delimit addresses when outside < > pairs.
	 ((and (eq char ?,)
821 822 823
	       (or (and (null <-pos)
			;; Handle ROUTE-ADDR address that is missing its <.
			(not (eq ?@ (char-after (1+ (point))))))
Eric S. Raymond's avatar
Eric S. Raymond committed
824 825
		   (and >-pos
			;; handle weird munged addresses
826 827 828 829 830 831 832 833 834 835 836 837
			;; BUG FIX: This test was reversed.  Thanks to the
			;; brilliant Rod Whitby <rwhitby@research.canon.oz.au>
			;; for discovering this!
			(< (mail-extr-last <-pos) (car >-pos)))))
;; It'd be great if some day this worked, but for now, punt.
;;	  (setq multiple-addresses t)
;;	  ;; *** Why do I want this:
;;	  (mail-extr-delete-char 1)
;;	  (narrow-to-region (point-min) (point))
	  (delete-region (point) (point-max))
	  (setq char ?\() ; HAVE I NO SHAME??
	  )
Eric S. Raymond's avatar
Eric S. Raymond committed
838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858
	 ;; record the position of various interesting chars, determine
	 ;; legality later.
	 ((setq record-pos-symbol
		(cdr (assq char
			   '((?< . <-pos) (?> . >-pos) (?@ . @-pos)
			     (?: . :-pos) (?, . ,-pos) (?! . !-pos)
			     (?% . %-pos) (?\; . \;-pos)))))
	  (set record-pos-symbol
	       (cons (point) (symbol-value record-pos-symbol)))
	  (forward-char 1))
	 ((eq char ?.)
	  (forward-char 1))
	 ((memq char '(
		       ;; comment terminator illegal
		       ?\)
		       ;; domain literal terminator illegal
		       ?\]
		       ;; \ allowed only within quoted strings,
		       ;; domain literals, and comments
		       ?\\
		       ))
859
	  (mail-extr-nuke-char-at (point))
Eric S. Raymond's avatar
Eric S. Raymond committed
860 861 862 863
	  (forward-char 1))
	 (t
	  (forward-word 1)))
	(or (eq char ?\()
864 865 866
	    ;; At the end of first address of a multiple address header.
	    (and (eq char ?,)
		 (eobp))
Eric S. Raymond's avatar
Eric S. Raymond committed
867 868 869 870
	    (setq last-real-pos (point))))
      
      ;; Use only the leftmost <, if any.  Replace all others with spaces.
      (while (cdr <-pos)
871
	(mail-extr-nuke-char-at (car <-pos))
Eric S. Raymond's avatar
Eric S. Raymond committed
872 873 874 875
	(setq <-pos (cdr <-pos)))
      
      ;; Use only the rightmost >, if any.  Replace all others with spaces.
      (while (cdr >-pos)
876
	(mail-extr-nuke-char-at (nth 1 >-pos))
Eric S. Raymond's avatar
Eric S. Raymond committed
877 878 879
	(setcdr >-pos (nthcdr 2 >-pos)))
      
      ;; If multiple @s and a :, but no < and >, insert around buffer.
880
      ;; Example: @foo.bar.dom,@xxx.yyy.zzz:mailbox@aaa.bbb.ccc
Eric S. Raymond's avatar
Eric S. Raymond committed
881 882
      ;; This commonly happens on the UUCP "From " line.  Ugh.
      (cond ((and (> (length @-pos) 1)
883 884
		  (eq 1 (length :-pos))	;TODO: check if between last two @s
		  (not \;-pos)
Eric S. Raymond's avatar
Eric S. Raymond committed
885 886
		  (not <-pos))
	     (goto-char (point-min))
887
	     (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
888 889 890 891 892 893 894 895 896 897 898 899 900
	     (setq <-pos (list (point)))
	     (insert ?<)))
      
      ;; If < but no >, insert > in rightmost possible position
      (cond ((and <-pos
		  (null >-pos))
	     (goto-char (point-max))
	     (setq >-pos (list (point)))
	     (insert ?>)))
      
      ;; If > but no <, replace > with space.
      (cond ((and >-pos
		  (null <-pos))
901
	     (mail-extr-nuke-char-at (car >-pos))
Eric S. Raymond's avatar
Eric S. Raymond committed
902 903 904 905 906 907 908 909 910 911 912
	     (setq >-pos nil)))

      ;; Turn >-pos and <-pos into non-lists
      (setq >-pos (car >-pos)
	    <-pos (car <-pos))
      
      ;; Trim other punctuation lists of items outside < > pair to handle
      ;; stupid MTAs.
      (cond (<-pos			; don't need to check >-pos also
	     ;; handle bozo software that violates RFC 822 by sticking
	     ;; punctuation marks outside of a < > pair
913
	     (mail-extr-nuke-outside-range @-pos <-pos >-pos t)
Eric S. Raymond's avatar
Eric S. Raymond committed
914 915 916
	     ;; RFC 822 says nothing about these two outside < >, but
	     ;; remove those positions from the lists to make things
	     ;; easier.
917 918
	     (mail-extr-nuke-outside-range !-pos <-pos >-pos t)
	     (mail-extr-nuke-outside-range %-pos <-pos >-pos t)))
Eric S. Raymond's avatar
Eric S. Raymond committed
919 920 921 922 923 924 925 926 927 928 929 930 931 932
      
      ;; Check for : that indicates GROUP list and for : part of
      ;; ROUTE-ADDR spec.
      ;; Can't possibly be more than two :.  Nuke any extra.
      (while :-pos
	(setq temp (car :-pos)
	      :-pos (cdr :-pos))
	(cond ((and <-pos >-pos
		    (> temp <-pos)
		    (< temp >-pos))
	       (if (or route-addr-:-pos
		       (< (length @-pos) 2)
		       (> temp (car @-pos))
		       (< temp (nth 1 @-pos)))
933
		   (mail-extr-nuke-char-at temp)
Eric S. Raymond's avatar
Eric S. Raymond committed
934 935 936 937 938 939 940 941 942 943 944 945 946 947
		 (setq route-addr-:-pos temp)))
	      ((or (not <-pos)
		   (and <-pos
			(< temp <-pos)))
	       (setq group-:-pos temp))))
      
      ;; Nuke any ; that is in or to the left of a < > pair or to the left
      ;; of a GROUP starting :.  Also, there may only be one ;.
      (while \;-pos
	(setq temp (car \;-pos)
	      \;-pos (cdr \;-pos))
	(cond ((and <-pos >-pos
		    (> temp <-pos)
		    (< temp >-pos))
948
	       (mail-extr-nuke-char-at temp))
Eric S. Raymond's avatar
Eric S. Raymond committed
949 950 951 952 953
	      ((and (or (not group-:-pos)
			(> temp group-:-pos))
		    (not group-\;-pos))
	       (setq group-\;-pos temp))))
      
954 955 956 957 958 959 960 961 962 963
      ;; Nuke unmatched GROUP syntax characters.
      (cond ((and group-:-pos (not group-\;-pos))
	     ;; *** Do I really need to erase it?
	     (mail-extr-nuke-char-at group-:-pos)
	     (setq group-:-pos nil)))
      (cond ((and group-\;-pos (not group-:-pos))
	     ;; *** Do I really need to erase it?
	     (mail-extr-nuke-char-at group-\;-pos)
	     (setq group-\;-pos nil)))
      
Eric S. Raymond's avatar
Eric S. Raymond committed
964 965
      ;; Handle junk like ";@host.company.dom" that sendmail adds.
      ;; **** should I remember comment positions?
966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991
      (cond
       (group-\;-pos
	;; this is fine for now
	(mail-extr-nuke-outside-range !-pos group-:-pos group-\;-pos t)
	(mail-extr-nuke-outside-range @-pos group-:-pos group-\;-pos t)
	(mail-extr-nuke-outside-range %-pos group-:-pos group-\;-pos t)
	(mail-extr-nuke-outside-range ,-pos group-:-pos group-\;-pos t)
	(and last-real-pos
	     (> last-real-pos (1+ group-\;-pos))
	     (setq last-real-pos (1+ group-\;-pos)))
	;; *** This may be wrong:
        (and cend
             (> cend group-\;-pos)
             (setq cend nil
                   cbeg nil))
	(and quote-end
	     (> quote-end group-\;-pos)
	     (setq quote-end nil
		   quote-beg nil))
	;; This was both wrong and unnecessary:
	;;(narrow-to-region (point-min) group-\;-pos)

	;; *** The entire handling of GROUP addresses seems rather lame.
	;; *** It deserves a complete rethink, except that these addresses
	;; *** are hardly ever seen.
	))
Eric S. Raymond's avatar
Eric S. Raymond committed
992 993 994 995 996 997
      
      ;; Any commas must be between < and : of ROUTE-ADDR.  Nuke any
      ;; others.
      ;; Hell, go ahead an nuke all of the commas.
      ;; **** This will cause problems when we start handling commas in
      ;; the PHRASE part .... no it won't ... yes it will ... ?????
998
      (mail-extr-nuke-outside-range ,-pos 1 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
999 1000 1001 1002 1003 1004 1005 1006
      
      ;; can only have multiple @s inside < >.  The fact that some MTAs
      ;; put de-bracketed ROUTE-ADDRs in the UUCP-style "From " line is
      ;; handled above.
      
      ;; Locate PHRASE part of ROUTE-ADDR.
      (cond (<-pos
	     (goto-char <-pos)
1007
	     (mail-extr-skip-whitespace-backward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1008 1009 1010
	     (setq phrase-end (point))
	     (goto-char (or ;;group-:-pos
			    (point-min)))
1011
	     (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028
	     (if (< (point) phrase-end)
		 (setq phrase-beg (point))
	       (setq phrase-end nil))))
      
      ;; handle ROUTE-ADDRS with real ROUTEs.
      ;; If there are multiple @s, then we assume ROUTE-ADDR syntax, and
      ;; any % or ! must be semantically meaningless.
      ;; TODO: do this processing into canonicalization buffer
      (cond (route-addr-:-pos
	     (setq !-pos nil
		   %-pos nil
		   >-pos (copy-marker >-pos)
		   route-addr-:-pos (copy-marker route-addr-:-pos))
	     (goto-char >-pos)
	     (insert-before-markers ?X)
	     (goto-char (car @-pos))
	     (while (setq @-pos (cdr @-pos))
1029
	       (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040
	       (setq %-pos (cons (point-marker) %-pos))
	       (insert "%")
	       (goto-char (1- >-pos))
	       (save-excursion
		 (insert-buffer-substring extraction-buffer
					  (car @-pos) route-addr-:-pos)
		 (delete-region (car @-pos) route-addr-:-pos))
	       (or (cdr @-pos)
		   (setq saved-@-pos (list (point)))))
	     (setq @-pos saved-@-pos)
	     (goto-char >-pos)
1041 1042 1043
	     (mail-extr-delete-char -1)
	     (mail-extr-nuke-char-at route-addr-:-pos)
	     (mail-extr-demarkerize route-addr-:-pos)
Eric S. Raymond's avatar
Eric S. Raymond committed
1044
	     (setq route-addr-:-pos nil
1045 1046
		   >-pos (mail-extr-demarkerize >-pos)
		   %-pos (mapcar 'mail-extr-demarkerize %-pos))))
Eric S. Raymond's avatar
Eric S. Raymond committed
1047 1048 1049 1050 1051 1052 1053
      
      ;; de-listify @-pos
      (setq @-pos (car @-pos))
      
      ;; TODO: remove comments in the middle of an address
      
      (set-buffer canonicalization-buffer)
1054 1055
      (fundamental-mode)
      (kill-all-local-variables)
Jim Blandy's avatar
Jim Blandy committed
1056
      (buffer-disable-undo canonicalization-buffer)
1057
      (set-syntax-table mail-extr-address-syntax-table)
Eric S. Raymond's avatar
Eric S. Raymond committed
1058 1059 1060 1061 1062 1063 1064 1065 1066
      (setq case-fold-search nil)
      
      (widen)
      (erase-buffer)
      (insert-buffer-substring extraction-buffer)
      
      (if <-pos
	  (narrow-to-region (progn
			      (goto-char (1+ <-pos))
1067
			      (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1068 1069
			      (point))
			    >-pos)
1070 1071 1072 1073 1074 1075
	(if (and first-real-pos last-real-pos)
	    (narrow-to-region first-real-pos last-real-pos)
	  ;; ****** Oh no!  What if the address is completely empty!
	  ;; *** Is this correct?
	  (narrow-to-region (point-max) (point-max))
	  ))
Eric S. Raymond's avatar
Eric S. Raymond committed
1076 1077
      
      (and @-pos %-pos
1078
	   (mail-extr-nuke-outside-range %-pos (point-min) @-pos))
Eric S. Raymond's avatar
Eric S. Raymond committed
1079
      (and %-pos !-pos
1080
	   (mail-extr-nuke-outside-range !-pos (point-min) (car %-pos)))
Eric S. Raymond's avatar
Eric S. Raymond committed
1081
      (and @-pos !-pos (not %-pos)
1082
	   (mail-extr-nuke-outside-range !-pos (point-min) @-pos))
Eric S. Raymond's avatar
Eric S. Raymond committed
1083 1084
      
      ;; Error condition:?? (and %-pos (not @-pos))
1085 1086 1087 1088 1089 1090 1091 1092 1093
      
      ;; WARNING: THIS CODE IS DUPLICATED BELOW.
      (cond ((and %-pos
		  (not @-pos))
	     (goto-char (car %-pos))
	     (mail-extr-delete-char 1)
	     (setq @-pos (point))
	     (insert "@")
	     (setq %-pos (cdr %-pos))))
Eric S. Raymond's avatar
Eric S. Raymond committed
1094

1095
      (if mail-extr-mangle-uucp
Eric S. Raymond's avatar
Eric S. Raymond committed
1096 1097 1098 1099 1100
      (cond (!-pos
	     ;; **** I don't understand this save-restriction and the
	     ;; narrow-to-region inside it.  Why did I do that?
	     (save-restriction
	       (cond ((and @-pos
1101
			   mail-extr-@-binds-tighter-than-!)
Eric S. Raymond's avatar
Eric S. Raymond committed
1102 1103 1104
		      (goto-char @-pos)
		      (setq %-pos (cons (point) %-pos)
			    @-pos nil)
1105
		      (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1106 1107
		      (insert "%")
		      (setq insert-point (point-max)))
1108
		     (mail-extr-@-binds-tighter-than-!
Eric S. Raymond's avatar
Eric S. Raymond committed
1109 1110
		      (setq insert-point (point-max)))
		     (%-pos
1111 1112
		      (setq insert-point (mail-extr-last %-pos)
			    saved-%-pos (mapcar 'mail-extr-markerize %-pos)
Eric S. Raymond's avatar
Eric S. Raymond committed
1113
			    %-pos nil
1114
			    @-pos (mail-extr-markerize @-pos)))
Eric S. Raymond's avatar
Eric S. Raymond committed
1115 1116
		     (@-pos
		      (setq insert-point @-pos)
1117
		      (setq @-pos (mail-extr-markerize @-pos)))
Eric S. Raymond's avatar
Eric S. Raymond committed
1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137
		     (t
		      (setq insert-point (point-max))))
	       (narrow-to-region (point-min) insert-point)
	       (setq saved-!-pos (car !-pos))
	       (while !-pos
		 (goto-char (point-max))
		 (cond ((and (not @-pos)
			     (not (cdr !-pos)))
			(setq @-pos (point))
			(insert-before-markers "@ "))
		       (t
			(setq %-pos (cons (point) %-pos))
			(insert-before-markers "% ")))
		 (backward-char 1)
		 (insert-buffer-substring 
		  (current-buffer)
		  (if (nth 1 !-pos)
		      (1+ (nth 1 !-pos))
		    (point-min))
		  (car !-pos))
1138
		 (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1139
		 (or (save-excursion
1140 1141
		       (mail-extr-safe-move-sexp -1)
		       (mail-extr-skip-whitespace-backward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1142 1143 1144
		       (eq ?. (preceding-char)))
		     (insert-before-markers
		      (if (save-excursion
1145
			    (mail-extr-skip-whitespace-backward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1146 1147 1148 1149 1150 1151
			    (eq ?. (preceding-char)))
			  ""
			".")
		      "uucp"))
		 (setq !-pos (cdr !-pos))))
	     (and saved-%-pos
1152 1153 1154 1155 1156 1157 1158
		  (setq %-pos (append (mapcar 'mail-extr-demarkerize
					      saved-%-pos)
				      %-pos)))
	     (setq @-pos (mail-extr-demarkerize @-pos))
	     (narrow-to-region (1+ saved-!-pos) (point-max)))))

      ;; WARNING: THIS CODE IS DUPLICATED ABOVE.
Eric S. Raymond's avatar
Eric S. Raymond committed
1159 1160 1161
      (cond ((and %-pos
		  (not @-pos))
	     (goto-char (car %-pos))
1162
	     (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1163 1164 1165
	     (setq @-pos (point))
	     (insert "@")
	     (setq %-pos (cdr %-pos))))
1166

Eric S. Raymond's avatar
Eric S. Raymond committed
1167 1168 1169 1170 1171 1172 1173 1174 1175
      (setq %-pos (nreverse %-pos))
      ;; RFC 1034 doesn't approve of this, oh well:
      (downcase-region (or (car %-pos) @-pos (point-max)) (point-max))
      (cond (%-pos			; implies @-pos valid
	     (setq temp %-pos)
	     (catch 'truncated
	       (while temp
		 (goto-char (or (nth 1 temp)
				@-pos))
1176
		 (mail-extr-skip-whitespace-backward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1177
		 (save-excursion
1178
		   (mail-extr-safe-move-sexp -1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1179
		   (setq domain-pos (point))
1180
		   (mail-extr-skip-whitespace-backward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1181 1182
		   (setq \.-pos (eq ?. (preceding-char))))
		 (cond ((and \.-pos
1183 1184 1185 1186 1187
			     ;; #### string consing
			     (let ((s (intern-soft
				       (buffer-substring domain-pos (point))
				       mail-extr-all-top-level-domains)))
			       (and s (get s 'domain-name))))
Eric S. Raymond's avatar
Eric S. Raymond committed
1188 1189
			(narrow-to-region (point-min) (point))
			(goto-char (car temp))
1190
			(mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205
			(setq @-pos (point))
			(setcdr temp nil)
			(setq %-pos (delq @-pos %-pos))
			(insert "@")
			(throw 'truncated t)))
		 (setq temp (cdr temp))))))
      (setq mbox-beg (point-min)
	    mbox-end (if %-pos (car %-pos)
		       (or @-pos
			   (point-max))))
      
      ;; Done canonicalizing address.
      
      (set-buffer extraction-buffer)
      
1206 1207 1208 1209
      ;; Decide what part of the address to search to find the full name.
      (cond (
	     ;; Example: "First M. Last" <fml@foo.bar.dom>
	     (and phrase-beg
Eric S. Raymond's avatar
Eric S. Raymond committed
1210 1211 1212
		  (eq quote-beg phrase-beg)
		  (<= quote-end phrase-end))
	     (narrow-to-region (1+ quote-beg) (1- quote-end))
1213 1214 1215
	     (mail-extr-undo-backslash-quoting (point-min) (point-max)))

	    ;; Example: First Last <fml@foo.bar.dom>
Eric S. Raymond's avatar
Eric S. Raymond committed
1216 1217
	    (phrase-beg
	     (narrow-to-region phrase-beg phrase-end))
1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244

	    ;; Example: fml@foo.bar.dom (First M. Last)
	    (cbeg
	     (narrow-to-region (1+ cbeg) (1- cend))
	     (mail-extr-undo-backslash-quoting (point-min) (point-max))
	     
	     ;; Deal with spacing problems
	     (goto-char (point-min))
;	     (cond ((not (search-forward " " nil t))
;		    (goto-char (point-min))
;		    (cond ((search-forward "_" nil t)
;			   ;; Handle the *idiotic* use of underlines as spaces.
;			   ;; Example: fml@foo.bar.dom (First_M._Last)
;			   (goto-char (point-min))
;			   (while (search-forward "_" nil t)
;			     (replace-match " " t)))
;			  ((search-forward "." nil t)
;			   ;; Fix . used as space
;			   ;; Example: danj1@cb.att.com (daniel.jacobson)
;			   (goto-char (point-min))
;			   (while (re-search-forward mail-extr-bad-dot-pattern nil t)
;			     (replace-match "\\1 \\2" t))))))
	     )
	    
	    ;; Otherwise we try to get the name from the mailbox portion
	    ;; of the address.
	    ;; Example: First_M_Last@foo.bar.dom
Eric S. Raymond's avatar
Eric S. Raymond committed
1245 1246
	    (t
	     ;; *** Work in canon buffer instead?  No, can't.  Hmm.
1247 1248
	     (goto-char (point-max))
	     (narrow-to-region (point) (point))
Eric S. Raymond's avatar
Eric S. Raymond committed
1249 1250 1251
	     (insert-buffer-substring canonicalization-buffer
				      mbox-beg mbox-end)
	     (goto-char (point-min))
1252 1253 1254 1255
	     
	     ;; Example: First_Last.XXX@foo.bar.dom
	     (setq \.-ends-name (re-search-forward "[_0-9]" nil t))
	     
Eric S. Raymond's avatar
Eric S. Raymond committed
1256
	     (goto-char (point-min))
1257 1258 1259 1260

	     (if (not mail-extr-mangle-uucp)
		 (modify-syntax-entry ?! "w" (syntax-table)))

Eric S. Raymond's avatar
Eric S. Raymond committed
1261
	     (while (progn
1262
		      (mail-extr-skip-whitespace-forward)
Eric S. Raymond's avatar
Eric S. Raymond committed
1263 1264 1265 1266 1267
		      (not (eobp)))
	       (setq char (char-after (point)))
	       (cond
		((eq char ?\")
		 (setq quote-beg (point))
1268
		 (or (mail-extr-safe-move-sexp 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1269 1270 1271 1272 1273 1274
		     ;; TODO: handle this error condition!!!!!
		     (forward-char 1))
		 ;; take into account deletions
		 (setq quote-end (- (point) 2))
		 (save-excursion
		   (backward-char 1)
1275
		   (mail-extr-delete-char 1)
Eric S. Raymond's avatar
Eric S. Raymond committed
1276
		   (goto-char quote-beg)
1277 1278 1279
		   (mail-extr-delete-char 1))
		 (mail-extr-undo-backslash-quoting quote-beg quote-end)
		 (or (eq ?\  (char-after (point)))
Eric S. Raymond's avatar
Eric S. Raymond committed
1280
		     (insert " "))
1281
;;		 (setq mailbox-name-processed-flag t)
Eric S. Raymond's avatar
Eric S. Raymond committed
1282 1283
		 (setq \.-ends-name t))
		((eq char ?.)
1284
		 (if (memq (char-after (1+ (point))) '(?_ ?=))
Eric S. Raymond's avatar
Eric S. Raymond committed
1285 1286
		     (progn
		       (forward-char 1)
1287 1288
		       (mail-extr-delete-char 1)
		       (insert ?\ ))
Eric S. Raymond's avatar
Eric S. Raymond committed
1289 1290
		   (if \.-ends-name
		       (narrow-to-region (point-min) (point))
1291 1292 1293 1294
		     (mail-extr-delete-char 1)
		     (insert " ")))
;;		 (setq mailbox-name-processed-flag t)
		 )
Eric S. Raymond's avatar
Eric S. Raymond committed
1295
		((memq (char-syntax char) '(?. ?\\))
1296 1297 1298 1299
		 (mail-extr-delete-char 1)
		 (insert " ")
;;		 (setq mailbox-name-processed-flag t)
		 )
Eric S. Raymond's avatar
Eric S. Raymond committed
1300 1301 1302 1303
		(t
		 (setq atom-beg (point))
		 (forward-word 1)
		 (setq atom-end (point))
1304
		 (goto-char atom-beg)
Eric S. Raymond's avatar
Eric S. Raymond committed
1305 1306
		 (save-restriction
		   (narrow-to-region atom-beg atom-end)
1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354