GitLab

* bidi.c (bidi_cache_size): Now EMACS_INT, not size_t.
(elsz): Now a signed constant, not a size_t var.  We prefer signed
types to unsigned, to avoid integer comparison confusion.  Without
this change, GCC 4.6.1 with -Wunsafe-loop-optimizations complains
"cannot optimize loop, the loop counter may overflow", a symptom
of the confusion.

 src/bidi.c (bidi_dump_cached_states): Fix format of displaying
 bidi_cache_idx.

 src/bidi.c (bidi_cache_fetch_state, bidi_cache_search)
 (bidi_cache_find_level_change, bidi_cache_ensure_space)
 (bidi_cache_iterator_state, bidi_cache_find)
 (bidi_find_other_level_edge, bidi_cache_start_stack): All
 variables related to cache indices are now EMACS_INT.

See <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9079#11>.

(bidi_cache_idx, bidi_cache_last_idx, bidi_cache_fetch_state)
(bidi_cache_search, bidi_cache_find_level_change)
(bidi_cache_iterator_state, bidi_cache_find, bidi_find_other_level_edge)
(bidi_dump_cached_states):
Don't arbitrarily limit cache indexes to int; use ptrdiff_t instead.
(bidi_cache_size): Use ptrdiff_t rather than size_t, as we prefer
signed integers.
(elsz): Make it a (signed) constant.
(bidi_cache_iterator_state): Check for size-calculation overflow.

Solves crashes due to overflow of bidi cache stack.
Make sure cache is saved and restored around all temporary iterations.

 src/dispnew.c (buffer_posn_from_coords): Save and restore the bidi
 cache around display iteration.
 src/window.c (Fwindow_end, window_scroll_pixel_based)
 (displayed_window_lines, Frecenter): Save and restore the bidi
 cache around display iteration.
 src/bidi.c (bidi_unshelve_cache): Ensure we have enough space before
 restoring the shelved cache.
 (bidi_cache_ensure_space): Don't assume the required size is just
 one BIDI_CACHE_CHUNK away.
 src/xdisp.c (back_to_previous_visible_line_start, reseat_1)
 (init_iterator): Empty the bidi cache "stack".

Not tested, just compiled.

 src/bidi.c (bidi_shelve_cache, bidi_unshelve_cache): New functions.
 src/dispextern.h (bidi_shelve_cache, bidi_unshelve_cache): Declare
 prototypes.
 src/xdisp.c (SAVE_IT, RESTORE_IT): New macros.
 (pos_visible_p, face_before_or_after_it_pos)
 (back_to_previous_visible_line_start)
 (move_it_in_display_line_to, move_it_in_display_line)
 (move_it_to, move_it_vertically_backward, move_it_by_lines)
 (try_scrolling, redisplay_window, display_line): Use them when
 saving a temporary copy of the iterator and restoring it back.

Hit a MAJOR design problem in pushing and popping bidi iterator state.

 src/xdisp.c (reseat_1): Call bidi_init_it to resync the bidi
 iterator with IT's position.
 (handle_stop, back_to_previous_visible_line_start, reseat_1):
 Reset the from_disp_prop_p flag.
 src/bidi.c (bidi_cache_search): Don't assume bidi_cache_last_idx is
 always valid if bidi_cache_idx is valid.
 (bidi_cache_find_level_change): xassert that bidi_cache_last_idx
 is valid if it's going to be used.

Support bidi reordering of unibyte strings.  Fix crash displaying "All" in mode line of an empty buffer.

 src/dispextern.h (struct bidi_string_data): New member `unibyte'.
 src/xdisp.c (handle_single_display_spec, next_overlay_string)
 (get_overlay_strings_1, reseat_1, reseat_to_string)
 (push_display_prop): Set up the `unibyte' member of bidi_it.string
 correctly.  Don't assume unibyte strings are not bidi-reordered.
 (compute_display_string_pos)
 (compute_display_string_end): Fix handling the case of C string.
 src/bidi.c (bidi_count_bytes, bidi_char_at_pos): Accept an
 additional argument UNIBYTE, and support unibyte strings.  All
 callers changed.
 (bidi_fetch_char): Support unibyte strings.

 src/bidi.c (bidi_paragraph_init): Test for ZV_BYTE before calling
 bidi_at_paragraph_end, since fast_looking_at doesn't like to be
 called at ZV.

 src/xdisp.c (set_iterator_to_next, get_visually_first_element): Use
 it->bidi_it.string.schars rather than it->string_nchars when
 testing whether we're beyond string end, because string_nchars is
 zero for strings that come from overlays and display properties.
 src/bidi.c (bidi_cache_iterator_state): Fix a bug with testing
 character positions against the cached range, when we use a
 stacked cache.
 src/dispextern.h (struct iterator_stack_entry): New member
 paragraph_embedding.
 src/xdisp.c (push_it, pop_it): Save and restore it.

Not tested yet, just compiled.

 src/xdisp.c (handle_single_display_spec, next_overlay_string)
 (get_overlay_strings_1, push_display_prop): Set up the bidi
 iterator for displaying display or overlay strings.
 (forward_to_next_line_start): Don't use the shortcut if
 bidi-iterating.
 (back_to_previous_visible_line_start): If handle_display_prop
 pushed the iterator stack, restore the internal state of the bidi
 iterator by calling bidi_pop_it same number of times.
 (reseat_at_next_visible_line_start): If ON_NEWLINE_P is non-zero,
 and we are bidi-iterating, don't decrement the iterator position;
 instead, set the first_elt flag in the bidi iterator, to produce
 the same effect.
 (reseat_1): Remove redundant setting of string_from_display_prop_p.
 (push_display_prop): xassert that we are iterating a buffer.
 (push_it, pop_it): Save and restore the state of the
 bidi iterator.  Save and restore the bidi_p flag.
 (pop_it): Iterate out of display property for string iteration as
 well.
 (iterate_out_of_display_property): Support iteration over strings.
 (handle_single_display_spec): Set up it->bidi_it for iteration
 over a display string, and call bidi_init_it.
 src/bidi.c (bidi_cache_start_stack, bidi_push_it): Use IT_STACK_SIZE.
 src/dispextern.h (struct iterator_stack_entry): New member bidi_p.
 (struct it): Member bidi_p is now a bit field 1 bit wide.

 src/bidi.c (bidi_push_it, bidi_pop_it): New functions.
 (bidi_initialize): Initialize the bidi cache start stack pointer.
 (bidi_cache_ensure_space): New function, refactored from part of
 bidi_cache_iterator_state.
 src/dispextern.h (bidi_push_it, bidi_pop_it): Add prototypes.

No changes in functionality.

GCPRO Lisp string inside bidi.c.
Force L2R direction in buffer menu buffer.

 src/xdisp.c (tool_bar_lines_needed, redisplay_tool_bar)
 (display_menu_bar): Force left-to-right direction.  Add a FIXME
 comment for making that be controlled by a user option.
 src/bidi.c (bidi_move_to_visually_next): GCPRO the Lisp string we
 are iterating.
 lisp/buff-menu.el (Buffer-menu-mode, list-buffers-noselect): Force
 left-to-right paragraph direction.

Composed characters still don't work.

 src/xdisp.c (init_iterator): Don't initialize it->bidi_p for strings
 here.
 (reseat_to_string): Initialize it->bidi_p for strings here.
 (next_element_from_string, next_element_from_c_string)
 (next_element_from_buffer): Add xassert's for correspondence
 between IT's object being iterated and it->bidi_it.string
 structure.
 src/bidi.c (bidi_level_of_next_char): Fix the logic for looking up
 the sentinel state in the cache.

Bugs in mode-line display still there.

 src/xdisp.c (compute_display_string_pos)
 (compute_display_string_end, reseat_to_string): Don't assume
 it->bidi_it.string.s always points to string.lstring's data.
 src/bidi.c (bidi_fetch_char, bidi_paragraph_init)
 (bidi_resolve_explicit_1, bidi_resolve_explicit)
 (bidi_resolve_weak, bidi_level_of_next_char): Don't assume
 string.s always points to string.lstring's data.

Just compiled, not yet tested.

 src/xdisp.c (compute_display_string_pos): First arg is now struct
 `text_pos *'; all callers changed.  Support display properties on
 Lisp strings.
 (compute_display_string_end): Support display properties on Lisp
 strings.
 (init_iterator, reseat_1, reseat_to_string): Initialize the
 string.bufpos member to 0 (zero, for compatibility with IT_CHARPOS
 when iterating on a string not from display properties).
 src/bidi.c (bidi_fetch_char): Support strings with display
 properties.
 src/dispextern.h (struct bidi_string_data): New member bufpos.
 (compute_display_string_pos): Update prototype.

For now, ifdef'ed away.  Some more testing.

 src/bidi.c (bidi_level_of_next_char): Allow the sentinel "position"
 to pass the test for valid cached positions.
 src/xdisp.c (init_iterator): Call bidi_init_it only of a valid
 buffer position was specified.  Initialize paragraph_embedding to L2R.
 (reseat_to_string): Initialize the bidi iterator (for now ifdef'ed out).
 (display_string): If we need to ignore text properties of
 LISP_STRING, set IT->stop_charpos to IT->end_charpos.  (The
 original value of -1 will not work with bidi.)
 src/dispextern.h (struct bidi_string_data): New member lstring.

 src/bidi.c (bidi_paragraph_info): Delete unused struct.
 (bidi_cache_idx, bidi_cache_last_idx): Declare EMACS_INT.
 (bidi_cache_start): New variable.
 (bidi_cache_reset): Reset bidi_cache_idx to bidi_cache_start, not
 to zero.
 (bidi_cache_fetch_state, bidi_cache_search)
 (bidi_cache_find_level_change, bidi_cache_iterator_state)
 (bidi_cache_find, bidi_peek_at_next_level)
 (bidi_level_of_next_char, bidi_find_other_level_edge)
 (bidi_move_to_visually_next): Compare cache index with
 bidi_cache_start rather than with zero.
 (bidi_fetch_char): Accept new argument STRING; all callers
 changed.  Support iteration over a string.
 (bidi_paragraph_init, bidi_resolve_explicit_1)
 (bidi_resolve_explicit, bidi_resolve_weak)
 (bidi_level_of_next_char, bidi_move_to_visually_next): Support
 iteration over a string.
 (bidi_set_sor_type, bidi_resolve_explicit_1)
 (bidi_resolve_explicit, bidi_type_of_next_char): ignore_bn_limit
 can now be zero (for strings); special values 0 and -1 were
 changed to -1 and -2, respectively.
 (bidi_char_at_pos): New function.
 (bidi_paragraph_init, bidi_resolve_explicit, bidi_resolve_weak):
 Call it instead of FETCH_MULTIBYTE_CHAR.
 (bidi_move_to_visually_next): Abort if charpos or bytepos were not
 initialized to valid values.
 (bidi_init_it): Don't initialize charpos and bytepos with invalid
 values.
 src/xdisp.c (compute_display_string_pos)
 (compute_display_string_end): Accept additional argument STRING.
 (init_iterator, reseat_1): Initialize bidi_it->string.s to NULL.
 (reseat_to_string): Initialize bidi_it->string.s and
 bidi_it->string.schars.
 src/dispextern.h (struct bidi_string_data): New structure.
 (struct bidi_it): New member `string'.  Make flag members be 1-bit
 fields, and put them last in the struct.
 (compute_display_string_pos, compute_display_string_end): Update
 prototypes.

 src/bidi.c (bidi_level_of_next_char): clen should be EMACS_NT, not int.

 src/bidi.c (bidi_fetch_char_advance): Remove unused and
 unimplemented function.

Use 'inline', not 'INLINE'.
* configure.in, autogen/config.in (INLINE): Remove.
[lib-src/ChangeLog]
Use 'inline', not 'INLINE'.
* etags.c (hash): Now inline unconditionally.
* make-docfile.c (put_char): inline, not INLINE.
[nt/ChangeLog]
Use 'inline', not 'INLINE'.
* config.nt (INLINE): Remove.
[src/ChangeLog]
Use 'inline', not 'INLINE'.
* alloc.c, fontset.c (INLINE): Remove.
* alloc.c, bidi.c, charset.c, coding.c, dispnew.c, fns.c, image.c:
* intervals.c, keyboard.c, process.c, syntax.c, textprop.c, w32term.c:
* xdisp.c, xfaces.c, xterm.c: Replace all uses of INLINE with inline.
* gmalloc.c (register_heapinfo): Use inline unconditionally.
* lisp.h (LISP_MAKE_RVALUE): Use inline, not __inline__.

as the display engine.  Tested OK with display properties whose
value is a list.

 src/dispextern.h (struct bidi_it): New member frame_window_p.
 (bidi_init_it, compute_display_string_pos): Update prototypes.
 src/bidi.c (bidi_fetch_char): Accept additional argument
 FRAME_WINDOW_P and pass it to compute_display_string_pos.  All
 callers changed.
 (bidi_init_it): Accept additional argument FRAME_WINDOW_P and use
 it to initialize the frame_window_p member of struct bidi_it.
 src/xdisp.c (handle_display_spec): New function, refactored from the
 last portion of handle_display_prop.
 (compute_display_string_pos): Accept additional argument
 FRAME_WINDOW_P.  Call handle_display_spec to determine whether the
 value of a `display' property is a "replacing spec".
 (handle_single_display_spec): Accept 2 additional arguments BUFPOS
 and FRAME_WINDOW_P.  If IT is NULL, don't set up the iterator from
 the display property, but just return a value indicating whether
 the display property will replace the characters it covers.
 (Fcurrent_bidi_paragraph_direction): Initialize the nchars and
 frame_window_p members of struct bidi_it.

Cursor positioning is not yet right near the overlay.

 src/xdisp.c (compute_display_string_pos): Non-trivial implementation.
 (compute_display_string_end): New function.
 src/dispextern.h (compute_display_string_end): Declare prototype.
 src/bidi.c (bidi_resolve_explicit_1): Use ZV for disp_pos.
 (bidi_fetch_char): Implement support for runs of characters
 covered by display strings.

 src/bidi.c (bidi_fetch_char): Accept also character position
 corresponding to BYTEPOS.  All callers changed.
 (bidi_cache_iterator_state, bidi_resolve_explicit_1)
 (bidi_resolve_explicit, bidi_resolve_weak)
 (bidi_level_of_next_char, bidi_move_to_visually_next): Abort if
 bidi_it->nchars is non-positive.
 (bidi_level_of_next_char): Don't try to lookup the cache for the
 next/previous character if nothing is cached there yet, or if we
 were just reseat()'ed to a new position.
 (bidi_paragraph_init, bidi_resolve_explicit_1)
 (bidi_level_of_next_char): Fix arguments in the calls to
 bidi_fetch_char.

Only compiled, not tested.

 src/xdisp.c (compute_display_string_pos): New function.
 (reseat_1): Initialize bidi_it.disp_pos.
 src/bidi.c (bidi_copy_it): Use offsetof.
 (bidi_fetch_char, bidi_fetch_char_advance): New functions.
 (bidi_cache_search, bidi_cache_iterator_state)
 (bidi_paragraph_init, bidi_resolve_explicit, bidi_resolve_weak)
 (bidi_level_of_next_char, bidi_move_to_visually_next): Support
 character positions inside a run of characters covered by a
 display string.
 (bidi_paragraph_init, bidi_resolve_explicit_1)
 (bidi_level_of_next_char): Call bidi_fetch_char and
 bidi_fetch_char_advance instead of FETCH_CHAR and
 FETCH_CHAR_ADVANCE.
 (bidi_init_it): Initialize new members.
 (LRE_CHAR, RLE_CHAR, PDF_CHAR, LRO_CHAR, RLO_CHAR): Remove macro
 definitions.
 (bidi_explicit_dir_char): Lookup character type in bidi_type_table,
 instead of using explicit *_CHAR codes.
 (bidi_resolve_explicit, bidi_resolve_weak): Use
 FETCH_MULTIBYTE_CHAR instead of FETCH_CHAR, as reordering of
 bidirectional text is supported only in multibyte buffers.
 src/dispextern.h (struct bidi_it): New members nchars and disp_pos.
 ch_len is now EMACS_INT.
 (compute_display_string_pos): Declare prototype.

* lisp.h (pI): New macro, generalizing old pEd macro to other
conversion specifiers.  For example, use "...%"pI"d..." rather
than "...%"pEd"...".
(pEd): Remove.  All uses replaced with similar uses of pI.
* src/m/amdx86-64.h, src/m/ia64.h, src/m/ibms390x.h: Likewise.
* alloc.c (check_pure_size): Don't overflow by converting size to int.
* bidi.c (bidi_dump_cached_states): Use pI to avoid cast.
* data.c (Fnumber_to_string): Use pI instead of if-then-else-abort.
* dbusbind.c (xd_append_arg): Use pI to avoid cast.
(Fdbus_method_return_internal, Fdbus_method_error_internal): Likewise.
* font.c (font_unparse_xlfd): Avoid potential buffer overrun on
64-bit hosts.
(font_unparse_xlfd, font_unparse_fcname): Use pI to avoid casts.
* keyboard.c (record_char, modify_event_symbol): Use pI to avoid casts.
* print.c (safe_debug_print, print_object): Likewise.
(print_object): Don't overflow by converting EMACS_INT or EMACS_UINT
to int.
Use pI instead of if-then-else-abort.  Use %p to avoid casts.
* process.c (Fmake_network_process): Use pI to avoid cast.
* region-cache.c (pp_cache): Likewise.
* xdisp.c (decode_mode_spec): Likewise.
* xrdb.c (x_load_resources) [USE_MOTIF]: Use pI to avoid undefined
behavior on 64-bit hosts with printf arg.
* xselect.c (x_queue_event): Use %p to avoid casts.
(x_stop_queuing_selection_requests): Likewise.
(x_get_window_property): Don't truncate byte count to an 'int'
when tracing.

so that it's not optimized away.
(bidi_ignore_explicit_marks_for_paragraph_level): Likewise.
* dispextern.h (bidi_dump_cached_states): Remove, since it's
exported only to the debugger.

* src/xfont.c:
* src/w32term.c:
* src/w32reg.c:
* src/w32inevt.c:
* src/w32heap.c:
* src/w32console.c:
* src/w16select.c:
* src/unexsol.c:
* src/term.c:
* src/sound.c:
* src/scroll.c (m):
* src/gtkutil.c:
* src/font.c:
* src/filelock.c:
* src/fileio.c:
* src/dosfns.c:
* src/dbusbind.c:
* src/bidi.c:
* src/callproc.c:
* src/process.c:
* src/msdos.c:
* src/charset.c: Do not include stdlib.h and string.h, config.h does it.

* configure.in: Include stdlib.h and string.h unconditionally.