GitLab

* src/module-env-28.h: Add new module environment functions to
module environment for Emacs 28.

* src/emacs-module.h.in: Document that 'emacs_finalizer' also works
for function finalizers.

* src/emacs-module.c (CHECK_MODULE_FUNCTION): New function.
(struct Lisp_Module_Function): Add finalizer data member.
(module_make_function): Initialize finalizer.
(module_get_function_finalizer)
(module_set_function_finalizer): New module environment functions.
(module_finalize_function): New function.
(initialize_environment): Initialize new environment functions.

* src/alloc.c (cleanup_vector): Call potential module function
finalizer during garbage collection.

* test/data/emacs-module/mod-test.c (signal_error): New helper
function.
(memory_full): Use it.
(finalizer): New example function finalizer.
(Fmod_test_make_function_with_finalizer)
(Fmod_test_function_finalizer_calls): New test module functions.
(emacs_module_init): Define them.

* test/src/emacs-module-tests.el (module/function-finalizer): New unit
test.

* doc/lispref/internals.texi (Module Functions): Document new
functionality.
(Module Misc): Move description of 'emacs_finalizer' type to 'Module
Functions' node, and add a reference to it.

* etc/NEWS: Mention new functionality.

Run "TZ=UTC0 admin/update-copyright $(git ls-files)".

On macOS, shared libraries typically have the suffix .dylib.  This
commit switches the module suffix to .dylib on Darwin to account for
that.  To also support the .so suffix, introduce the concept of a
secondary module suffix.

* configure.ac: Switch MODULES_SUFFIX to .dylib for Darwin, introduce
MODULES_SECONDARY_SUFFIX.

* src/lread.c (Fload, syms_of_lread): Also use
MODULES_SECONDARY_SUFFIX if defined.

* test/src/emacs-module-tests.el (module-darwin-secondary-suffix): New
unit test.

* test/data/emacs-module/mod-test.c (extract_big_integer): Fix
calculation of size of 'magnitude' when 'emacs_limb_t' is wider
than 'unsigned long'.

* test/src/emacs-module-tests.el
(module--test-assertions--load-non-live-object)
(module--test-assertions--call-emacs-from-gc): On MS-Windows,
check also mod-test-emacs with ".exe" appended, before
skipping the tests.

Without this fix, time arithmetic yielded results that were not
mathematically accurate, even though the exact results were
representable; for example, (time-add 0 1e-13) yielded a timestamp
equal to 0 instead of to 1e-13.
* lisp/timezone.el (timezone-time-from-absolute):
Let time-add do its thing rather than using floating point
internally, which has rounding errors.  We now have bignums and so
don’t need floating point to avoid overflow issues.
* src/timefns.c (timeform_sub_ps_p): New function.
(time_arith): If either argument is a float, represent the
result exactly instead of discarding sub-ps info.
* test/lisp/timezone-tests.el (timezone-tests-time-from-absolute):
Don’t assume (HI LO US PS) timestamp format.
* test/src/emacs-module-tests.el (mod-test-add-nanosecond/valid):
Don’t assume that time-add discards sub-ns info.
* test/src/timefns-tests.el (time-rounding-tests):
Add regression test to detect time-add rounding bug.

* test/src/emacs-module-tests.el (mod-test-nanoseconds):
Don’t assume -1000000000 is a fixnum.

* admin/charsets/mapconv:
* build-aux/gitlog-to-changelog (parse_amend_file, git_dir_option):
* lisp/progmodes/verilog-mode.el (verilog-delay-re):
(verilog-type-font-keywords, verilog-read-always-signals-recurse):
(verilog-is-number):
* lisp/progmodes/vera-mode.el (vera-font-lock-keywords):
* test/src/emacs-module-tests.el (mod-test-sum-test):
* lisp/xml.el: (xml--entity-replacement-text):
* lisp/version.el (emacs-repository-version-git):
* lisp/textmodes/sgml-mode.el (sgml-quote):
* lisp/textmodes/css-mode.el (css-escapes-re)
(css--colors-regexp):
* lisp/progmodes/prolog.el (prolog-syntax-propertize-function):
* lisp/progmodes/hideif.el (hif-token-regexp, hif-tokenize):
* lisp/progmodes/ebnf-dtd.el: (ebnf-dtd-attlistdecl)
(ebnf-dtd-entitydecl, ebnf-dtd-lex):
* lisp/progmodes/ebnf-ebx.el (ebnf-ebx-hex-character):
* lisp/progmodes/ebnf-abn.el (ebnf-abn-character):
* lisp/progmodes/cperl-mode.el (cperl-highlight-charclass)
(cperl-find-pods-heres):
* lisp/progmodes/cc-mode.el (c-maybe-quoted-number-head)
(c-maybe-quoted-number, c-parse-quotes-before-change)
(c-parse-quotes-after-change, c-quoted-number-head-before-point)
(c-quoted-number-straddling-point):
* lisp/progmodes/ada-mode.el (featurep, ada-in-numeric-literal-p)
(ada-font-lock-keywords):
* lisp/org/org-mobile.el (org-mobile-copy-agenda-files)
* lisp/org/org-table.el (org-table-number-regexp):
(org-mobile-update-checksum-for-capture-file):
* lisp/nxml/xsd-regexp.el (xsdre-gen-categories):
* lisp/nxml/xmltok.el (let*):
* lisp/nxml/rng-xsd.el (rng-xsd-convert-hex-binary)
(rng-xsd-convert-any-uri):
* lisp/nxml/rng-uri.el (rng-uri-file-name-1)
(rng-uri-unescape-multibyte, rng-uri-unescape-unibyte)
(rng-uri-unescape-unibyte-match)
(rng-uri-unescape-unibyte-replace):
* lisp/nxml/rng-cmpct.el (rng-c-process-escapes):
* lisp/nxml/nxml-maint.el (nxml-insert-target-repertoire-glyph-set):
* lisp/net/shr-color.el (shr-color->hexadecimal):
* lisp/mail/rfc2231.el (rfc2231-decode-encoded-string):
* lisp/international/mule-cmds.el (read-char-by-name):
* lisp/htmlfontify.el (hfy-hex-regex):
* lisp/gnus/nneething.el (nneething-decode-file-name):
* lisp/gnus/mml-sec.el (mml-secure-find-usable-keys):
* lisp/gnus/gnus-art.el (gnus-button-mid-or-mail-heuristic-alist):
* lisp/faces.el (read-color):
* lisp/epg.el (epg--status-ERRSIG, epg--status-VALIDSIG)
(epg--status-SIG_CREATED, epg--decode-percent-escape)
(epg--decode-hexstring, epg--decode-quotedstring)
(epg-dn-from-string):
* lisp/emulation/cua-rect.el (cua-incr-rectangle):
* lisp/dnd.el (dnd-unescape-uri):
* lisp/cedet/semantic/lex.el (semantic-lex-number-expression):
* lisp/cedet/semantic/java.el (semantic-java-number-regexp):
* lisp/calc/calc-lang.el (pascal):
* lisp/calc/calc-ext.el (math-read-number-fancy):
* lisp/calc/calc-aent.el (math-read-token):
Replace various combinations of [0-9a-fA-F] with [[:xdigit:]].
(Bug#36167)

* test/data/emacs-module/mod-test.c (Fmod_test_sleep_until): Use
extract_time for time argument.
(signal_wrong_type_argument): Remove unused function.

* test/src/emacs-module-tests.el (mod-test-sleep-until): Remove
unnecessary ‘float-time’.

* test/src/emacs-module-tests.el (multiply-string): Remove unused
variable ‘i’.

* test/src/emacs-module-tests.el: Add package name and standard
sections.
(multiply-string): Add a docstring.

* doc/lispref/internals.texi (Module Values): Clarify that the
truncation is towards negative infinity.

* test/data/emacs-module/mod-test.c (Fmod_test_nanoseconds): Add test
function.
(emacs_module_init): Define it.

* test/src/emacs-module-tests.el (mod-test-nanoseconds): New unit test.

make_time is documented to return a (TICKS . HZ) pair, so we can’t use
make_lisp_time.  Introduce a new conversion function instead.

* src/emacs-module.c (module_make_time): Use timespec_to_lisp to
correct return type.

* src/timefns.c (timespec_to_lisp): New function.
(make_lisp_time): Use it.

* test/src/emacs-module-tests.el (mod-test-add-nanosecond/valid):
Check return type.

* src/module-env-27.h: Add new module functions to convert big
integers.

* src/emacs-module.h.in (emacs_mpz): Define if GMP is available.

* src/emacs-module.c (module_extract_big_integer)
(module_make_big_integer): New functions.
(initialize_environment): Use them.

* test/data/emacs-module/mod-test.c (Fmod_test_double): New test
function.
(emacs_module_init): Define it.

* test/src/emacs-module-tests.el (mod-test-double): New unit test.

* doc/lispref/internals.texi (Module Values): Document new functions.

Time values are a fundamental data type, and such conversions are hard
to implement within modules because of the various forms of time
values in Emacs Lisp.  Adding dedicated conversion functions can
significantly simplify module code dealing with times.

This approach uses nanosecond precision.  While Emacs in theory has
support for higher-precision time values, in practice most languages
and standards, such as POSIX, C, Java, and Go, have settled on
nanosecond-precision integers to represent time.

* src/emacs-module.h.in: Add header for struct timespec.

* src/module-env-27.h: Add module functions for time conversion.

* src/emacs-module.c (module_extract_time, module_make_time): New
functions.
(initialize_environment): Use them.

* test/data/emacs-module/mod-test.c (Fmod_test_add_nanosecond): New
test function.
(emacs_module_init): Define it.

* test/src/emacs-module-tests.el (mod-test-add-nanosecond/valid)
(mod-test-add-nanosecond/nil, mod-test-add-nanosecond/invalid): New
unit tests.

* doc/lispref/internals.texi (Module Values): Document time
conversion functions.

This reverts commit 093d3e78,
which reverted ee7ad83f,
which reverted 3eb93c07.

This reverts commit ee7ad83f.

There was no consensus on reverting
3eb93c07, so doing that will have to
wait until the discussion ends.

This reverts commit 3eb93c07.

There was no consensus for that commit, see
https://lists.gnu.org/archive/html/emacs-devel/2016-01/msg00150.html.
Also, reverting this commit should fix Bug#31238.

pending_signals is often set if no quit is pending.  This results in
bugs in module code if the module returns but no quit is actually
pending.

As a better alternative, add a new process_input environment function
for Emacs 27.  That function processes signals (like maybe_quit).

* configure.ac: Add module snippet for Emacs 27.

* src/module-env-27.h: New file.

* src/emacs-module.h.in: Add process_input function to environment
interface.

* src/emacs-module.c (module_should_quit): Use QUITP macro to check
whether the caller should quit.
(module_process_input): New function.
(initialize_environment): Use it.

* src/eval.c: Remove obsolete comment.

* test/data/emacs-module/mod-test.c (signal_wrong_type_argument)
(signal_errno): New helper functions.
(Fmod_test_sleep_until): New test module function.

* test/src/emacs-module-tests.el (mod-test-sleep-until): New unit
test.

* doc/lispref/internals.texi (Module Misc): Document process_input.

Run 'TZ=UTC0 admin/update-copyright $(git ls-files)'.

* src/emacs-module.c (module_free_global_ref): Compare a value to be
freed with all entries of the list.

* test/data/emacs-module/mod-test.c (Fmod_test_globref_free): New
function.
(emacs_module_init): Make it accessible from Lisp.
* test/src/emacs-module-tests.el (mod-test-globref-free-test): New
test which uses it.

* test/src/emacs-module-tests.el (mod-test-sum-test): Update
for bignums.

* doc/lispref/control.texi (Iteration):
* doc/misc/cl.texi (Iteration): Document deprecation of its use.
* doc/lispintro/emacs-lisp-intro.texi (dotimes):
* test/src/emacs-module-tests.el (multiply-string):
* test/lisp/filenotify-tests.el (file-notify-test07-many-events):
Place RESULT field after the form.

Fix a portability bug when emacs-module-tests.el is byte-compiled
with a 32-bit Emacs (where #x20000000 evaluates to a
floating-point number) and then is run on a 64-bit Emacs (where
the floating-point number causes a test failure).
* test/src/emacs-module-tests.el (mod-test-sum-test):
Don’t assume #x20000000 can be represented as an Emacs integer.

* src/lread.c (Fload): Don't defer to module-load immediately when
encountering a module, but use the normal loading machinery to
properly set up load-history, check for recursive loads, print
messages, etc.

* test/src/emacs-module-tests.el (module/load-history): New test.
(module/describe-function-1): Adapt test.

* etc/NEWS: Mention fixed behavior.

This puts functions defined in the module into the 'load-history'.

* test/data/emacs-module/mod-test.c (bind_function): Use 'defalias'
instead of 'fset'.

* test/src/emacs-module-tests.el (module/describe-function-1): Adapt
unit test.

* lisp/help-fns.el (help-fns-function-description-header): Handle
module functions.

* test/src/emacs-module-tests.el (module/describe-function-1): New
test.

* lisp/emacs-lisp/cl-preloaded.el (cl--typeof-types): Add module
types.

* test/src/emacs-module-tests.el (emacs-module-tests--generic): New
helper function.
(module-function-object, mod-test-userptr-fun-test): Test that type
dispatching works with module types.

* test/src/emacs-module-tests.el (module-function-object): Verify that
a module function is treated as a function.

* test/Makefile.in (test_module_dir): Build the test module library in
a subdirectory of the build directory (not the source tree).
(MODULE_CFLAGS): Fix location of emacs-module.h header file.
(test_module): Move built library out of the source tree.
* test/src/emacs-module-tests.el (mod-test-file): Locate the test module
library relative to the running Emacs executable.

Run admin/update-copyright.

Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November.  Change these FTP URLs to https://ftp.gnu.org
instead.  Make similar changes for URLs to other organizations moving
away from FTP.  Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.

* src/w32fns.c (syms_of_w32fns) <w32-disable-abort-dialog>: New
variable.
(emacs_abort): If w32-disable-abort-dialog is non-nil, abort right
away, without displaying the Abort dialog, which waits for the user.

* test/src/emacs-module-tests.el (module--test-assertion): Run the
inferior Emacs with the w32 abort dialog disabled.  Expect the
status of the aborted Emacs sub-process to be 3 on MS-Windows and
2 on MS-DOS.

* test/src/emacs-module-tests.el
(module--test-assertions--load-non-live-object)
(module--test-assertions--call-emacs-from-gc):
Avoid test failures due to backtraces.

It's technically possible to write a user pointer finalizer that calls
into Emacs module functions.  This would be disastrous because it
would allow arbitrary Lisp code to run during garbage collection.
Therefore extend the module assertions to check for this case.

* src/emacs-module.c (module_assert_thread): Also check whether a
garbage collection is in progress.

* test/data/emacs-module/mod-test.c (invalid_finalizer)
(Fmod_test_invalid_finalizer): New test module functions.
(emacs_module_init): Register new test function.

* test/src/emacs-module-tests.el (module--test-assertion)
(module--with-temp-directory): New helper macros.
(module--test-assertions--load-non-live-object): Rename existing
unit test, use helper macros.
(module--test-assertions--call-emacs-from-gc): New unit test.

* test/src/emacs-module-tests.el (module--test-assertions):
Rephrase final check to give a more informative failure.

* test/src/emacs-module-tests.el (module--test-assertions):
Use a temporary directory to contain any core dumps.

* test/src/emacs-module-tests.el (module--test-assertions):
Don't rely on the precise form of an "Abort" message.

Add a new command-line option '-module-assertions' that users can
enable developing or debugging a module.  If this option is present,
Emacs performs additional checks to verify that modules fulfill their
requirements.  These checks are expensive and crash Emacs if modules
are invalid, so disable them by default.

This is a command-line option instead of an ordinary variable because
changing it while Emacs is running would cause data structure
imbalances.

* src/emacs.c (main): New command line option '-module-assertions'.

* src/emacs-module.c (module_assert_main_thread)
(module_assert_runtime, module_assert_env, module_assert_value):
New functions to assert module requirements.
(syms_of_module): New uninterned variable 'module-runtimes'.
(init_module_assertions, in_main_thread, module_abort): New helper
functions.
(initialize_environment): Initialize value list.  If assertions are
enabled, use a heap-allocated environment object.
(finalize_environment): Add assertion that environment list is never
empty.
(finalize_runtime_unwind): Pop module runtime object stack.
(value_to_lisp): Assert that the value is valid.
(lisp_to_value): Record new value if assertions are enabled.
(mark_modules): Mark allocated object list.
(MODULE_FUNCTION_BEGIN_NO_CATCH)
(module_non_local_exit_check, module_non_local_exit_clear)
(module_non_local_exit_get, module_non_local_exit_signal)
(module_non_local_exit_throw): Assert thread and environment.
(module_get_environment): Assert thread and runtime.
(module_make_function, module_funcall, module_intern)
(module_funcall, module_make_integer, module_make_float)
(module_make_string, module_make_user_ptr, module_vec_get)
(funcall_module, Fmodule_load): Adapt callers.
(module_make_global_ref): If assertions are enabled, use the global
environment to store global values.
(module_free_global_ref): Remove value from global value list.

* test/Makefile.in (EMACSOPT): Enable module assertions when testing
modules.

* test/data/emacs-module/mod-test.c (Fmod_test_invalid_store)
(Fmod_test_invalid_load): New functions to test module assertions.
(emacs_module_init): Bind the new functions.

* test/src/emacs-module-tests.el (mod-test-emacs): New constant for
the Emacs binary file.
(mod-test-file): New constant for the test module file name.
(module--test-assertions): New unit test.