GitLab

Time values are a fundamental data type, and such conversions are hard
to implement within modules because of the various forms of time
values in Emacs Lisp.  Adding dedicated conversion functions can
significantly simplify module code dealing with times.

This approach uses nanosecond precision.  While Emacs in theory has
support for higher-precision time values, in practice most languages
and standards, such as POSIX, C, Java, and Go, have settled on
nanosecond-precision integers to represent time.

* src/emacs-module.h.in: Add header for struct timespec.

* src/module-env-27.h: Add module functions for time conversion.

* src/emacs-module.c (module_extract_time, module_make_time): New
functions.
(initialize_environment): Use them.

* test/data/emacs-module/mod-test.c (Fmod_test_add_nanosecond): New
test function.
(emacs_module_init): Define it.

* test/src/emacs-module-tests.el (mod-test-add-nanosecond/valid)
(mod-test-add-nanosecond/nil, mod-test-add-nanosecond/invalid): New
unit tests.

* doc/lispref/internals.texi (Module Values): Document time
conversion functions.

This reverts commit 093d3e78,
which reverted ee7ad83f,
which reverted 3eb93c07.

This reverts commit ee7ad83f.

There was no consensus on reverting
3eb93c07, so doing that will have to
wait until the discussion ends.

This reverts commit 3eb93c07.

There was no consensus for that commit, see
https://lists.gnu.org/archive/html/emacs-devel/2016-01/msg00150.html.
Also, reverting this commit should fix Bug#31238.

pending_signals is often set if no quit is pending.  This results in
bugs in module code if the module returns but no quit is actually
pending.

As a better alternative, add a new process_input environment function
for Emacs 27.  That function processes signals (like maybe_quit).

* configure.ac: Add module snippet for Emacs 27.

* src/module-env-27.h: New file.

* src/emacs-module.h.in: Add process_input function to environment
interface.

* src/emacs-module.c (module_should_quit): Use QUITP macro to check
whether the caller should quit.
(module_process_input): New function.
(initialize_environment): Use it.

* src/eval.c: Remove obsolete comment.

* test/data/emacs-module/mod-test.c (signal_wrong_type_argument)
(signal_errno): New helper functions.
(Fmod_test_sleep_until): New test module function.

* test/src/emacs-module-tests.el (mod-test-sleep-until): New unit
test.

* doc/lispref/internals.texi (Module Misc): Document process_input.

Run 'TZ=UTC0 admin/update-copyright $(git ls-files)'.

* src/emacs-module.c (module_free_global_ref): Compare a value to be
freed with all entries of the list.

* test/data/emacs-module/mod-test.c (Fmod_test_globref_free): New
function.
(emacs_module_init): Make it accessible from Lisp.
* test/src/emacs-module-tests.el (mod-test-globref-free-test): New
test which uses it.

This puts functions defined in the module into the 'load-history'.

* test/data/emacs-module/mod-test.c (bind_function): Use 'defalias'
instead of 'fset'.

* test/src/emacs-module-tests.el (module/describe-function-1): Adapt
unit test.

* test/Makefile.in (abs_top_srcdir): Add variable, needed by
CPPFLAGS.
* test/data/emacs-module/mod-test.c: Include <limits.h>.
(pT, pZ, T_TYPE, Z_TYPE): Compatibility macros, for systems that
don't support %td and %zu format specs.
(emacs_module_init): Use compatibility macros to make the error
messages print meaningful values (and avoid compiler warnings).

Run admin/update-copyright.

Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November.  Change these FTP URLs to https://ftp.gnu.org
instead.  Make similar changes for URLs to other organizations moving
away from FTP.  Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.

It's technically possible to write a user pointer finalizer that calls
into Emacs module functions.  This would be disastrous because it
would allow arbitrary Lisp code to run during garbage collection.
Therefore extend the module assertions to check for this case.

* src/emacs-module.c (module_assert_thread): Also check whether a
garbage collection is in progress.

* test/data/emacs-module/mod-test.c (invalid_finalizer)
(Fmod_test_invalid_finalizer): New test module functions.
(emacs_module_init): Register new test function.

* test/src/emacs-module-tests.el (module--test-assertion)
(module--with-temp-directory): New helper macros.
(module--test-assertions--load-non-live-object): Rename existing
unit test, use helper macros.
(module--test-assertions--call-emacs-from-gc): New unit test.

* test/data/emacs-module/mod-test.c (emacs_module_init): Print
compile-time and runtime sizes of module structures to ease debugging

* test/data/emacs-module/mod-test.c (emacs_module_init): Fix
off-by-one error.

* test/data/emacs-module/mod-test.c (emacs_module_init): Add necessary
version checks.

Add a new command-line option '-module-assertions' that users can
enable developing or debugging a module.  If this option is present,
Emacs performs additional checks to verify that modules fulfill their
requirements.  These checks are expensive and crash Emacs if modules
are invalid, so disable them by default.

This is a command-line option instead of an ordinary variable because
changing it while Emacs is running would cause data structure
imbalances.

* src/emacs.c (main): New command line option '-module-assertions'.

* src/emacs-module.c (module_assert_main_thread)
(module_assert_runtime, module_assert_env, module_assert_value):
New functions to assert module requirements.
(syms_of_module): New uninterned variable 'module-runtimes'.
(init_module_assertions, in_main_thread, module_abort): New helper
functions.
(initialize_environment): Initialize value list.  If assertions are
enabled, use a heap-allocated environment object.
(finalize_environment): Add assertion that environment list is never
empty.
(finalize_runtime_unwind): Pop module runtime object stack.
(value_to_lisp): Assert that the value is valid.
(lisp_to_value): Record new value if assertions are enabled.
(mark_modules): Mark allocated object list.
(MODULE_FUNCTION_BEGIN_NO_CATCH)
(module_non_local_exit_check, module_non_local_exit_clear)
(module_non_local_exit_get, module_non_local_exit_signal)
(module_non_local_exit_throw): Assert thread and environment.
(module_get_environment): Assert thread and runtime.
(module_make_function, module_funcall, module_intern)
(module_funcall, module_make_integer, module_make_float)
(module_make_string, module_make_user_ptr, module_vec_get)
(funcall_module, Fmodule_load): Adapt callers.
(module_make_global_ref): If assertions are enabled, use the global
environment to store global values.
(module_free_global_ref): Remove value from global value list.

* test/Makefile.in (EMACSOPT): Enable module assertions when testing
modules.

* test/data/emacs-module/mod-test.c (Fmod_test_invalid_store)
(Fmod_test_invalid_load): New functions to test module assertions.
(emacs_module_init): Bind the new functions.

* test/src/emacs-module-tests.el (mod-test-emacs): New constant for
the Emacs binary file.
(mod-test-file): New constant for the test module file name.
(module--test-assertions): New unit test.

Instead of a lambda, create a new type containing all data required to
call the function, and support it in the evaluator.  Because this type
now also needs to store the function documentation, it is too big for
Lisp_Misc; use a pseudovector instead.  That also has the nice benefit
that we don't have to add special support to the garbage collector.

Since the new type is user-visible, give it a predicate.

Now we can easily support 'help-function-args' and 'func-arity'; add
unit tests for these.

* src/lisp.h (allocate_module_function, MODULE_FUNCTIONP)
(XMODULE_FUNCTION): New pseudovector type 'module function'.

* src/eval.c (FUNCTIONP): Also treat module functions as functions.
(funcall_lambda, Ffuncall, eval_sub): Add support for calling module
functions.
(Ffunc_arity): Add support for detecting the arity of module
functions.

* src/emacs-module.c (module_make_function): Adapt to new structure.
Return module function object directly instead of wrapping it in a
lambda; remove FIXME.
(funcall_module): New function to call module functions.  Replaces
`internal--module-call' and is called directly from eval.c.
(syms_of_module): Remove internal helper function, which is no longer
needed.
(module_function_arity): New helper function.

* src/data.c (Ftype_of): Adapt to new implementation.
(Fmodule_function_p, syms_of_data): New user-visible function.  Now
that module functions are first-class objects, they deserve a
predicate.  Define it even if not compiled with --enable-modules so
that Lisp code doesn't have to check for the function's existence.

* src/doc.c (Fdocumentation): Support module functions.

* src/print.c (print_object): Adapt to new implementation.

* src/alloc.c (mark_object): Specialized garbage collector support is
no longer needed.

* lisp/help.el (help-function-arglist): Support module functions.
While there, simplify the arity calculation by using `func-arity',
which does the right thing for all kinds of functions.

* test/data/emacs-module/mod-test.c: Amend docstring so we can test
the argument list.

* test/src/emacs-module-tests.el (mod-test-sum-docstring): Adapt to
new docstring.
(mod-test-non-local-exit-signal-test): Because `internal--module-call'
is gone, the backtrace has changed and no longer leaks the
implementation.
(module--func-arity): New test for `func-arity'.
(module--help-function-arglist): New test for `help-function-arglist'.

* test/Makefile.in (ELFILES): Exclude module test if modules aren't
configured.
(EMACS_TEST_DIRECTORY): Expand test directory so that it's set
correctly even if Emacs changes the current directory.
($(srcdir)/src/emacs-module-tests.log)
($(test_module)): Proper dependency tracking for test module.

* test/data/emacs-module/Makefile (ROOT): Adapt to new location.
Remove 'check' target and EMACS variable, which are no longer
necessary.
(SO): Change to include period.

* test/src/emacs-module-tests.el (mod-test): Use EMACS_TEST_DIRECTORY
environment variable to reliably find test data.

* configure.ac (HAVE_MODULES, MODULES_SUFFIX): Add necessary
substitutions.

Run admin/update-copyright.

Work around Bug#22884 by rewording comments and strings to avoid ‘(’
at the start of a line unless it starts a function.  This change
is a short-term hack; in the longer run we plan to fix cc-mode’s
performance for C files that have ‘(’ at the start of a line in a
comment or string.

Run admin/update-copyright.

* src/emacs-module.c (struct emacs_value_tag)
(struct emacs_value_frame, struct emacs_value_storage): Remove.
(value_frame_size): Remove constant.
(struct emacs_env_private): Use Lisp_Object for non_local_exit info.
(lisp_to_value): Remove first arg.
(module_nil): New constant.
Use it instead of NULL when returning an emacs_value.
(module_make_function): Adjust to new calling convention of
Qinternal_module_call.
(DEFUN): Receive args in an array rather than a list.
Use SAFE_ALLOCA rather than xnmalloc.  Skip the lisp_to_value loop when
we don't have WIDE_EMACS_INT.  Adjust to new type of non_local_exit info.
(module_non_local_exit_signal_1, module_non_local_exit_throw_1):
Adjust to new type of non_local_exit info.
(ltv_mark) [WIDE_EMACS_INT]: New constant.
(value_to_lisp, lisp_to_value): Rewrite.
(initialize_frame, initialize_storage, finalize_storage): Remove functions.
(allocate_emacs_value): Remove function.
(mark_modules): Gut it.
(initialize_environment): Don't initialize storage any more.
Keep the actual env object on Vmodule_environments.
(finalize_environment): Don't finalize storage any more.
(syms_of_module): Initialize ltv_mark and module_nil.

* src/emacs-module.h (emacs_value): Make it more clear that this type
is really opaque, including the fact that NULL may not be valid.

* modules/mod-test/mod-test.c (Fmod_test_signal, Fmod_test_throw):
Don't assume that NULL is a valid emacs_value.

* modules/mod-test/mod-test.c (Fmod_test_return_t)
(Fmod_test_sum, Fmod_test_signal, Fmod_test_throw)
(Fmod_test_non_local_exit_funcall, Fmod_test_globref_make)
(Fmod_test_string_a_to_b, Fmod_test_userptr_make)
(Fmod_test_userptr_get, Fmod_test_vector_fill)
(Fmod_test_vector_eq): Arg counts are ptrdiff_t, not int.
(finalizer): Remove; no longer used.

* modules/mod-test/mod-test.c: Include stdlib.h, to avoid warnings
about missing prototype of malloc.
* modules/mod-test/Makefile (CFLAGS): Add -std=gnu99, to avoid
compiler warnings.

Generally speaking, at the C level the Emacs source code prefers
signed types like ‘ptrdiff_t’ to unsigned types like ‘size_t’,
partly to avoid the usual signedness confusion when comparing values.
Change the module API to follow this convention.
Use ‘int’ for small values that can’t exceed INT_MAX.
* modules/mod-test/mod-test.c (Fmod_test_globref_make)
(Fmod_test_string_a_to_b, Fmod_test_vector_fill)
(Fmod_test_vector_eq):
* src/emacs-module.c (struct emacs_value_frame)
(module_make_global_ref, module_free_global_ref)
(module_copy_string_contents, module_make_string)
(module_vec_set, module_vec_get, module_vec_size):
* src/emacs-module.h (struct emacs_runtime, struct emacs_env_25):
* src/lread.c (suffix_p):
Prefer signed to unsigned integer types.

Remove ‘const’ qualifier from locals that were newly added.
We don’t normally bother declaring locals with ‘const’ even
though they are not modified, for the same reason we don’t
bother declaring them with ‘register’ even though their
addresses are not taken; the advantage in compile-time
checking isn’t worth the loss of readability.
* modules/mod-test/mod-test.c (Fmod_test_non_local_exit_funcall)
(Fmod_test_vector_fill, Fmod_test_vector_eq):
* src/emacs-module.c (MODULE_SETJMP_1)
(module_make_global_ref, module_free_global_ref)
(module_non_local_exit_get, module_make_function)
(module_extract_integer, module_extract_float)
(module_get_user_ptr, module_set_user_ptr)
(module_get_user_finalizer, module_set_user_finalizer)
(module_vec_get, Fmodule_call)
(module_non_local_exit_signal_1)
(module_non_local_exit_throw_1, lisp_to_value)
(finalize_storage, allocate_emacs_value, mark_modules)
(module_handle_signal, module_handle_throw)
(module_format_fun_env):
* src/eval.c (push_handler, push_handler_nosignal)
(init_handler):
* src/lread.c (suffix_p):
Omit unnecessary ‘const’.

* modules/mod-test/mod-test.c (sum, Fmod_test_sum):
* src/emacs-module.c (module_extract_integer)
(module_make_integer):
* src/emacs-module.h (struct emacs_env_25):
Prefer intmax_t to int64_t.  This doesn’t change the generated
code on any of the machines Emacs currently ports to, but it’s
at least in theory more future-proof as C99 doesn’t guarantee
that int64_t exists.

* src/emacs-module.c: Rename from src/module.c.
* src/emacs-module.h: Rename from src/module.h.
All uses changed.

This is mostly indenting and spacing changes.  Also, remove
some unnecessary static decls instead of bothering to reindent them.
* src/module.h (EMACS_EXTERN_C_BEGIN): Remove, and do this inline,
as most other Emacs files do for this sort of thing.

Put them in the usual format for GNU Emacs copyright notices.

* src/module.h: Rename from src/emacs_module.h.
All uses changed.

Add 'modhelp.py' script (python2) to automate module testing and
module generation.

To build and test all modules in the modules/ dir
  $ ./modhelp.py test

To generate a module from template code (good starting point)
  $ ./modhelp init mynewtestmodule

See the script -h option for more documentation.

* modules/modhelp.py: New module helper script.
* modules/mod-test/Makefile: New file. Makefile for the test module.
* modules/mod-test/mod-test.c: New file. Test module source file.
* modules/mod-test/test.el: New file. ert test suite for the test module.
* modules/.gitignore: New file. Local .gitignore file.
Co-authored-by: Philipp Stephani <phst@google.com>