Commit 8af26410 authored by João Távora's avatar João Távora
Browse files

Add lisp/jsonrpc.el

* doc/lispref/text.texi (Text): Add JSONRPC.
(JSONRPC): New node.

* etc/NEWS (New Modes and Packages in Emacs 27.1): Mention jsonrpc.el

* lisp/jsonrpc.el: New file.

* test/lisp/jsonrpc-tests.el: New file.
parent 852395ba
......@@ -62,6 +62,7 @@ the character after point.
* GnuTLS Cryptography:: Cryptographic algorithms imported from GnuTLS.
* Parsing HTML/XML:: Parsing HTML and XML.
* Parsing JSON:: Parsing and generating JSON values.
* JSONRPC:: JSON Remote Procedure Call protocol
* Atomic Changes:: Installing several buffer changes atomically.
* Change Hooks:: Supplying functions to be run when text is changed.
@end menu
......@@ -5132,6 +5133,192 @@ doesn't move point. The arguments @var{args} are interpreted as in
@end defun
@section JSONRPC communication
@cindex JSON remote procedure call protocol
The @code{jsonrpc} library implements the @acronym{JSONRPC}
specification, version 2.0, as it is described in
@uref{}. As the name suggests, JSONRPC is a
generic @code{Remote Procedure Call} protocol designed around
@acronym{JSON} objects, which you can convert to and from Lisp objects
(@pxref{Parsing JSON}).
@node JSONRPC Overview
@subsection Overview
Quoting from the @uref{, spec}, JSONRPC "is
transport agnostic in that the concepts can be used within the same
process, over sockets, over http, or in many various message passing
To model this agnosticism, the @code{jsonrpc} library uses objects of
a @code{jsonrpc-connection} class, which represent a connection the
remote JSON endpoint (for details on Emacs's object system,
@pxref{Top,EIEIO,,eieio,EIEIO}). In modern object-oriented parlance,
this class is ``abstract'', i.e. the actual class of a useful
connection object used is always a subclass of it. Nevertheless, we
can define two distinct API's around the @code{jsonrpc-connection}
@item A user interface for building JSONRPC applications
In this scenario, the JSONRPC application instantiates
@code{jsonrpc-connection} objects of one of its concrete subclasses
using @code{make-instance}. To initiate a contact to the remote
endpoint, the JSONRPC application passes this object to the functions
@code{jsonrpc-notify'}, @code{jsonrpc-request} and
@code{jsonrpc-async-request}. For handling remotely initiated
contacts, which generally come in asynchronously, the instantiation
should include @code{:request-dispatcher} and
@code{:notification-dispatcher} initargs, which are both functions of
3 arguments: the connection object; a symbol naming the JSONRPC method
invoked remotely; and a JSONRPC "params" object.
The function passed as @code{:request-dispatcher} is responsible for
handling the remote endpoint's requests, which expect a reply from the
local endpoint (in this case, the program you're building). Inside
that function, you may either return locally (normally) or non-locally
(error). A local return value must be a Lisp object serializable as
JSON (@pxref{Parsing JSON}). This determines a success response, and
the object is forwarded to the server as the JSONRPC "result" object.
A non-local return, achieved by calling the function
@code{jsonrpc-error}, causes an error response to be sent to the
server. The details of the accompanying JSONRPC "error" are filled
out with whatever was passed to @code{jsonrpc-error}. A non-local
return triggered by an unexpected error of any other type also causes
an error response to be sent (unless you have set
@code{debug-on-error}, in which case this should land you in the
debugger, @pxref{Error Debugging}).
@item A inheritance interface for building JSONRPC transport implementations
In this scenario, @code{jsonrpc-connection} is subclassed to implement
a different underlying transport strategy (for details on how to
subclass, @pxref{Inheritance,Inheritance,,eieio}). Users of the
application-building interface can then instantiate objects of this
concrete class (using the @code{make-instance} function) and connect
to JSONRPC endpoints using that strategy.
This API has mandatory and optional parts.
To allow its users to initiate JSONRPC contacts (notifications or
requests) or reply to endpoint requests, the method
@code{jsonrpc-connection-send} must be implemented for the subclass.
Likewise, for handling the three types of remote contacts (requests,
notifications and responses to local requests) the transport
implementation must arrange for the function
@code{jsonrpc-connection-receive} to be called after noticing a new
JSONRPC message on the wire (whatever that "wire" may be).
Finally, and optionally, the @code{jsonrpc-connection} subclass should
implement @code{jsonrpc-shutdown} and @code{jsonrpc-running-p} if
these concepts apply to the transport. If they do, then any system
resources (e.g. processes, timers, etc..) used listen for messages on
the wire should be released in @code{jsonrpc-shutdown}, i.e. they
should only be needed while @code{jsonrpc-running-p} is non-nil.
@end enumerate
@node Process-based JSONRPC connections
@subsection Process-based JSONRPC connections
For convenience, the @code{jsonrpc} library comes built-in with a
@code{jsonrpc-process-connection} transport implementation that can
talk to local subprocesses (using the standard input and standard
output); or TCP hosts (using sockets); or any other remote endpoint
that Emacs's process object can represent (@pxref{Processes}).
Using this transport, the JSONRPC messages are encoded on the wire as
plain text and prefaced by some basic HTTP-style enveloping headers,
such as ``Content-Length''.
For an example of an application using this transport scheme on top of
JSONRPC, see the
Language Server Protocol}.
Along with the mandatory @code{:request-dispatcher} and
@code{:notification-dispatcher} initargs, users of the
@code{jsonrpc-process-connection} class should pass the following
initargs as keyword-value pairs to @code{make-instance}:
@table @code
@item :process
Value must be a live process object or a function of no arguments
producing one such object. If passed a process object, that is
expected to contain an pre-established connection; otherwise, the
function is called immediately after the object is made.
@item :on-shutdown
Value must be a function of a single argument, the
@code{jsonrpc-process-connection} object. The function is called
after the underlying process object has been deleted (either
deliberately by @code{jsonrpc-shutdown} or unexpectedly, because of
some external cause).
@end table
@node JSONRPC JSON object format
@subsection JSON object format
JSON objects are exchanged as Lisp plists (@pxref{Parsing JSON}):
JSON-compatible plists are handed to the dispatcher functions and,
likewise, JSON-compatible plists should be given to
@code{jsonrpc-notify}, @code{jsonrpc-request} and
To facilitate handling plists, this library make liberal use of
@code{cl-lib} library and suggests (but doesn't force) its clients to
do the same. A macro @code{jsonrpc-lambda} can be used to create a
lambda for destructuring a JSON-object like in this example:
myproc :frobnicate `(:foo "trix")
:success-fn (jsonrpc-lambda (&key bar baz &allow-other-keys)
(message "Server replied back with %s and %s!"
bar baz))
:error-fn (jsonrpc-lambda (&key code message _data)
(message "Sadly, server reports %s: %s"
code message)))
@end example
@node JSONRPC deferred requests
@subsection Deferred requests
In many @acronym{RPC} situations, synchronization between the two
communicating endpoints is a matter of correctly designing the RPC
application: when synchronization is needed, requests (which are
blocking) should be used; when it isn't, notifications should suffice.
However, when Emacs acts as one of these endpoints, asynchronous
events (e.g. timer- or process-related) may be triggered while there
is still uncertainty about the state of the remote endpoint.
Furthermore, acting on these events may only sometimes demand
synchronization, depending on the event's specific nature.
The @code{:deferred} keyword argument to @code{jsonrpc-request} and
@code{jsonrpc-async-request} is designed to let the caller indicate
that the specific request needs synchronization and its actual
issuance may be delayed to the future, until some condition is
satisfied. Specifying @code{:deferred} for a request doesn't mean it
@emph{will} be delayed, only that it @emph{can} be. If the request
isn't sent immediately, @code{jsonrpc} will make renewed efforts to
send it at certain key times during communication, such as when
receiving or sending other messages to the endpoint.
Before any attempt to send the request, the application-specific
conditions are checked. Since the @code{jsonrpc} library can't known
what these conditions are, the programmer may use the
@code{jsonrpc-connection-ready-p} generic function (@pxref{Generic
Functions}) to specify them. The default method for this function
returns @code{t}, but you can add overriding methods that return
@code{nil} in some situations, based on the arguments passed to it,
which are the @code{jsonrpc-connection} object (@pxref{JSONRPC
Overview}) and whichever value you passed as the @code{:deferred}
keyword argument.
@node Atomic Changes
@section Atomic Change Groups
......@@ -579,6 +579,15 @@ This feature uses Tramp and works only on systems which support GVFS,
i.e. GNU/Linux, roughly spoken. See the chapter "(tramp) Archive file
names" in the Tramp manual for full documentation of these facilities.
** New library for writing JSONRPC applications (
The 'jsonrpc' library enables writing Emacs Lisp applications that
rely on this protocol. Since the protocol is designed to be
transport-agnostic, the library provides an API to implement new
transport strategies as well as a separate API to use them. A
transport implementation for process-based communication, such as is
used by the Language Server Protocol (LSP), is readily available.
* Incompatible Lisp Changes in Emacs 27.1
This diff is collapsed.
;;; jsonrpc-tests.el --- tests for jsonrpc.el -*- lexical-binding: t; -*-
;; Copyright (C) 2018 Free Software Foundation, Inc.
;; Author: João Távora <>
;; Maintainer: João Távora <>
;; Keywords: tests
;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; GNU General Public License for more details.
;; You should have received a copy of the GNU General Public License
;; along with this program. If not, see <>.
;;; Commentary:
;; About "deferred" tests, `jsonrpc--test-client' has a flag that we
;; test this flag in the this `jsonrpc-connection-ready-p' API method.
;; It holds any `jsonrpc-request's and `jsonrpc-async-request's
;; explicitly passed `:deferred'. After clearing the flag, the held
;; requests are actually sent to the server in the next opportunity
;; (when receiving or sending something to the server).
;;; Code:
(require 'ert)
(require 'jsonrpc)
(require 'eieio)
(defclass jsonrpc--test-endpoint (jsonrpc-process-connection)
((scp :accessor jsonrpc--shutdown-complete-p)))
(defclass jsonrpc--test-client (jsonrpc--test-endpoint)
((hold-deferred :initform t :accessor jsonrpc--hold-deferred)))
(cl-defmacro jsonrpc--with-emacsrpc-fixture ((endpoint-sym) &body body)
(declare (indent 1) (debug t))
(let ((server (gensym "server-")) (listen-server (gensym "listen-server-")))
`(let* (,server
:name "Emacs RPC server" :server t :host "localhost"
:service 0
:log (lambda (_server client _message)
(setq ,server
:name (process-name client)
:process client
(lambda (_endpoint method params)
(unless (memq method '(+ - * / vconcat append
sit-for ignore))
(signal 'jsonrpc-error
. "Sorry, this isn't allowed")
(jsonrpc-error-code . -32601))))
(apply method (append params nil)))
(lambda (conn)
(setf (jsonrpc--shutdown-complete-p conn) t)))))))
(,endpoint-sym (make-instance
"Emacs RPC client"
(open-network-stream "JSONRPC test tcp endpoint"
nil "localhost"
(process-contact ,listen-server
(lambda (conn)
(setf (jsonrpc--shutdown-complete-p conn) t)))))
(cl-assert ,endpoint-sym)
(kill-buffer (jsonrpc--events-buffer ,endpoint-sym))
(when ,server
(kill-buffer (jsonrpc--events-buffer ,server))))
(jsonrpc-shutdown ,endpoint-sym)
(jsonrpc-shutdown ,server)
(cl-loop do (delete-process ,listen-server)
while (progn (accept-process-output nil 0.1)
(process-live-p ,listen-server))
do (jsonrpc--message
"test listen-server is still running, waiting"))))))))
(ert-deftest returns-3 ()
"A basic test for adding two numbers in our test RPC."
(jsonrpc--with-emacsrpc-fixture (conn)
(should (= 3 (jsonrpc-request conn '+ [1 2])))))
(ert-deftest errors-with--32601 ()
"Errors with -32601"
(jsonrpc--with-emacsrpc-fixture (conn)
(condition-case err
(jsonrpc-request conn 'delete-directory "~/tmp")
(ert-fail "A `jsonrpc-error' should have been signalled!"))
(should (= -32601 (cdr (assoc 'jsonrpc-error-code (cdr err)))))))))
(ert-deftest signals-an--32603-JSONRPC-error ()
"Signals an -32603 JSONRPC error."
(jsonrpc--with-emacsrpc-fixture (conn)
(condition-case err
(jsonrpc-request conn '+ ["a" 2])
(ert-fail "A `jsonrpc-error' should have been signalled!"))
(should (= -32603 (cdr (assoc 'jsonrpc-error-code (cdr err)))))))))
(ert-deftest times-out ()
"Request for 3-sec sit-for with 1-sec timeout times out."
(jsonrpc--with-emacsrpc-fixture (conn)
(jsonrpc-request conn 'sit-for [3] :timeout 1))))
(ert-deftest doesnt-time-out ()
:tags '(:expensive-test)
"Request for 1-sec sit-for with 2-sec timeout succeeds."
(jsonrpc--with-emacsrpc-fixture (conn)
(jsonrpc-request conn 'sit-for [1] :timeout 2)))
(ert-deftest stretching-it-but-works ()
"Vector of numbers or vector of vector of numbers are serialized."
(jsonrpc--with-emacsrpc-fixture (conn)
;; (vconcat [1 2 3] [3 4 5]) => [1 2 3 3 4 5] which can be
;; serialized.
(should (equal
[1 2 3 3 4 5]
(jsonrpc-request conn 'vconcat [[1 2 3] [3 4 5]])))))
(ert-deftest json-el-cant-serialize-this ()
"Can't serialize a response that is half-vector/half-list."
(jsonrpc--with-emacsrpc-fixture (conn)
;; (append [1 2 3] [3 4 5]) => (1 2 3 . [3 4 5]), which can't be
;; serialized
(jsonrpc-request conn 'append [[1 2 3] [3 4 5]]))))
(cl-defmethod jsonrpc-connection-ready-p
((conn jsonrpc--test-client) what)
(and (cl-call-next-method)
(or (not (string-match "deferred" what))
(not (jsonrpc--hold-deferred conn)))))
(ert-deftest deferred-action-toolate ()
:tags '(:expensive-test)
"Deferred request fails because noone clears the flag."
(jsonrpc--with-emacsrpc-fixture (conn)
(jsonrpc-request conn '+ [1 2]
:deferred "deferred-testing" :timeout 0.5)
:type 'jsonrpc-error)
(= 3 (jsonrpc-request conn '+ [1 2]
:timeout 0.5)))))
(ert-deftest deferred-action-intime ()
:tags '(:expensive-test)
"Deferred request barely makes it after event clears a flag."
;; Send an async request, which returns immediately. However the
;; success fun which sets the flag only runs after some time.
(jsonrpc--with-emacsrpc-fixture (conn)
(jsonrpc-async-request conn
'sit-for [0.5]
(lambda (_result)
(setf (jsonrpc--hold-deferred conn) nil)))
;; Now wait for an answer to this request, which should be sent as
;; soon as the previous one is answered.
(= 3 (jsonrpc-request conn '+ [1 2]
:deferred "deferred"
:timeout 1)))))
(ert-deftest deferred-action-complex-tests ()
:tags '(:expensive-test)
"Test a more complex situation with deferred requests."
(jsonrpc--with-emacsrpc-fixture (conn)
(let (n-deferred-1
;; This returns immediately
'sit-for [0.1]
(lambda (_result)
;; this only gets runs after the "first deferred" is stashed.
(setq n-deferred-1
(hash-table-count (jsonrpc--deferred-actions conn)))))
;; This stashes the request and waits. It will error because
;; no-one clears the "hold deferred" flag.
(jsonrpc-request conn 'ignore ["first deferred"]
:deferred "first deferred"
:timeout 0.5)
:type 'jsonrpc-error)
;; The error means the deferred actions stash is now empty
(should (zerop (hash-table-count (jsonrpc--deferred-actions conn))))
;; Again, this returns immediately.
'sit-for [0.1]
(lambda (_result)
;; This gets run while "third deferred" below is waiting for
;; a reply. Notice that we clear the flag in time here.
(setq n-deferred-2 (hash-table-count (jsonrpc--deferred-actions conn)))
(setf (jsonrpc--hold-deferred conn) nil)))
;; This again stashes a request and returns immediately.
(jsonrpc-async-request conn 'ignore ["second deferred"]
:deferred "second deferred"
:timeout 1
(lambda (_result)
(setq second-deferred-went-through-p t)))
;; And this also stashes a request, but waits. Eventually the
;; flag is cleared in time and both requests go through.
(jsonrpc-request conn 'ignore ["third deferred"]
:deferred "third deferred"
:timeout 1)
(should second-deferred-went-through-p)
(should (eq 1 n-deferred-1))
(should (eq 2 n-deferred-2))
(should (eq 0 (hash-table-count (jsonrpc--deferred-actions conn)))))))
(provide 'jsonrpc-tests)
;;; jsonrpc-tests.el ends here
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment