Hello everyone. I'm Robin Templeton, and I'm going to talk
about Emacs Beguiled and recent progress on the Guile-Emacs
project.
First of all, if you're not familiar with Guile, it's an
implementation of the Scheme programming language, which
is a dialect of Lisp, and in the same family as Emacs Lisp, and
Guile is GNU's official extension language. The goal of
the Guile-Emacs project is to use Guile as the basis for
Emacs's Lisp support. It has two main components: a new
Emacs Lisp compiler built on top of Guile, and a variant of
Emacs in which the built-in Lisp implementation is
entirely replaced with Guile Elisp. We expect the
combination of these two projects to have several
benefits. One is improved performance. Another is
increased expressiveness for Elisp and making it easier to
extend and experiment with the language. Finally, it
will reduce Emacs's reliance on C for two reasons. Guile will
be responsible for the language implementation, so Emacs
will no longer have to include a Lisp interpreter. It
will also become possible to implement much more of Emacs in
Lisp than is currently feasible. Of course, this raises
the question of why Guile is suitable for this project. And
we chose Guile for a few reasons. Guile is primarily a Scheme
implementation, but it also has built-in support for
multiple languages using its compiler tower. To add
support for a new language to Guile, you only have to write a
compiler from the source language to Tree-IL, which is
essentially a low-level, minimal representation of
Scheme. All of Guile's compiler optimizations occur at the
Tree-IL layer or lower, so you don't need to worry about the
lower-level details of the compiler when initially
implementing your language. Guile also has some Lisp
features that are very rare in Scheme implementations. For
example, it has a nil value that counts as both false and an
empty list, just like in Elisp, and it also has a version of
the Common Lisp Object System and its metaobject protocol,
which is called GOOPS.
The idea of Guile-Emacs has a pretty long history, going back
at least three decades. There have been about half a dozen
previous implementation attempts. But the current
iteration began with a series of six Summer of Code
internships: Daniel Kraft's in 2009, and then my
internships from 2010 to 2014. My basic implementation
strategy was pretty straightforward. I implemented a core
subset of Elisp, which was enough to run some batch mode
programs outside of Emacs. In Emacs, I modified the garbage
collector and the data structures for Lisp objects to use
their libguile equivalents. I replaced Emacs' Lisp
evaluator with the one provided by Guile Elisp.
After a little over a year of work, at the end of the 2014
internship, I ended up with a fully functional prototype of
Guile-Emacs. It used Guile Elisp alone as its Lisp
implementation and was completely compatible with Emacs
functionality and with external extensions. One caveat
was that performance was pretty bad, because I was focused
on correctness, as well as ease of integration with the
Emacs C code. But it was nonetheless a major milestone for
the project. Let's take just a moment to look at
Guile-Elisp.
For starters, we have access to Guile modules. If we call
Guile's version function, we can see that we're running
under Guile 3.0. We have access to some of the numeric tower via
the arithmetic functions. We also have multiple values. We
have to be careful to use Guile's values procedure here, not
the CL library's, but you can see that this works properly
rather than being an emulation. Finally, we have tail
call elimination. Naturally, we're going to use factorial
to demonstrate it. If n is zero, return the answer, else
recurse with n less one and n times a.
Of course, this definition works correctly, but it gets
more interesting if we communicate the answer with an
error,
in order to look at a backtrace. You can see here that there are no
calls to fact visible in between the request to evaluate and
the error communicating the answer. That's because
this tail call has been optimized into effectively a goto.
This is essential for any kind of serious functional
programming.
That's a peek at Guile-Elisp. In 2015, I left university
to go work on web technologies, and the project was dormant
for a very long time. But that's been changing recently.
During the last few months, I've been working with Larry
Valkama to rebase Guile-Emacs onto the development branch
of upstream Emacs, including the past decade's worth of
upstream development. What we've ended up with is a series
of rebases onto different versions of Emacs. The older ones
tend to work pretty well. The newer ones have increasingly
bad problems where they haven't been properly adjusted for
changes in the Emacs implementation. But we do have by now a
version of Emacs 30 which boots correctly and can be used for
interactive debugging, as well as the ability to bisect the
revisions of Emacs and find out where regressions were
introduced. Our immediate goal is of course to complete
the rebase. At the same time, we want to improve Guile Elisp's
performance to at least be competitive with ordinary Emacs
Lisp. Just to characterize the performance situation,
Guile Elisp is usually about half as fast as ordinary Elisp,
while Guile Scheme is quite often an order of magnitude
faster than ordinary Elisp, and that's based on micro
benchmarks like the Gabriel benchmarks. But there's
clearly a lot of room to improve our compiler's output.
If you want to mark your calendars, we're expecting to have a
usable version of Guile-Emacs 30 out sometime next spring. We're
also going to put some effort into either extracting old
work or doing new work that could be contributed upstream.
On the Guile side, we'll probably start out with optimizing
the dynamic binding facilities, which are used very seldom
in Scheme, but are used all the time in traditional Lisp
dialects. On the Emacs side, we'll be working initially on
abstracting away the details of the Lisp implementation
where they're not relevant. And that will clean up the Emacs
code base a bit. It'll make it easier to integrate Emacs and
Guile Elisp. It will probably be helpful for anyone who
is working on ordinary Elisp on their own.
We're also going to be adding new features to Emacs Lisp.
We've seen a few of them already. The numeric tower, tail
call optimization, Common Lisp compatibility. We're also
going to provide access to Fibers, which is a Guile library
based on ideas from Concurrent ML that provides much more
powerful facilities for concurrent and parallel
programming than what Emacs currently offers.
This plan meets Guile-Emacs' basic goals, and it's work
that we could maybe get integrated upstream in a reasonable
amount of time. But it's also worth considering what more we
can do, and what effect Guile-Emacs might have on Emacs if it
becomes simply Emacs.
For context, the amount of C code in Emacs has increased by
around 50% in the last decade, and now it constitutes around
a quarter of the code base. C can be a bit of a barrier to
customizing and extending Emacs. For example, there are
about 1500 C subroutines. Around 500 are used in C code, as
well as available to Lisp code, and being written in C means
that they can't be practically redefined. The use of C can
become a barrier to extending Emacs or customizing its
behavior. We might consider writing as much of Emacs as
possible in Lisp. One way to speed up this process would
be to provide a Common Lisp implementation for Guile. Note
that between Guile Elisp and Guile Scheme, we have all of
the essential ingredients for a Common Lisp environment.
We can also share code with other Common Lisp
implementations such as SBCL and SICL. Overall, the
duration of the project will be better measured in months
rather than years, despite Common Lisp's reputation for
being a large language. This could have multiple uses, of
course. It could be a model for future improvements to
Elisp, because Elisp and CL can interact directly without
problems. And it would be very easy for Elisp to borrow
language features from Common Lisp. But for the purpose of a
C to Lisp transition, it would also provide us with instant
access to a huge number of high-quality libraries for
things that Guile is not necessarily equipped to deal with,
such as access to low-level Windows APIs, as well as lots of
other libraries, such as interfaces to GUI toolkits for a
variety of operating systems.
At a certain point, this has technical advantages. If
most of Emacs is written in Lisp, then we could consider
using Guile Hoot to compile Emacs to WebAssembly, making it
available perhaps in web browsers or on systems with the
WebAssembly System Interface. But it would also be a great
victory for practical software freedom. That's the idea
that Freedom One, the freedom to study and modify programs,
should not just be legally and technically possible, but
should be actively encouraged by our computing
environments. Emacs is really one of the archetypal
examples of this, but we can and should go further.
When Emacs is implemented primarily in Lisp, the entirety
of the system will be transparent to examination and open to
modification. Every part of Emacs will be instantaneously
inspectable, redefinable, and debuggable.
This will be a fundamental change in what is possible to
do with Emacs extensions. For example, one experiment I'd
be interested in is using the Common Lisp Interface Manager
as the basis for Emacs's user interface. Screwlisp is
giving a talk about McCLIM later today, but for present
purposes, just think of it as a super-powered version of
Emacs's concept of interactive functions. It would be a
pretty long-term project in Emacs as it currently exists,
but it would be almost trivial if Emacs were customizable at
the lowest layers via Lisp.
We'll certainly be looking at the practicality of these
kinds of changes as we continue developing Guile-Emacs.
Finally, how can you get involved with and support Guile
Emacs? One way to help is just by trying it out and letting us
know what your experiences are like. There will be a
snapshot available on the Codeberg project site of the
version that I'm using to give this presentation. It will be
available both as a Guix package and as a portable tarball.
This will be more interesting as we get closer to a complete
rebase. We're also always happy to talk to potential
contributors or potential collaborators from other
projects.
We can always use bug reports, and we're interested in what
kind of features people actually want to see in Guile-Emacs.
Guile-Emacs is also being developed by a small worker
cooperative, so donations are a pretty direct way to
support the project. If you do nothing else, I recommend
going to the website and subscribing to our mailing lists so
that you can keep up with news on the project. If you're
watching this at EmacsConf, there will be a Q&A session
immediately following this, and thanks for watching!