WEBVTT captioned by sachac and robin
00:00:00.000 --> 00:00:03.839
Hello everyone. I'm Robin Templeton, and I'm going to talk
00:00:03.840 --> 00:00:07.919
about Emacs Beguiled and recent progress on the Guile-Emacs
00:00:07.920 --> 00:00:13.919
project.
00:00:13.920 --> 00:00:16.839
First of all, if you're not familiar with Guile, it's an
00:00:16.840 --> 00:00:20.239
implementation of the Scheme programming language, which
00:00:20.240 --> 00:00:24.799
is a dialect of Lisp, and in the same family as Emacs Lisp, and
00:00:24.800 --> 00:00:28.759
Guile is GNU's official extension language. The goal of
00:00:28.760 --> 00:00:32.679
the Guile-Emacs project is to use Guile as the basis for
00:00:32.680 --> 00:00:37.599
Emacs's Lisp support. It has two main components: a new
00:00:37.600 --> 00:00:41.919
Emacs Lisp compiler built on top of Guile, and a variant of
00:00:41.920 --> 00:00:45.199
Emacs in which the built-in Lisp implementation is
00:00:45.200 --> 00:00:50.239
entirely replaced with Guile Elisp. We expect the
00:00:50.240 --> 00:00:53.439
combination of these two projects to have several
00:00:53.440 --> 00:00:57.999
benefits. One is improved performance. Another is
00:00:58.000 --> 00:01:03.839
increased expressiveness for Elisp and making it easier to
00:01:03.840 --> 00:01:07.839
extend and experiment with the language. Finally, it
00:01:07.840 --> 00:01:13.079
will reduce Emacs's reliance on C for two reasons. Guile will
00:01:13.080 --> 00:01:16.959
be responsible for the language implementation, so Emacs
00:01:16.960 --> 00:01:21.559
will no longer have to include a Lisp interpreter. It
00:01:21.560 --> 00:01:25.759
will also become possible to implement much more of Emacs in
00:01:25.760 --> 00:01:30.279
Lisp than is currently feasible. Of course, this raises
00:01:30.280 --> 00:01:34.119
the question of why Guile is suitable for this project. And
00:01:34.120 --> 00:01:38.079
we chose Guile for a few reasons. Guile is primarily a Scheme
00:01:38.080 --> 00:01:41.119
implementation, but it also has built-in support for
00:01:41.120 --> 00:01:44.399
multiple languages using its compiler tower. To add
00:01:44.400 --> 00:01:50.079
support for a new language to Guile, you only have to write a
00:01:50.080 --> 00:01:53.399
compiler from the source language to Tree-IL, which is
00:01:53.400 --> 00:01:57.439
essentially a low-level, minimal representation of
00:01:57.440 --> 00:02:02.479
Scheme. All of Guile's compiler optimizations occur at the
00:02:02.480 --> 00:02:07.599
Tree-IL layer or lower, so you don't need to worry about the
00:02:07.600 --> 00:02:10.159
lower-level details of the compiler when initially
00:02:10.160 --> 00:02:14.639
implementing your language. Guile also has some Lisp
00:02:14.640 --> 00:02:18.879
features that are very rare in Scheme implementations. For
00:02:18.880 --> 00:02:22.599
example, it has a nil value that counts as both false and an
00:02:22.600 --> 00:02:27.759
empty list, just like in Elisp, and it also has a version of
00:02:27.760 --> 00:02:32.319
the Common Lisp Object System and its metaobject protocol,
00:02:32.320 --> 00:02:37.239
which is called GOOPS.
00:02:37.240 --> 00:02:42.199
The idea of Guile-Emacs has a pretty long history, going back
00:02:42.200 --> 00:02:45.319
at least three decades. There have been about half a dozen
00:02:45.320 --> 00:02:48.519
previous implementation attempts. But the current
00:02:48.520 --> 00:02:51.519
iteration began with a series of six Summer of Code
00:02:51.520 --> 00:02:56.279
internships: Daniel Kraft's in 2009, and then my
00:02:56.280 --> 00:03:02.519
internships from 2010 to 2014. My basic implementation
00:03:02.520 --> 00:03:06.319
strategy was pretty straightforward. I implemented a core
00:03:06.320 --> 00:03:09.679
subset of Elisp, which was enough to run some batch mode
00:03:09.680 --> 00:03:15.399
programs outside of Emacs. In Emacs, I modified the garbage
00:03:15.400 --> 00:03:19.679
collector and the data structures for Lisp objects to use
00:03:19.680 --> 00:03:24.679
their libguile equivalents. I replaced Emacs' Lisp
00:03:24.680 --> 00:03:32.199
evaluator with the one provided by Guile Elisp.
00:03:32.200 --> 00:03:35.919
After a little over a year of work, at the end of the 2014
00:03:35.920 --> 00:03:41.079
internship, I ended up with a fully functional prototype of
00:03:41.080 --> 00:03:46.039
Guile-Emacs. It used Guile Elisp alone as its Lisp
00:03:46.040 --> 00:03:53.319
implementation and was completely compatible with Emacs
00:03:53.320 --> 00:03:57.559
functionality and with external extensions. One caveat
00:03:57.560 --> 00:04:01.399
was that performance was pretty bad, because I was focused
00:04:01.400 --> 00:04:05.639
on correctness, as well as ease of integration with the
00:04:05.640 --> 00:04:10.559
Emacs C code. But it was nonetheless a major milestone for
00:04:10.560 --> 00:04:15.759
the project. Let's take just a moment to look at
00:04:15.760 --> 00:04:19.599
Guile-Elisp.
00:04:19.600 --> 00:04:23.879
For starters, we have access to Guile modules. If we call
00:04:23.880 --> 00:04:26.959
Guile's version function, we can see that we're running
00:04:26.960 --> 00:04:33.879
under Guile 3.0. We have access to some of the numeric tower via
00:04:33.880 --> 00:04:41.279
the arithmetic functions. We also have multiple values. We
00:04:41.280 --> 00:04:45.599
have to be careful to use Guile's values procedure here, not
00:04:45.600 --> 00:04:48.839
the CL library's, but you can see that this works properly
00:04:48.840 --> 00:04:52.879
rather than being an emulation. Finally, we have tail
00:04:52.880 --> 00:04:57.999
call elimination. Naturally, we're going to use factorial
00:04:58.000 --> 00:05:07.159
to demonstrate it. If n is zero, return the answer, else
00:05:07.160 --> 00:05:14.199
recurse with n less one and n times a.
00:05:14.200 --> 00:05:17.119
Of course, this definition works correctly, but it gets
00:05:17.120 --> 00:05:21.759
more interesting if we communicate the answer with an
00:05:21.760 --> 00:05:27.759
error,
00:05:27.760 --> 00:05:32.359
in order to look at a backtrace. You can see here that there are no
00:05:32.360 --> 00:05:37.839
calls to fact visible in between the request to evaluate and
00:05:37.840 --> 00:05:42.199
the error communicating the answer. That's because
00:05:42.200 --> 00:05:53.319
this tail call has been optimized into effectively a goto.
00:05:53.320 --> 00:05:55.759
This is essential for any kind of serious functional
00:05:55.760 --> 00:06:00.279
programming.
00:06:00.280 --> 00:06:05.359
That's a peek at Guile-Elisp. In 2015, I left university
00:06:05.360 --> 00:06:09.479
to go work on web technologies, and the project was dormant
00:06:09.480 --> 00:06:14.679
for a very long time. But that's been changing recently.
00:06:14.680 --> 00:06:17.039
During the last few months, I've been working with Larry
00:06:17.040 --> 00:06:23.399
Valkama to rebase Guile-Emacs onto the development branch
00:06:23.400 --> 00:06:28.319
of upstream Emacs, including the past decade's worth of
00:06:28.320 --> 00:06:33.399
upstream development. What we've ended up with is a series
00:06:33.400 --> 00:06:38.839
of rebases onto different versions of Emacs. The older ones
00:06:38.840 --> 00:06:44.239
tend to work pretty well. The newer ones have increasingly
00:06:44.240 --> 00:06:49.799
bad problems where they haven't been properly adjusted for
00:06:49.800 --> 00:06:55.599
changes in the Emacs implementation. But we do have by now a
00:06:55.600 --> 00:06:58.919
version of Emacs 30 which boots correctly and can be used for
00:06:58.920 --> 00:07:04.959
interactive debugging, as well as the ability to bisect the
00:07:04.960 --> 00:07:08.919
revisions of Emacs and find out where regressions were
00:07:08.920 --> 00:07:13.199
introduced. Our immediate goal is of course to complete
00:07:13.200 --> 00:07:19.719
the rebase. At the same time, we want to improve Guile Elisp's
00:07:19.720 --> 00:07:22.799
performance to at least be competitive with ordinary Emacs
00:07:22.800 --> 00:07:29.279
Lisp. Just to characterize the performance situation,
00:07:29.280 --> 00:07:34.479
Guile Elisp is usually about half as fast as ordinary Elisp,
00:07:34.480 --> 00:07:37.839
while Guile Scheme is quite often an order of magnitude
00:07:37.840 --> 00:07:43.319
faster than ordinary Elisp, and that's based on micro
00:07:43.320 --> 00:07:47.799
benchmarks like the Gabriel benchmarks. But there's
00:07:47.800 --> 00:07:52.319
clearly a lot of room to improve our compiler's output.
00:07:52.320 --> 00:07:57.759
If you want to mark your calendars, we're expecting to have a
00:07:57.760 --> 00:08:04.199
usable version of Guile-Emacs 30 out sometime next spring. We're
00:08:04.200 --> 00:08:06.799
also going to put some effort into either extracting old
00:08:06.800 --> 00:08:13.599
work or doing new work that could be contributed upstream.
00:08:13.600 --> 00:08:17.559
On the Guile side, we'll probably start out with optimizing
00:08:17.560 --> 00:08:22.839
the dynamic binding facilities, which are used very seldom
00:08:22.840 --> 00:08:27.199
in Scheme, but are used all the time in traditional Lisp
00:08:27.200 --> 00:08:31.399
dialects. On the Emacs side, we'll be working initially on
00:08:31.400 --> 00:08:35.919
abstracting away the details of the Lisp implementation
00:08:35.920 --> 00:08:39.999
where they're not relevant. And that will clean up the Emacs
00:08:40.000 --> 00:08:44.279
code base a bit. It'll make it easier to integrate Emacs and
00:08:44.280 --> 00:08:49.919
Guile Elisp. It will probably be helpful for anyone who
00:08:49.920 --> 00:08:51.559
is working on ordinary Elisp on their own.
00:08:51.560 --> 00:08:57.199
We're also going to be adding new features to Emacs Lisp.
00:08:57.200 --> 00:09:01.639
We've seen a few of them already. The numeric tower, tail
00:09:01.640 --> 00:09:05.919
call optimization, Common Lisp compatibility. We're also
00:09:05.920 --> 00:09:10.359
going to provide access to Fibers, which is a Guile library
00:09:10.360 --> 00:09:14.639
based on ideas from Concurrent ML that provides much more
00:09:14.640 --> 00:09:17.679
powerful facilities for concurrent and parallel
00:09:17.680 --> 00:09:20.679
programming than what Emacs currently offers.
00:09:20.680 --> 00:09:33.759
This plan meets Guile-Emacs' basic goals, and it's work
00:09:33.760 --> 00:09:36.879
that we could maybe get integrated upstream in a reasonable
00:09:36.880 --> 00:09:41.799
amount of time. But it's also worth considering what more we
00:09:41.800 --> 00:09:47.239
can do, and what effect Guile-Emacs might have on Emacs if it
00:09:47.240 --> 00:09:49.079
becomes simply Emacs.
00:09:49.080 --> 00:09:54.599
For context, the amount of C code in Emacs has increased by
00:09:54.600 --> 00:09:58.559
around 50% in the last decade, and now it constitutes around
00:09:58.560 --> 00:10:06.399
a quarter of the code base. C can be a bit of a barrier to
00:10:06.400 --> 00:10:13.279
customizing and extending Emacs. For example, there are
00:10:13.280 --> 00:10:20.439
about 1500 C subroutines. Around 500 are used in C code, as
00:10:20.440 --> 00:10:26.519
well as available to Lisp code, and being written in C means
00:10:26.520 --> 00:10:31.519
that they can't be practically redefined. The use of C can
00:10:31.520 --> 00:10:35.839
become a barrier to extending Emacs or customizing its
00:10:35.840 --> 00:10:40.479
behavior. We might consider writing as much of Emacs as
00:10:40.480 --> 00:10:46.039
possible in Lisp. One way to speed up this process would
00:10:46.040 --> 00:10:52.199
be to provide a Common Lisp implementation for Guile. Note
00:10:52.200 --> 00:10:56.199
that between Guile Elisp and Guile Scheme, we have all of
00:10:56.200 --> 00:10:58.839
the essential ingredients for a Common Lisp environment.
00:10:58.840 --> 00:11:03.279
We can also share code with other Common Lisp
00:11:03.280 --> 00:11:12.479
implementations such as SBCL and SICL. Overall, the
00:11:12.480 --> 00:11:15.959
duration of the project will be better measured in months
00:11:15.960 --> 00:11:19.479
rather than years, despite Common Lisp's reputation for
00:11:19.480 --> 00:11:23.959
being a large language. This could have multiple uses, of
00:11:23.960 --> 00:11:29.199
course. It could be a model for future improvements to
00:11:29.200 --> 00:11:38.399
Elisp, because Elisp and CL can interact directly without
00:11:38.400 --> 00:11:41.319
problems. And it would be very easy for Elisp to borrow
00:11:41.320 --> 00:11:45.479
language features from Common Lisp. But for the purpose of a
00:11:45.480 --> 00:11:49.559
C to Lisp transition, it would also provide us with instant
00:11:49.560 --> 00:11:52.599
access to a huge number of high-quality libraries for
00:11:52.600 --> 00:11:58.159
things that Guile is not necessarily equipped to deal with,
00:11:58.160 --> 00:12:03.879
such as access to low-level Windows APIs, as well as lots of
00:12:03.880 --> 00:12:08.799
other libraries, such as interfaces to GUI toolkits for a
00:12:08.800 --> 00:12:12.079
variety of operating systems.
00:12:12.080 --> 00:12:21.799
At a certain point, this has technical advantages. If
00:12:21.800 --> 00:12:26.119
most of Emacs is written in Lisp, then we could consider
00:12:26.120 --> 00:12:30.759
using Guile Hoot to compile Emacs to WebAssembly, making it
00:12:30.760 --> 00:12:35.159
available perhaps in web browsers or on systems with the
00:12:35.160 --> 00:12:40.679
WebAssembly System Interface. But it would also be a great
00:12:40.680 --> 00:12:44.759
victory for practical software freedom. That's the idea
00:12:44.760 --> 00:12:48.359
that Freedom One, the freedom to study and modify programs,
00:12:48.360 --> 00:12:52.039
should not just be legally and technically possible, but
00:12:52.040 --> 00:12:54.719
should be actively encouraged by our computing
00:12:54.720 --> 00:12:58.959
environments. Emacs is really one of the archetypal
00:12:58.960 --> 00:13:02.519
examples of this, but we can and should go further.
00:13:02.520 --> 00:13:10.919
When Emacs is implemented primarily in Lisp, the entirety
00:13:10.920 --> 00:13:14.599
of the system will be transparent to examination and open to
00:13:14.600 --> 00:13:20.359
modification. Every part of Emacs will be instantaneously
00:13:20.360 --> 00:13:23.319
inspectable, redefinable, and debuggable.
00:13:23.320 --> 00:13:30.559
This will be a fundamental change in what is possible to
00:13:30.560 --> 00:13:36.159
do with Emacs extensions. For example, one experiment I'd
00:13:36.160 --> 00:13:40.319
be interested in is using the Common Lisp Interface Manager
00:13:40.320 --> 00:13:46.479
as the basis for Emacs's user interface. Screwlisp is
00:13:46.480 --> 00:13:52.879
giving a talk about McCLIM later today, but for present
00:13:52.880 --> 00:13:55.919
purposes, just think of it as a super-powered version of
00:13:55.920 --> 00:14:01.279
Emacs's concept of interactive functions. It would be a
00:14:01.280 --> 00:14:04.799
pretty long-term project in Emacs as it currently exists,
00:14:04.800 --> 00:14:10.519
but it would be almost trivial if Emacs were customizable at
00:14:10.520 --> 00:14:11.599
the lowest layers via Lisp.
00:14:11.600 --> 00:14:19.599
We'll certainly be looking at the practicality of these
00:14:19.600 --> 00:14:23.839
kinds of changes as we continue developing Guile-Emacs.
00:14:23.840 --> 00:14:31.719
Finally, how can you get involved with and support Guile
00:14:31.720 --> 00:14:35.999
Emacs? One way to help is just by trying it out and letting us
00:14:36.000 --> 00:14:40.519
know what your experiences are like. There will be a
00:14:40.520 --> 00:14:44.079
snapshot available on the Codeberg project site of the
00:14:44.080 --> 00:14:48.759
version that I'm using to give this presentation. It will be
00:14:48.760 --> 00:14:52.719
available both as a Guix package and as a portable tarball.
00:14:52.720 --> 00:14:58.799
This will be more interesting as we get closer to a complete
00:14:58.800 --> 00:15:05.479
rebase. We're also always happy to talk to potential
00:15:05.480 --> 00:15:10.879
contributors or potential collaborators from other
00:15:10.880 --> 00:15:11.599
projects.
00:15:11.600 --> 00:15:18.159
We can always use bug reports, and we're interested in what
00:15:18.160 --> 00:15:21.719
kind of features people actually want to see in Guile-Emacs.
00:15:21.720 --> 00:15:27.359
Guile-Emacs is also being developed by a small worker
00:15:27.360 --> 00:15:32.159
cooperative, so donations are a pretty direct way to
00:15:32.160 --> 00:15:36.039
support the project. If you do nothing else, I recommend
00:15:36.040 --> 00:15:40.719
going to the website and subscribing to our mailing lists so
00:15:40.720 --> 00:15:45.879
that you can keep up with news on the project. If you're
00:15:45.880 --> 00:15:49.239
watching this at EmacsConf, there will be a Q&A session
00:15:49.240 --> 00:15:57.080
immediately following this, and thanks for watching!