path: root/2024/captions/emacsconf-2024-guile--beguiling-emacs-guileemacs-relaunched--robin-templeton--main.vtt

WEBVTT captioned by anush, checked by robin

00:00.000 --> 00:03.066
Hello everyone. I'm Robin Templeton,

00:03.083 --> 00:05.750
and I'm going to talk about Emacs Beguiled

00:05.766 --> 00:13.866
and recent progress on the Guile-Emacs project.

00:13.883 --> 00:16.433
First of all, if you're not familiar with Guile,

00:16.450 --> 00:19.716
it's an implementation of the Scheme programming language,

00:19.733 --> 00:22.150
which is a dialect of Lisp,

00:22.166 --> 00:24.550
and in the same family as Emacs Lisp,

00:24.566 --> 00:28.150
and Guile is GNU's official extension language.

00:28.166 --> 00:30.400
The goal of the Guile-Emacs project

00:30.400 --> 00:34.950
is to use Guile as the basis for Emacs's Lisp support.

00:34.966 --> 00:37.116
It has two main components:

00:37.133 --> 00:41.033
a new Emacs Lisp compiler built on top of Guile,

00:41.050 --> 00:42.550
and a variant of Emacs

00:42.566 --> 00:45.316
in which the built-in Lisp implementation

00:45.333 --> 00:49.716
is entirely replaced with Guile Elisp.

00:49.733 --> 00:52.800
We expect the combination of these two projects

00:52.800 --> 00:57.350
to have several benefits. One is improved performance.

00:57.366 --> 01:00.200
Another is increased expressiveness for Elisp

01:00.200 --> 01:04.350
and making it easier to extend

01:04.366 --> 01:07.200
and experiment with the language.

01:07.200 --> 01:08.550
Finally, it will reduce

01:08.566 --> 01:12.316
Emacs's reliance on C for two reasons.

01:12.333 --> 01:16.316
Guile will be responsible for the language implementation,

01:16.333 --> 01:21.350
so Emacs will no longer have to include a Lisp interpreter.

01:21.366 --> 01:23.200
It will also become possible

01:23.200 --> 01:25.666
to implement much more of Emacs in Lisp

01:25.683 --> 01:29.233
than is currently feasible.

01:29.250 --> 01:31.116
Of course, this raises the question of

01:31.133 --> 01:34.033
why Guile is suitable for this project.

01:34.050 --> 01:36.666
And we chose Guile for a few reasons.

01:36.683 --> 01:39.400
Guile is primarily a Scheme implementation,

01:39.400 --> 01:42.150
but it also has built-in support for multiple languages

01:42.166 --> 01:43.466
using its compiler tower.

01:43.483 --> 01:46.866
To add support for a new language to Guile,

01:46.883 --> 01:50.066
you only have to write a compiler

01:50.083 --> 01:52.550
from the source language to Tree-IL,

01:52.566 --> 01:55.800
which is essentially a low-level,

01:55.800 --> 01:58.866
minimal representation of Scheme.

01:58.883 --> 02:01.800
All of Guile's compiler optimizations

02:01.800 --> 02:04.433
occur at the Tree-IL layer or lower,

02:04.450 --> 02:06.033
so you don't need to worry

02:06.050 --> 02:09.633
about the lower-level details of the compiler

02:09.650 --> 02:12.350
when initially implementing your language.

02:12.366 --> 02:14.633
Guile also has some Lisp features

02:14.650 --> 02:18.316
that are very rare in Scheme implementations.

02:18.333 --> 02:20.033
For example, it has a nil value

02:20.050 --> 02:23.916
that counts as both false and an empty list,

02:23.933 --> 02:25.633
just like in Elisp,

02:25.650 --> 02:30.466
and it also has a version of the Common Lisp Object System

02:30.483 --> 02:37.200
and its metaobject protocol, which is called GOOPS.

02:37.200 --> 02:40.150
The idea of Guile-Emacs has a pretty long history,

02:40.166 --> 02:43.866
going back at least three decades.

02:43.883 --> 02:44.550
There have been about

02:44.566 --> 02:48.000
half a dozen previous implementation attempts.

02:48.000 --> 02:49.950
But the current iteration began with

02:49.966 --> 02:52.866
a series of six Summer of Code internships:

02:52.883 --> 02:56.033
Daniel Kraft's in 2009,

02:56.050 --> 03:01.200
and then my internships from 2010 to 2014.

03:01.200 --> 03:03.000
My basic implementation strategy

03:03.000 --> 03:05.316
was pretty straightforward.

03:05.333 --> 03:07.466
I implemented a core subset of Elisp,

03:07.483 --> 03:10.400
which was enough to run some batch mode programs

03:10.400 --> 03:12.833
outside of Emacs.

03:12.850 --> 03:15.266
In Emacs, I modified the garbage collector

03:15.283 --> 03:18.600
and the data structures for Lisp objects

03:18.600 --> 03:23.033
to use their libguile equivalents.

03:23.050 --> 03:26.950
I replaced Emacs' Lisp evaluator

03:26.966 --> 03:32.200
with the one provided by Guile Elisp.

03:32.200 --> 03:34.033
After a little over a year of work,

03:34.050 --> 03:37.950
at the end of the 2014 internship,

03:37.966 --> 03:44.316
I ended up with a fully functional prototype of Guile-Emacs.

03:44.333 --> 03:48.916
It used Guile Elisp alone as its Lisp implementation

03:48.933 --> 03:53.916
and was completely compatible with Emacs functionality

03:53.933 --> 03:56.716
and with external extensions.

03:56.733 --> 03:59.433
One caveat was that performance was pretty bad,

03:59.450 --> 04:03.033
because I was focused on correctness,

04:03.050 --> 04:07.600
as well as ease of integration with the Emacs C code.

04:07.600 --> 04:11.550
But it was nonetheless a major milestone for the project.

04:11.566 --> 04:19.600
Let's take just a moment to look at Guile-Elisp.

04:19.600 --> 04:23.233
For starters, we have access to Guile modules.

04:23.250 --> 04:25.116
If we call Guile's <i>version</i> function,

04:25.133 --> 04:30.516
we can see that we're running under Guile 3.0.

04:30.533 --> 04:33.233
We have access to some of the numeric tower

04:33.250 --> 04:39.516
via the arithmetic functions. We also have multiple values.

04:39.533 --> 04:43.950
We have to be careful to use Guile's <i>values</i> procedure here,

04:43.966 --> 04:46.666
not the CL library's,

04:46.683 --> 04:48.833
but you can see that this works properly

04:48.850 --> 04:51.550
rather than being an emulation.

04:51.566 --> 04:54.033
Finally, we have tail call elimination.

04:54.050 --> 05:02.866
Naturally, we're going to use factorial to demonstrate it.

05:02.883 --> 05:05.633
If <i>n</i> is zero, return the answer,

05:05.650 --> 05:14.266
else recurse with <i>n</i> less one and <i>n</i> times <i>a</i>.

05:14.283 --> 05:16.150
Of course, this definition works correctly,

05:16.166 --> 05:18.950
but it gets more interesting

05:18.966 --> 00:05:25.000
if we communicate the answer with an error,

00:05:25.100 --> 05:29.633
in order to look at a backtrace.

05:29.650 --> 05:32.350
You can see here that there are

05:32.366 --> 05:35.516
no calls to <i>fact</i> visible in between

05:35.533 --> 05:37.833
the request to evaluate

05:37.850 --> 05:42.200
and the error communicating the answer.

05:42.200 --> 05:44.200
That's because this tail call

05:44.200 --> 05:48.350
has been optimized into effectively a goto.

05:48.366 --> 05:54.916
This is essential for any kind

05:54.933 --> 00:05:59.916
of serious functional programming.

00:06:00.116 --> 06:03.033
That's a peek at Guile-Elisp.

06:03.050 --> 06:08.066
In 2015, I left university to go work on web technologies,

06:08.083 --> 06:11.316
and the project was dormant for a very long time.

06:11.333 --> 06:13.433
But that's been changing recently.

06:13.450 --> 06:16.066
During the last few months,

06:16.083 --> 06:17.633
I've been working with Larry Valkama

06:17.650 --> 06:20.716
to rebase Guile-Emacs

06:20.733 --> 06:24.833
onto the development branch of upstream Emacs,

06:24.850 --> 06:29.666
including the past decade's worth of upstream development.

06:29.683 --> 00:06:33.967
What we've ended up with is a series of

00:06:34.267 --> 00:06:37.550
rebases onto different versions of Emacs.

06:37.566 --> 06:39.516
The older ones tend to work pretty well.

06:39.533 --> 06:46.866
The newer ones have increasingly bad problems

06:46.883 --> 06:49.800
where they haven't been properly adjusted

06:49.800 --> 06:52.200
for changes in the Emacs implementation.

06:52.200 --> 06:56.833
But we do have by now a version of Emacs 30

06:56.850 --> 06:57.800
which boots correctly

06:57.800 --> 06:59.833
and can be used for interactive debugging,

06:59.850 --> 07:06.150
as well as the ability to bisect the revisions of Emacs

07:06.166 --> 07:10.516
and find out where regressions were introduced.

07:10.533 --> 07:14.033
Our immediate goal is of course to complete the rebase.

07:14.050 --> 07:16.233
At the same time,

07:16.250 --> 07:20.633
we want to improve Guile Elisp's performance

07:20.650 --> 07:24.350
to at least be competitive with ordinary Emacs Lisp.

07:24.366 --> 07:29.266
Just to characterize the performance situation,

07:29.283 --> 07:32.750
Guile Elisp is usually about half

07:32.766 --> 07:34.466
as fast as ordinary Elisp,

07:34.483 --> 07:37.833
while Guile Scheme is quite often

07:37.850 --> 00:07:41.250
an order of magnitude faster than ordinary Elisp,

00:07:41.350 --> 07:43.916
and that's based on micro benchmarks

07:43.933 --> 00:07:46.133
like the Gabriel benchmarks.

00:07:46.233 --> 00:07:50.900
But there's clearly a lot of room

00:07:50.900 --> 00:07:53.150
to improve our compiler's output.

00:07:53.350 --> 07:56.633
If you want to mark your calendars,

07:56.650 --> 08:00.150
we're expecting to have a usable version of Guile-Emacs 30

08:00.166 --> 00:08:03.016
out sometime next spring.

00:08:03.116 --> 08:05.433
We're also going to put some effort

08:05.450 --> 00:08:09.000
into either extracting old work

00:08:09.100 --> 08:12.600
or doing new work that could be contributed upstream.

08:12.600 --> 00:08:16.650
On the Guile side, we'll probably start out with

00:08:16.750 --> 00:08:21.033
optimizing the dynamic binding facilities,

00:08:21.233 --> 08:23.433
which are used very seldom in Scheme,

08:23.450 --> 08:27.833
but are used all the time in traditional Lisp dialects.

08:27.850 --> 08:31.400
On the Emacs side, we'll be working initially

08:31.400 --> 08:35.316
on abstracting away the details of the Lisp implementation

08:35.333 --> 00:08:37.433
where they're not relevant.

00:08:37.533 --> 08:40.716
And that will clean up the Emacs code base a bit.

08:40.733 --> 08:45.000
It'll make it easier to integrate Emacs and Guile Elisp.

08:45.000 --> 08:47.916
It will probably be helpful for anyone

08:47.933 --> 08:51.550
who is working on ordinary Elisp on their own.

08:51.566 --> 08:57.200
We're also going to be adding new features to Emacs Lisp.

08:57.200 --> 08:59.316
We've seen a few of them already.

08:59.333 --> 09:02.633
The numeric tower, tail call optimization,

09:02.650 --> 09:04.550
Common Lisp compatibility.

09:04.566 --> 09:07.950
We're also going to provide access to Fibers,

09:07.966 --> 09:12.466
which is a Guile library based on ideas from Concurrent ML

09:12.483 --> 09:15.716
that provides much more powerful facilities

09:15.733 --> 09:18.266
for concurrent and parallel programming

09:18.283 --> 00:09:24.566
than what Emacs currently offers.

00:09:24.666 --> 09:32.233
This plan meets Guile-Emacs' basic goals,

09:32.250 --> 09:36.316
and it's work that we could maybe get integrated upstream

09:36.333 --> 00:09:38.100
in a reasonable amount of time.

00:09:38.200 --> 00:09:42.500
But it's also worth considering what more we can do,

00:09:42.600 --> 09:46.600
and what effect Guile-Emacs might have on Emacs

09:46.600 --> 00:09:50.566
if it becomes simply Emacs.

00:09:50.666 --> 09:54.033
For context, the amount of C code in Emacs

09:54.050 --> 09:57.400
has increased by around 50% in the last decade,

09:57.400 --> 09:59.950
and now it constitutes around a quarter of the code base.

09:59.966 --> 10:06.400
C can be a bit of a barrier

10:06.400 --> 00:10:10.900
to customizing and extending Emacs.

00:10:11.000 --> 10:15.516
For example, there are about 1500 C subroutines.

10:15.533 --> 10:19.633
Around 500 are used in C code,

10:19.650 --> 10:23.150
as well as available to Lisp code,

10:23.166 --> 10:25.800
and being written in C means

10:25.800 --> 10:28.066
that they can't be practically redefined.

10:28.083 --> 10:34.433
The use of C can become a barrier to extending Emacs

10:34.450 --> 10:36.233
or customizing its behavior.

10:36.250 --> 10:39.200
We might consider writing

10:39.200 --> 00:10:42.816
as much of Emacs as possible in Lisp.

00:10:42.916 --> 10:46.033
One way to speed up this process

10:46.050 --> 10:49.400
would be to provide a Common Lisp implementation for Guile.

10:49.400 --> 10:54.833
Note that between Guile Elisp and Guile Scheme,

10:54.850 --> 10:57.516
we have all of the essential ingredients

10:57.533 --> 11:03.200
for a Common Lisp environment. We can also share code

11:03.200 --> 00:11:06.016
with other Common Lisp implementations

11:06.016 --> 11:10.200
such as SBCL and SICL.

11:10.200 --> 11:13.800
Overall, the duration of the project

11:13.800 --> 11:16.916
will be better measured in months rather than years,

11:16.933 --> 11:19.466
despite Common Lisp's reputation

11:19.483 --> 00:11:21.116
for being a large language.

00:11:21.216 --> 11:24.466
This could have multiple uses, of course.

11:24.483 --> 11:29.633
It could be a model for future improvements to Elisp,

11:29.650 --> 11:34.866
because Elisp and CL can interact directly without problems.

11:34.883 --> 11:38.400
And it would be very easy for Elisp

11:38.400 --> 11:41.466
to borrow language features from Common Lisp.

11:41.483 --> 11:46.600
But for the purpose of a C to Lisp transition,

11:46.600 --> 11:50.066
it would also provide us with instant access

11:50.083 --> 11:52.600
to a huge number of high-quality libraries

11:52.600 --> 11:54.833
for things that

11:54.850 --> 11:58.116
Guile is not necessarily equipped to deal with,

11:58.133 --> 12:01.350
such as access to low-level Windows APIs,

12:01.366 --> 12:05.150
as well as lots of other libraries,

12:05.166 --> 12:10.000
such as interfaces to GUI toolkits

12:10.000 --> 00:12:13.766
for a variety of operating systems.

00:12:13.866 --> 12:20.550
At a certain point, this has technical advantages.

12:20.566 --> 00:12:24.216
If most of Emacs is written in Lisp,

00:12:24.216 --> 12:27.233
then we could consider using Guile Hoot

12:27.250 --> 12:29.666
to compile Emacs to WebAssembly,

12:29.683 --> 12:33.200
making it available perhaps in web browsers

12:33.200 --> 12:37.233
or on systems with the WebAssembly System Interface.

12:37.250 --> 12:41.266
But it would also be a great victory

12:41.283 --> 12:43.033
for practical software freedom.

12:43.050 --> 12:45.866
That's the idea that Freedom One,

12:45.883 --> 12:48.350
the freedom to study and modify programs,

12:48.366 --> 12:51.633
should not just be legally and technically possible,

12:51.650 --> 12:53.316
but should be actively encouraged

12:53.333 --> 12:57.066
by our computing environments.

12:57.083 --> 13:00.116
Emacs is really one of the archetypal examples of this,

13:00.133 --> 00:13:03.116
but we can and should go further.

00:13:03.216 --> 13:08.400
When Emacs is implemented primarily in Lisp,

13:08.400 --> 13:11.466
the entirety of the system

13:11.483 --> 13:14.600
will be transparent to examination

13:14.600 --> 13:16.066
and open to modification.

13:16.083 --> 13:21.200
Every part of Emacs will be instantaneously inspectable,

13:21.200 --> 00:13:24.916
redefinable, and debuggable.

00:13:25.016 --> 13:28.266
This will be a fundamental change

13:28.283 --> 13:32.800
in what is possible to do with Emacs extensions.

13:32.800 --> 13:37.000
For example, one experiment I'd be interested in

13:37.000 --> 13:40.316
is using the Common Lisp Interface Manager

13:40.333 --> 13:43.233
as the basis for Emacs's user interface.

13:43.250 --> 13:48.516
Screwlisp is giving a talk about McCLIM later today,

13:48.533 --> 13:53.233
but for present purposes,

13:53.250 --> 13:55.633
just think of it as a super-powered version

13:55.650 --> 13:58.350
of Emacs's concept of interactive functions.

13:58.366 --> 14:02.800
It would be a pretty long-term project

14:02.800 --> 14:04.800
in Emacs as it currently exists,

14:04.800 --> 14:06.600
but it would be almost trivial

14:06.600 --> 14:12.633
if Emacs were customizable at the lowest layers via Lisp.

14:12.650 --> 14:19.150
We'll certainly be looking at the practicality

14:19.166 --> 14:20.950
of these kinds of changes

14:20.966 --> 00:14:25.033
as we continue developing Guile-Emacs.

00:14:25.133 --> 00:14:29.933
Finally, how can you get involved

00:14:30.033 --> 00:14:32.400
with and support Guile Emacs?

00:14:32.500 --> 14:35.316
One way to help is just by trying it out

14:35.333 --> 00:14:37.716
and letting us know what your experiences are like.

00:14:37.816 --> 14:41.466
There will be a snapshot available

14:41.483 --> 00:14:44.166
on the Codeberg project site

00:14:44.266 --> 14:47.000
of the version that I'm using to give this presentation.

14:47.000 --> 14:51.116
It will be available both as a Guix package

14:51.133 --> 14:55.916
and as a portable tarball. This will be more interesting

14:55.933 --> 00:15:00.266
as we get closer to a complete rebase.

00:15:00.366 --> 15:06.516
We're also always happy to talk to potential contributors

15:06.533 --> 00:15:12.100
or potential collaborators from other projects.

00:15:12.200 --> 15:16.433
We can always use bug reports,

15:16.450 --> 15:18.866
and we're interested in what kind of features

15:18.883 --> 15:21.716
people actually want to see in Guile-Emacs.

15:21.733 --> 00:15:25.200
Guile-Emacs is also being developed

00:15:25.300 --> 00:15:27.816
by a small worker cooperative,

00:15:27.916 --> 00:15:33.100
so donations are a pretty direct way to support the project.

00:15:33.200 --> 15:37.150
If you do nothing else, I recommend going to the website

15:37.166 --> 15:40.716
and subscribing to our mailing lists

15:40.733 --> 15:45.600
so that you can keep up with news on the project.

15:45.600 --> 15:47.316
If you're watching this at EmacsConf,

15:47.333 --> 15:50.466
there will be a Q&A session immediately following this,

15:50.483 --> 15:57.066
and thanks for watching!