Hello everyone. I'm Robin Templeton,
and I'm going to talk about Emacs Beguiled
and recent progress on the Guile-Emacs project.
First of all, if you're not familiar with Guile,
it's an implementation of the Scheme programming language,
which is a dialect of Lisp,
and in the same family as Emacs Lisp,
and Guile is GNU's official extension language.
The goal of the Guile-Emacs project
is to use Guile as the basis for Emacs's Lisp support.
It has two main components:
a new Emacs Lisp compiler built on top of Guile,
and a variant of Emacs
in which the built-in Lisp implementation
is entirely replaced with Guile Elisp.
We expect the combination of these two projects
to have several benefits. One is improved performance.
Another is increased expressiveness for Elisp
and making it easier to extend
and experiment with the language.
Finally, it will reduce
Emacs's reliance on C for two reasons.
Guile will be responsible for the language implementation,
so Emacs will no longer have to include a Lisp interpreter.
It will also become possible
to implement much more of Emacs in Lisp
than is currently feasible.
Of course, this raises the question of
why Guile is suitable for this project.
And we chose Guile for a few reasons.
Guile is primarily a Scheme implementation,
but it also has built-in support for multiple languages
using its compiler tower.
To add support for a new language to Guile,
you only have to write a compiler
from the source language to Tree-IL,
which is essentially a low-level,
minimal representation of Scheme.
All of Guile's compiler optimizations
occur at the Tree-IL layer or lower,
so you don't need to worry
about the lower-level details of the compiler
when initially implementing your language.
Guile also has some Lisp features
that are very rare in Scheme implementations.
For example, it has a nil value
that counts as both false and an empty list,
just like in Elisp,
and it also has a version of the Common Lisp Object System
and its metaobject protocol, which is called GOOPS.
The idea of Guile-Emacs has a pretty long history,
going back at least three decades.
There have been about
half a dozen previous implementation attempts.
But the current iteration began with
a series of six Summer of Code internships:
Daniel Kraft's in 2009,
and then my internships from 2010 to 2014.
My basic implementation strategy
was pretty straightforward.
I implemented a core subset of Elisp,
which was enough to run some batch mode programs
outside of Emacs.
In Emacs, I modified the garbage collector
and the data structures for Lisp objects
to use their libguile equivalents.
I replaced Emacs' Lisp evaluator
with the one provided by Guile Elisp.
After a little over a year of work,
at the end of the 2014 internship,
I ended up with a fully functional prototype of Guile-Emacs.
It used Guile Elisp alone as its Lisp implementation
and was completely compatible with Emacs functionality
and with external extensions.
One caveat was that performance was pretty bad,
because I was focused on correctness,
as well as ease of integration with the Emacs C code.
But it was nonetheless a major milestone for the project.
Let's take just a moment to look at Guile-Elisp.
For starters, we have access to Guile modules.
If we call Guile's version function,
we can see that we're running under Guile 3.0.
We have access to some of the numeric tower
via the arithmetic functions. We also have multiple values.
We have to be careful to use Guile's values procedure here,
not the CL library's,
but you can see that this works properly
rather than being an emulation.
Finally, we have tail call elimination.
Naturally, we're going to use factorial to demonstrate it.
If n is zero, return the answer,
else recurse with n less one and n times a.
Of course, this definition works correctly,
but it gets more interesting
if we communicate the answer with an error,
in order to look at a backtrace.
You can see here that there are
no calls to fact visible in between
the request to evaluate
and the error communicating the answer.
That's because this tail call
has been optimized into effectively a goto.
This is essential for any kind
of serious functional programming.
That's a peek at Guile-Elisp.
In 2015, I left university to go work on web technologies,
and the project was dormant for a very long time.
But that's been changing recently.
During the last few months,
I've been working with Larry Valkama
to rebase Guile-Emacs
onto the development branch of upstream Emacs,
including the past decade's worth of upstream development.
What we've ended up with is a series of
rebases onto different versions of Emacs.
The older ones tend to work pretty well.
The newer ones have increasingly bad problems
where they haven't been properly adjusted
for changes in the Emacs implementation.
But we do have by now a version of Emacs 30
which boots correctly
and can be used for interactive debugging,
as well as the ability to bisect the revisions of Emacs
and find out where regressions were introduced.
Our immediate goal is of course to complete the rebase.
At the same time,
we want to improve Guile Elisp's performance
to at least be competitive with ordinary Emacs Lisp.
Just to characterize the performance situation,
Guile Elisp is usually about half
as fast as ordinary Elisp,
while Guile Scheme is quite often
an order of magnitude faster than ordinary Elisp,
and that's based on micro benchmarks
like the Gabriel benchmarks.
But there's clearly a lot of room
to improve our compiler's output.
If you want to mark your calendars,
we're expecting to have a usable version of Guile-Emacs 30
out sometime next spring.
We're also going to put some effort
into either extracting old work
or doing new work that could be contributed upstream.
On the Guile side, we'll probably start out with
optimizing the dynamic binding facilities,
which are used very seldom in Scheme,
but are used all the time in traditional Lisp dialects.
On the Emacs side, we'll be working initially
on abstracting away the details of the Lisp implementation
where they're not relevant.
And that will clean up the Emacs code base a bit.
It'll make it easier to integrate Emacs and Guile Elisp.
It will probably be helpful for anyone
who is working on ordinary Elisp on their own.
We're also going to be adding new features to Emacs Lisp.
We've seen a few of them already.
The numeric tower, tail call optimization,
Common Lisp compatibility.
We're also going to provide access to Fibers,
which is a Guile library based on ideas from Concurrent ML
that provides much more powerful facilities
for concurrent and parallel programming
than what Emacs currently offers.
This plan meets Guile-Emacs' basic goals,
and it's work that we could maybe get integrated upstream
in a reasonable amount of time.
But it's also worth considering what more we can do,
and what effect Guile-Emacs might have on Emacs
if it becomes simply Emacs.
For context, the amount of C code in Emacs
has increased by around 50% in the last decade,
and now it constitutes around a quarter of the code base.
C can be a bit of a barrier
to customizing and extending Emacs.
For example, there are about 1500 C subroutines.
Around 500 are used in C code,
as well as available to Lisp code,
and being written in C means
that they can't be practically redefined.
The use of C can become a barrier to extending Emacs
or customizing its behavior.
We might consider writing
as much of Emacs as possible in Lisp.
One way to speed up this process
would be to provide a Common Lisp implementation for Guile.
Note that between Guile Elisp and Guile Scheme,
we have all of the essential ingredients
for a Common Lisp environment. We can also share code
with other Common Lisp implementations
such as SBCL and SICL.
Overall, the duration of the project
will be better measured in months rather than years,
despite Common Lisp's reputation
for being a large language.
This could have multiple uses, of course.
It could be a model for future improvements to Elisp,
because Elisp and CL can interact directly without problems.
And it would be very easy for Elisp
to borrow language features from Common Lisp.
But for the purpose of a C to Lisp transition,
it would also provide us with instant access
to a huge number of high-quality libraries
for things that
Guile is not necessarily equipped to deal with,
such as access to low-level Windows APIs,
as well as lots of other libraries,
such as interfaces to GUI toolkits
for a variety of operating systems.
At a certain point, this has technical advantages.
If most of Emacs is written in Lisp,
then we could consider using Guile Hoot
to compile Emacs to WebAssembly,
making it available perhaps in web browsers
or on systems with the WebAssembly System Interface.
But it would also be a great victory
for practical software freedom.
That's the idea that Freedom One,
the freedom to study and modify programs,
should not just be legally and technically possible,
but should be actively encouraged
by our computing environments.
Emacs is really one of the archetypal examples of this,
but we can and should go further.
When Emacs is implemented primarily in Lisp,
the entirety of the system
will be transparent to examination
and open to modification.
Every part of Emacs will be instantaneously inspectable,
redefinable, and debuggable.
This will be a fundamental change
in what is possible to do with Emacs extensions.
For example, one experiment I'd be interested in
is using the Common Lisp Interface Manager
as the basis for Emacs's user interface.
Screwlisp is giving a talk about McCLIM later today,
but for present purposes,
just think of it as a super-powered version
of Emacs's concept of interactive functions.
It would be a pretty long-term project
in Emacs as it currently exists,
but it would be almost trivial
if Emacs were customizable at the lowest layers via Lisp.
We'll certainly be looking at the practicality
of these kinds of changes
as we continue developing Guile-Emacs.
Finally, how can you get involved
with and support Guile Emacs?
One way to help is just by trying it out
and letting us know what your experiences are like.
There will be a snapshot available
on the Codeberg project site
of the version that I'm using to give this presentation.
It will be available both as a Guix package
and as a portable tarball. This will be more interesting
as we get closer to a complete rebase.
We're also always happy to talk to potential contributors
or potential collaborators from other projects.
We can always use bug reports,
and we're interested in what kind of features
people actually want to see in Guile-Emacs.
Guile-Emacs is also being developed
by a small worker cooperative,
so donations are a pretty direct way to support the project.
If you do nothing else, I recommend going to the website
and subscribing to our mailing lists
so that you can keep up with news on the project.
If you're watching this at EmacsConf,
there will be a Q&A session immediately following this,
and thanks for watching!