WEBVTT captioned by anush 00:00.000 --> 00:03.066 Hello everyone, I'm Robin Templeton, 00:03.083 --> 00:05.750 and I'm going to talk about Emacs Beguiled 00:05.766 --> 00:13.866 and recent progress on the Guile Emacs project. 00:13.883 --> 00:16.433 First of all, if you're not familiar with Guile, 00:16.450 --> 00:19.716 it's an implementation of the Scheme programming language, 00:19.733 --> 00:22.150 which is a dialect of Lisp, 00:22.166 --> 00:24.550 and in the same family as Emacs Lisp, 00:24.566 --> 00:28.150 and Guile is GNU's official extension language. 00:28.166 --> 00:30.400 The goal of the Guile Emacs project 00:30.400 --> 00:34.950 is to use Guile as the basis for Emacs's Lisp support. 00:34.966 --> 00:37.116 It has two main components: 00:37.133 --> 00:41.033 a new Emacs Lisp compiler built on top of Guile, 00:41.050 --> 00:42.550 and a variant of Emacs 00:42.566 --> 00:45.316 in which the built-in Lisp implementation 00:45.333 --> 00:49.716 is entirely replaced with Guile Elisp. 00:49.733 --> 00:52.800 We expect the combination of these two projects 00:52.800 --> 00:57.350 to have several benefits. One is improved performance. 00:57.366 --> 01:00.200 Another is increased expressiveness for Elisp 01:00.200 --> 01:04.350 and making it easier to extend 01:04.366 --> 01:07.200 and experiment with the language. 01:07.200 --> 01:08.550 Finally, it will reduce 01:08.566 --> 01:12.316 Emacs's reliance on C for two reasons. 01:12.333 --> 01:16.316 Guile will be responsible for the language implementation, 01:16.333 --> 01:21.350 so Emacs will no longer have to include a Lisp interpreter. 01:21.366 --> 01:23.200 It will also become possible 01:23.200 --> 01:25.666 to implement much more of Emacs in Lisp 01:25.683 --> 01:29.233 than is currently feasible. 01:29.250 --> 01:31.116 Of course, this raises the question of 01:31.133 --> 01:34.033 why Guile is suitable for this product. 01:34.050 --> 01:36.666 And we chose Guile for a few reasons. 01:36.683 --> 01:39.400 Guile is primarily a Scheme implementation, 01:39.400 --> 01:42.150 but it also has built-in support for multiple languages 01:42.166 --> 01:43.466 using its compiler tower. 01:43.483 --> 01:46.866 To add support for a new language to Guile, 01:46.883 --> 01:50.066 You only have to write a compiler 01:50.083 --> 01:52.550 from the source language to TRIAL[??], 01:52.566 --> 01:55.800 which is essentially a low-level, 01:55.800 --> 01:58.866 minimal representation of Scheme. 01:58.883 --> 02:01.800 All of Guile's compiler optimizations 02:01.800 --> 02:04.433 occur at the TRIAL[??] layer or lower, 02:04.450 --> 02:06.033 so you don't need to worry 02:06.050 --> 02:09.633 about the lower-level details of the compiler 02:09.650 --> 02:12.350 when initially implementing your language. 02:12.366 --> 02:14.633 Guile also has some Lisp features 02:14.650 --> 02:18.316 that are very rare in schema implementations. 02:18.333 --> 02:20.033 For example, it has a nil value 02:20.050 --> 02:23.916 that counts as both false and an empty list, 02:23.933 --> 02:25.633 just like an Elisp, 02:25.650 --> 02:30.466 and it also has a version of the Common Lisp object system 02:30.483 --> 02:37.200 and its metoptic[??] protocol, which is called GOOPS. 02:37.200 --> 02:40.150 The idea of Guile Emacs has a pretty long history. 02:40.166 --> 02:43.866 going back at least three decades. 02:43.883 --> 02:44.550 There have been about 02:44.566 --> 02:48.000 half a dozen previous implementation attempts. 02:48.000 --> 02:49.950 But the current iteration began with 02:49.966 --> 02:52.866 a series of six Summer of Code internships, 02:52.883 --> 02:56.033 Daniel Kraft's[??] in 2009, 02:56.050 --> 03:01.200 and then my internships from 2010 to 2014. 03:01.200 --> 03:03.000 My basic implementation strategy 03:03.000 --> 03:05.316 was pretty straightforward. 03:05.333 --> 03:07.466 I implemented a core subset of Elisp, 03:07.483 --> 03:10.400 which was enough to run some batch mode programs 03:10.400 --> 03:12.833 outside of Emacs. 03:12.850 --> 03:15.266 In Emacs, I modified the garbage collector 03:15.283 --> 03:18.600 and the data structures for Lisp objects 03:18.600 --> 03:23.033 to use their libgal equivalents. 03:23.050 --> 03:26.950 I replaced Emacs' Lisp evaluator 03:26.966 --> 03:32.200 with the one provided by guile-elisp.[??] 03:32.200 --> 03:34.033 After a little over a year of work 03:34.050 --> 03:37.950 at the end of the 2014 internship, 03:37.966 --> 03:44.316 I ended up with a fully functional prototype of Guile Emacs. 03:44.333 --> 03:48.916 It used Guile Elisp alone as its Lisp implementation 03:48.933 --> 03:53.916 and was completely compatible with Emacs functionality 03:53.933 --> 03:56.716 and with external extensions. 03:56.733 --> 03:59.433 One caveat was that performance was pretty bad, 03:59.450 --> 04:03.033 because I was focused on correctness, 04:03.050 --> 04:07.600 as well as ease of integration with the Emacs C code. 04:07.600 --> 04:11.550 But it was nonetheless a major milestone for the project. 04:11.566 --> 04:19.600 Let's take just a moment to look at guile-elisp. 04:19.600 --> 04:23.233 For starters, we have access to guile modules. 04:23.250 --> 04:25.116 If we call guile's version function, 04:25.133 --> 04:30.516 we can see that we're running under guile 3.0, 04:30.533 --> 04:33.233 have access to some of the numeric tower 04:33.250 --> 04:39.516 via the arithmetic functions. We also have multiple values. 04:39.533 --> 04:43.950 We have to be careful to use Guile's values procedure here, 04:43.966 --> 04:46.666 not the CL libraries, 04:46.683 --> 04:48.833 but you can see that this works properly 04:48.850 --> 04:51.550 rather than being an emulation. 04:51.566 --> 04:54.033 Finally, we have tail call elimination. 04:54.050 --> 05:02.866 Naturally, we're going to use factorial to demonstrate it. 05:02.883 --> 05:05.633 If n is zero, return the answer, 05:05.650 --> 05:14.266 else recurse with n less one and n times a. 05:14.283 --> 05:16.150 Of course this definition works correctly, 05:16.166 --> 05:18.950 but it gets more interesting 05:18.966 --> 00:05:25.000 if we communicate the answer with an error. 00:05:25.100 --> 05:29.633 or to look at a backtrace. 05:29.650 --> 05:32.350 You can see here that there are 05:32.366 --> 05:35.516 no calls to fact visible in between 05:35.533 --> 05:37.833 the request to evaluate 05:37.850 --> 05:42.200 and the error communicating the answer. 05:42.200 --> 05:44.200 That's because this tail call 05:44.200 --> 05:48.350 has been optimized into effectively a goto. 05:48.366 --> 05:54.916 This is essential for any kind 05:54.933 --> 00:05:59.916 of serious functional programming. 00:06:00.116 --> 06:03.033 That's a peek at guile-elisp. 06:03.050 --> 06:08.066 In 2015, I left university to go work on web technologies, 06:08.083 --> 06:11.316 and the project was dormant for a very long time. 06:11.333 --> 06:13.433 But that's been changing recently. 06:13.450 --> 06:16.066 During the last few months, 06:16.083 --> 06:17.633 I've been working with Larry Valkama[??] 06:17.650 --> 06:20.716 to rebase guile-emacs 06:20.733 --> 06:24.833 onto the development branch of upstream emacs, 06:24.850 --> 06:29.666 including the past decade's worth of upstream development. 06:29.683 --> 00:06:33.967 What we've ended up with is a series of 00:06:34.267 --> 00:06:37.550 rebases onto different versions of Emacs. 06:37.566 --> 06:39.516 The older ones tend to work pretty well. 06:39.533 --> 06:46.866 The newer ones have increasingly bad problems 06:46.883 --> 06:49.800 where they haven't been properly adjusted 06:49.800 --> 06:52.200 for changes in the Emacs implementation. 06:52.200 --> 06:56.833 but we do have by now a version of Emacs 30 06:56.850 --> 06:57.800 which boots correctly 06:57.800 --> 06:59.833 and can be used for interactive debugging, 06:59.850 --> 07:06.150 as well as the ability to bisect the revisions of Emacs 07:06.166 --> 07:10.516 and find out where regressions were introduced. 07:10.533 --> 07:14.033 Our immediate goal is of course to complete the rebase. 07:14.050 --> 07:16.233 At the same time, 07:16.250 --> 07:20.633 we want to improve Guile Elisp's performance 07:20.650 --> 07:24.350 to at least be competitive with ordinary Emacs Lisp. 07:24.366 --> 07:29.266 Just to characterize the performance situation, 07:29.283 --> 07:32.750 Guile Elisp is usually about half 07:32.766 --> 07:34.466 as fast as ordinary Elisp, 07:34.483 --> 07:37.833 while Guile Scheme is quite often 07:37.850 --> 00:07:41.250 an order of magnitude faster than ordinary Elisp, 00:07:41.350 --> 07:43.916 and that's based on micro benchmarks 07:43.933 --> 00:07:46.133 like the Gabriel[??] benchmarks. 00:07:46.233 --> 00:07:50.900 but there's clearly a lot of room 00:07:50.900 --> 00:07:53.150 to improve our compiler's output. 00:07:53.350 --> 07:56.633 If you want to mark your calendars, 07:56.650 --> 08:00.150 we're expecting to have a usable version of Guile Emacs 30 08:00.166 --> 00:08:03.016 out sometime next spring. 00:08:03.116 --> 08:05.433 We're also going to put some effort 08:05.450 --> 00:08:09.000 into either extracting old work 00:08:09.100 --> 08:12.600 or doing new work that could be contributed upstream. 08:12.600 --> 00:08:16.650 On the Guile side, we'll probably start out with 00:08:16.750 --> 00:08:21.033 optimizing the dynamic binding facilities, 00:08:21.233 --> 08:23.433 which are used very seldom in Scheme, 08:23.450 --> 08:27.833 but are used all the time in traditional Lisp dialects. 08:27.850 --> 08:31.400 On the Emacs side, we'll be working initially 08:31.400 --> 08:35.316 on abstracting away the details of the Lisp implementation 08:35.333 --> 00:08:37.433 where they're not relevant, 00:08:37.533 --> 08:40.716 And that will clean up the Emacs code base a bit. 08:40.733 --> 08:45.000 It'll make it easier to integrate Emacs and Guile Elisp. 08:45.000 --> 08:47.916 It will probably be helpful for anyone 08:47.933 --> 08:51.550 who is working on ordinary Elisp on their own. 08:51.566 --> 08:57.200 We're also going to be adding new features to Emacs Lisp. 08:57.200 --> 08:59.316 We've seen a few of them already. 08:59.333 --> 09:02.633 The new [??] tower, tail call optimization, 09:02.650 --> 09:04.550 common list compatibility. 09:04.566 --> 09:07.950 We're also going to provide access to Fibers, 09:07.966 --> 09:12.466 which is a guide library based on ideas from concurrent ML 09:12.483 --> 09:15.716 that provides much more powerful facilities 09:15.733 --> 09:18.266 for concurrent and parallel programming 09:18.283 --> 00:09:24.566 than what Emacs currently offers. 00:09:24.666 --> 09:32.233 This plan meets Guile Emacs' basic goals, 09:32.250 --> 09:36.316 and it's work that we could maybe get integrated upstream 09:36.333 --> 00:09:38.100 in a reasonable amount of time. 00:09:38.200 --> 00:09:42.500 But it's also worth considering what more we can do, 00:09:42.600 --> 09:46.600 and what effect Guile Emacs might have on Emacs 09:46.600 --> 00:09:50.566 if it becomes simply Emacs. 00:09:50.666 --> 09:54.033 For context, the amount of C code in Emacs 09:54.050 --> 09:57.400 has increased by around 50% in the last decade, 09:57.400 --> 09:59.950 and now it constitutes around a quarter of the code base. 09:59.966 --> 10:06.400 C can be a bit of a barrier 10:06.400 --> 00:10:10.900 to customizing and extending Emacs. 00:10:11.000 --> 10:15.516 For example, there are about 1500 C subroutines. 10:15.533 --> 10:19.633 Around 500 are used in C code, 10:19.650 --> 10:23.150 as well as available to Lisp code, 10:23.166 --> 10:25.800 and being written in C means 10:25.800 --> 10:28.066 that they can't be practically redefined. 10:28.083 --> 10:34.433 the use of C can become a barrier to extending Emacs 10:34.450 --> 10:36.233 or customizing its behavior. 10:36.250 --> 10:39.200 We might consider writing 10:39.200 --> 00:10:42.816 as much of Emacs as possible in Lisp. 00:10:42.916 --> 10:46.033 One way to speed up this process 10:46.050 --> 10:49.400 would be to provide a common Lisp implementation for Guile. 10:49.400 --> 10:54.833 Note that between guile-elisp and guile-scheme, 10:54.850 --> 10:57.516 we have all of the essential ingredients 10:57.533 --> 11:03.200 for a Common Lisp environment. We can also share code 11:03.200 --> 00:11:06.016 with other Common Lisp implementations 11:06.016 --> 11:10.200 such as SBCL and SICL[??]. 11:10.200 --> 11:13.800 Overall, the duration of the project 11:13.800 --> 11:16.916 will be better measured in months rather than years, 11:16.933 --> 11:19.466 despite Common Lisp's reputation 11:19.483 --> 00:11:21.116 for being a large language. 00:11:21.216 --> 11:24.466 This could have multiple uses, of course. 11:24.483 --> 11:29.633 It could be a model for future improvements to Elisp, 11:29.650 --> 11:34.866 because Elisp and CL can interact directly without problems. 11:34.883 --> 11:38.400 and it would be very easy for Elisp 11:38.400 --> 11:41.466 to borrow language features from Common Lisp. 11:41.483 --> 11:46.600 But for the purpose of a C to Lisp transition, 11:46.600 --> 11:50.066 it would also provide us with instant access 11:50.083 --> 11:52.600 to a huge number of high-quality libraries 11:52.600 --> 11:54.833 for things that 11:54.850 --> 11:58.116 Guile is not necessarily equipped to deal with, 11:58.133 --> 12:01.350 such as access to low-level Windows APIs, 12:01.366 --> 12:05.150 as well as lots of other libraries, 12:05.166 --> 12:10.000 such as interfaces to GUI toolkits 12:10.000 --> 00:12:13.766 for a variety of operating systems. 00:12:13.866 --> 12:20.550 At a certain point, this has technical advantages. 12:20.566 --> 00:12:24.216 If most of Emacs is written in Lisp, 00:12:24.216 --> 12:27.233 then we could consider using Guile Hoot 12:27.250 --> 12:29.666 to compile Emacs to WebAssembly, 12:29.683 --> 12:33.200 making it available perhaps in web browsers 12:33.200 --> 12:37.233 or on systems with the WebAssembly system interface. 12:37.250 --> 12:41.266 But it would also be a great victory 12:41.283 --> 12:43.033 for practical software freedom. 12:43.050 --> 12:45.866 That's the idea that freedom one, 12:45.883 --> 12:48.350 the freedom to study and modify programs, 12:48.366 --> 12:51.633 should not just be legally and technically possible, 12:51.650 --> 12:53.316 but should be actively encouraged 12:53.333 --> 12:57.066 by our competing environments. 12:57.083 --> 13:00.116 Emacs is really one of the archetypal examples of this, 13:00.133 --> 00:13:03.116 but we can and should go further. 00:13:03.216 --> 13:08.400 When Emacs is implemented primarily in Lisp, 13:08.400 --> 13:11.466 the entirety of the system 13:11.483 --> 13:14.600 will be transparent to examination 13:14.600 --> 13:16.066 and open to modification. 13:16.083 --> 13:21.200 Every part of Emacs will be instantaneously inspectable, 13:21.200 --> 00:13:24.916 redefinable, and debuggable. 00:13:25.016 --> 13:28.266 This will be a fundamental change 13:28.283 --> 13:32.800 in what is possible to do with Emacs extensions. 13:32.800 --> 13:37.000 For example, one experiment I'd be interested in 13:37.000 --> 13:40.316 is using the Common Lisp Interface Manager 13:40.333 --> 13:43.233 as the basis for Emacs's user interface. 13:43.250 --> 13:48.516 Screwlisp is giving a talk about McCLIM later today, 13:48.533 --> 13:53.233 but for present purposes, 13:53.250 --> 13:55.633 just think of it as a super-powered version 13:55.650 --> 13:58.350 of Emacs's concept of interactive functions. 13:58.366 --> 14:02.800 It would be a pretty long-term project 14:02.800 --> 14:04.800 in Emacs as it currently exists, 14:04.800 --> 14:06.600 but it would be almost trivial 14:06.600 --> 14:12.633 if Emacs were customizable at the lowest layers via Lisp. 14:12.650 --> 14:19.150 We'll certainly be looking at the practicality 14:19.166 --> 14:20.950 of these kinds of changes 14:20.966 --> 00:14:25.033 as we continue developing Guile Emacs. 00:14:25.133 --> 00:14:29.933 Finally, how can you get involved 00:14:30.033 --> 00:14:32.400 with and support Guile Emacs? 00:14:32.500 --> 14:35.316 One way to help is just by trying it out 14:35.333 --> 00:14:37.716 and letting us know what your experiences are like. 00:14:37.816 --> 14:41.466 There will be a snapshot available 14:41.483 --> 00:14:44.166 on the Codeberg project site 00:14:44.266 --> 14:47.000 of the version that I'm using to give this presentation. 14:47.000 --> 14:51.116 It will be available both as a Guix package 14:51.133 --> 14:55.916 and as a portable tarball. This will be more interesting 14:55.933 --> 00:15:00.266 as we get closer to a complete rebase. 00:15:00.366 --> 15:06.516 We're also always happy to talk to potential contributors 15:06.533 --> 00:15:12.100 or potential collaborators from other projects. 00:15:12.200 --> 15:16.433 We can always use bug reports, 15:16.450 --> 15:18.866 and we're interested in what kind of features 15:18.883 --> 15:21.716 people actually want to see in Guile Emacs. 15:21.733 --> 00:15:25.200 Guile Emacs is also being developed 00:15:25.300 --> 00:15:27.816 by a small worker cooperative, 00:15:27.916 --> 00:15:33.100 so donations are a pretty direct way to support the project. 00:15:33.200 --> 15:37.150 If you do nothing else, I recommend going to the website 15:37.166 --> 15:40.716 and subscribing to our mailing lists 15:40.733 --> 15:45.600 so that you can keep up with news on the project. 15:45.600 --> 15:47.316 If you're watching this at Emacsconf, 15:47.333 --> 15:50.466 there will be a Q&A session immediately following this, 15:50.483 --> 15:57.066 and thanks for watching.