[[!meta title="Gypsum: my clone of Emacs and ELisp written in Scheme"]]
[[!meta copyright="Copyright © 2024 Ramin Honary"]]
[[!inline pages="internal(2024/info/gypsum-nav)" raw="yes"]]
<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->
# Gypsum: my clone of Emacs and ELisp written in Scheme
Ramin Honary (he/him)
- Source code: <https://codeberg.org/ramin_hal9001/gypsum>
- E-mail: <mailto:ramin.honary@gmail.com>
- ActivityPub: [@ramin_hal9001@fe.disroot.org](https://fe.disroot.org/@ramin_hal9001)
- Website: <https://tilde.town/~ramin_hal9001>
[[!inline pages="internal(2024/info/gypsum-before)" raw="yes"]]
## Slides
### Introduction
1. Ramin Honary
- Emacs enthusiast since 2017
- Software developer (full stack)
- I love Haskell, Scheme, functional programming
- Started learning Scheme about 2 years ago
2. My project: an Emacs Clone
- Tentative name: "Gypsum"
- Its not a great name, open to suggestions.
### Goal: to Clone **Emacs Lisp**
- Many clones already:
- Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile
- These only clone the key bindings, not Elisp
- Only XEmacs (a fork of GNU Emacs) provided an alternative Emacs Lisp
### Most people don't use Emacs for the keybindings
- Anecodtal, but yes really.
- Use Emacs because of the power of Emacs Lisp
- Emacs is as powerful as a system shell
- A good language is what makes it powerful
### Goal: use R7RS Standard Scheme
- I want it to work on a many Scheme implementations
- Guile is the reference implementation
- (more about this later)
### Goal: able to run any `init.el`
- Should be able to use `init.el` without significant changes
- Many invest significant time in their configs
- Suddenly not having your config is disruptive
- Such an Emacs clone would be more useful
### Why do this?
- I personally like Scheme's minimalism.
- Use Scheme as more than just an academic language.
- Seems to be a lot of interest in a project like this.
- Talk of "Guile Emacs" for about 30 years
### A long history of Guile Emacs (1/3)
- **Early 90s**: Initial discussion between RMS, Tom Lord,
Aubrey Jaffer, begin work on replacing Emacs Lisp with Scheme.
- **1999--2009**: Ken Raeburn's [Guile-Based Emacs](https://www.mit.edu/~raeburn/guilemacs/).
(My project is similar.)
> "*This project that I (Ken Raeburn) have started is for converting*
> *GNU Emacs to use Guile as its programming language. Support for*
> *Emacs Lisp will continue to exist, of course, but it may be*
> *through translation and/or interpretation; the Lisp engine itself*
> *may no longer be the core of the program.*"
### A long history of Guile Emacs (2/3)
- **2010**: Andy Wingo and Ludovic Courtes
take maintainership of Guile project.
- **2009--2011**: Emacs Lisp interpreter implemented in Guile.
Still ships with Guile.
- **2011**: Guile 2.0 is released
- **2011--2015**: Robin Templeton's GSoC project.
(Is presenting later today!)
### A long history of Guile Emacs (3/3)
- **2020**: Vasilij Schneidermann published an overview called
"[The State of Emacs Lisp on Guile](https://emacsninja.com/posts/state-of-emacs-lisp-on-guile.html)".
- **2020 to present**: Guile Emacs is dead? Andrea Corallo, GCC Emacs,
JIT-compiler for Emacs Lisp based on "libgccjit", brings into
question any need for combining Guile with Emacs.
### Demo
### GUI is barely working
- I have almost no experience with Gtk or GObject Introspection
- Hard to debug, crashes at C-level produce no stack traces
- Using GDB requires rebuilding all of Gtk, GIO, GLib, etc.
### Emacs Lisp parser based on Guile Emacs Lisp
- Foked the Guile Emacs Lisp implementation for easier development
- Have already submitted a patch to the parser upstream
### Emacs Lisp interpter is barely working
- Implementing my own interpreter in portable Scheme
- Monadic pattern matcher
### Can parse but not interpret "`subr.el`"
- "`subr.el`" is the first ELisp file run by Emacs
- A good way to determine what to work on first
### A call for help
### Latest Emacs has **1,393** built-in functions
- I could never implement that many functions alone
- Probably not all are required to create a useful editor
### My job is to make contributing easy
- Document the build and test process
- Document the system architecture
- Prioritize which built-in functions are most essential
- Find low-hanging fruit, use as means to teach others
### The work for which I will take responsibility
- Clone enough Elisp to be able to run ERT tests
- Then use GNU Emacs's own regression tests to test patches
- Make sure there is a usable GUI
- (Someday?) be able to contribute a patch from within
### Quick architectural overview
### The editor is based in Scheme, not Emacs Lisp
- Config, scripting, packages all done in Scheme
- Use of Emacs Lisp for scripting not encouraged
- Should still be able to run your `init.el`
- Ideally should be able to run ELPA packages
### Difference with Robin Templeton's project
- Guile-Emacs links Guile runtime into Emacs
- Not a Scheme application
- An IDE for Schemers
### Emacs Lisp is an "environment"
- "Environments" are a feature of Scheme
- Scheme procedures can be called from Emacs Lisp
- Scheme state can be mutated by Emacs Lisp
- (See "`./gypsum/elisp-eval.scm`", "`new-env`")
### "Functional Lenses"
- Because R7RS does not standardize MOP (not even in "large")
- Inspired by Haskell
- Composes getters and setters
- Single source file, easy to port
- Ported to 3 other Schemes
### A lot of work went into keymaps data structure
- Keybindings are an important part of Emacs
- Had to do this well from very beginning
- Keybindings work correctly in demo
### A lot of work went into separating GUI from Editor logic
- "Parameters" are a feature of Scheme
- Platform-specific APIs are always parameterized
- Windowing and widgets
- Translate key events to bindings
- Evaluating Scheme expressions
- Text buffering and rendering
- (See "`./gypsum/editor-impl.scm`")
### Monadic pattern matching
- Simpler, more portable
- (Not as feature-rich)
- Easier than porting SRFI-241 ("Match") to Guile
- No relation to SRFI-247 ("Syntatic Monads")
- You can still use pattern matching
### Monad pattern matching
Example program
(define push-stack (put-with cons))
(define collatz
(many
push-stack
(either
(try (check (λ (n) (<= n 1)))
(success))
(try (check odd?)
(next (λ (n) (+ 1 (* 3 n)))))
(try (check even?)
(next (λ (n) (quotient n 2))))
(fail "not an integer")
)))
### Conclusion
- I am just getting the ball rolling
- Helping others contribute is my top priority
- ActivityPub :: `ramin_hal9001@fe.disroot.org`
- E-mail :: <span class="spurious-link"
target="ramin.honary@gmail.com">*ramin.honary@gmail.com*</span>
- Homepage :: <https://tilde.town/~ramin_hal9001>
- Codeberg :: <https://codeberg.org/ramin_hal9001/gypsum>
- This presentation :: <https://emacsconf.org/2024/talks/gypsum/>
## Original presentation proposal
I would like to demonstrate an Emacs clone I have been
writing in Guile Scheme for the past year, which I am
tentatively calling "Gypsum". Unlike other editors which
only clone the Emacs keybindings (Edwin, Jed, jEdit, Jove,
Lem, MG, Yi, Zile), I hope my Emacs clone will also fully
clone the Emacs Lisp programming language well enough that
many of the packages in ELPA, Non-GNU ELPA, and perhaps even
MELPA, can be used in "Gypsum" without any modification. I
would also like to talk a little bit about the how I am
implementing it (the software architecture), and invite
others to contribute.
I think my project is of interest to many Emacs users
because, firstly, I have personally spoken with a relatively
large number of people who have expressed interest in making
Emacs programmable in Scheme. Secondly, there is a good
amount of prior art for Scheme implementations of
Emacs. There are even builds of Emacs that link to Guile
which provides a "scheme-eval" built-in function that
translates between Elisp data types and Scheme data
types. The Guile compiler itself ships with an Emacs Lisp
compiler as well, although it does not provide enough of
Emacs's built-in functions to be of much use.
So by using Guile, we can make use of a lot of the prior
art, in fact I am currently using the tokenizer and reader
used in Guile's built-in Elisp interpreter to implement
"Gypsum's" Elisp interpreter. That said, I have gone out of
my way to make my code fully R7RS compliant, so I hope I can
port it to other Scheme implementations like MIT Scheme,
Gambit, Stklos, and perhaps Chez Scheme with Gwen Weinholt's
R7-to-R6RS translator. I consider the Guile version of
Gypsum to be the reference implementation of what I hope
will become a fully cross-platform programming language and
text editor written in portable R7RS Scheme.
The reference implementation of "Gypsum" is a GUI
application based on Gtk using a library called
"Guile-GI". Guile-GI uses the GObject Introspection
framework to automatically generate Scheme language bindings
to libraries like Gtk and Glib which are written in the C
programming language. There is not yet any terminal-emulator
version of "Gypsum."
The next step of the project will be to implement enough of
Elisp that we can run tests written in the Emacs Regression
Testing (ERT) framework. We can then incorporate the
original GNU Emacs regression test suite into Gypsum. Any
new API added to Gypsum Elisp will most likely already have
regression tests we can use to make sure it is working in a
way that is compatible with GNU Emacs Lisp. I would like to
make it as easy as possible for people to contribute to this
project, and having a list of APIs to be implemented each
with a set of regression tests the APIs are expected to
pass, is a very good way to do that.
About the speaker:
My name is Ramin Honary, I have been a professional software
engineer of 16 years, lately mostly doing full-stack
software development. I have always been fascinated with
programming languages, and especially functional languages
like Lisp and Haskell. I have been using Emacs
since 2017. But lately it is with Scheme that I have been
spending most of my free time. I am only a Scheme
programming enthusiast, I am not involved with Scheme
professionally.
You may also like another talk by this speaker:
[EmacsConf - 2022 - talks - Build a Zettelkasten with the Hyperbole Rolodex](https://emacsconf.org/2022/talks/rolodex/)
# Discussion
## Questions and answers
- Q: Would it be possible to support a GUI toolkit other than GTK? Like how GNU Emacs still supports Lucid
- A: Yes this planed by having proper backend: emacs-lisp running
into a module and the GUI being another module. So normalized
communication. Currently GTK being standard implementation, also
done here.
- Q: Do you plan to provide improvements to Elisp as a language, or is the focus on a compatibility layer to facilitate doing all new extensions, etc. in Scheme?
- A: Plan is to keep up-to-date with new releases. So new GNU
feature should be included with each release. But also intend to
have support for pure Scheme features.
- Q: If Emacs Lisp support for Guile was documented
better, could you be nudged/convinced to (re)start using, and
contributing to that?
- A: Compatibility is the most important things. Documentation not
sufficient to convince users to switch.
- IRC: Where do you think elisp documentation should be improved? I've always found the built in documentation to be excellent
- janneke is referring to Guile's ELisp support, I believe; sorry, i meant the documentation of guile's elisp backend
- Q: Why is being able to interpret all of \`init.el\` an useful goal?
Sure, there is a lot of code written in elisp - can we consider a
translator like utility to convert elisp to scheme, once guile-emacs
becomes a reality?
- A: Probably, but first step is getting the interpretter working.
Emacs-lisp basically compiled down to intermediate
representation of the guile compiler \[this was one of the hard
things to get to work\]. But unclear how this works for other
schemes. Best solution probably translation elisp -\> scheme,
but this is not the approach that was done. Would be very cool
to have. Feel free to give a PR.
- Q: What is the plan to handle elisp packages that depend on 3rd party/external libraries? (libgit/magit or rg/ripgrep)?
- A: Will be tricky. If loading directly into the elisp process,
very hard. Cairo could help, but you need emacs lisp binding on
top of that. For magit, you can call regular process
communication via stdin/stdout; so you can reuse existing scheme
libraries. Dynamic libraries not a goal. Rg/ripgrep probably the
same with process communication.
- Q: Why is it not feasible for the Emacs layer that interprets Emacs Lisp (the core in C) ot have a Scheme interpreter, instead of using Guile?
- A: Guile is a scheme. Not sure what you mean.
- A: Check presentation later of Robin Templeton ("Beguiling
Emacs: Guile-Emacs relaunched!"): the attempt exists by
translating elisp to guile.
- thank you
- Q: Not really a question, but how about Schemacs as a name?
- - A: Cool name, but did not check if it is already used. Feel free
to discuss by email.
- Q: I'm curious to know how the hell guile-emacs deals with all of the dynamically scoped modules out there. Is there any effort to automatically modularize and namespace stuff?
## Notes
- oo neat, i didn't know about that first bit of history
- i've heard rms say that scheme (guile) is just a nicer
lisp; but didn't know there were concrete talks/attempts to use
guile for emacs that early
- robin: yes, guile-elisp not being portable might be a
showstopper for ramin
- I've heard good things about guile from guix people. Never really tried it out
- FWIW, I think there have been various attempts to make an Emacs clone in Common Lisp. I guess lem is currently the most active. https://github.com/lem-project/lem/
- I like how he edited his slide mid-presentation.
- https://www.emacswiki.org/emacs/GuileEmacsHistory has some info on very efforts; i was surprised that there were so many (especially with old-school guile, eww ;))
- joining for all the guiles and all the emacsen
- He's got a long way to go.
- of course there were, RMS had decreed that guile was the GNU scripting language and it was a bit embarrassing that it was so little-used...
- At least Guix uses it now.
- oh yes, guile 3 was a great step forward, and I do wonder how much of that was due to the impetus of its having real users :)
- if you're interested in guile emacs, be sure to check out robin's guile talk this afternoon
- nice, my silly https://gitlab.com/janneke/guimax also used guile-gi but this looks much more mature
- developed actively til 2014, but more recent work is on a branch so may not be as obvious...
- so...i guess that some basic documentation on elisp may be very helpful
- wbn if you could join efforts somehow
- there should definitely be some overlap between the projects
- I have 6500 interactive ones, according to Vertico...
- I got 8021 interactive ones ;)
- 7690 here
- 34557 callables \o/
- working towards a similar goal approached from different directions
- however, working on guile's elisp backend may be a common ground
- ramin's probably talking about subrs, i.e. primitives not themselves implemented in elisp
- you mean providing modularization for elisp programs? or something else
- Being more specific, removing the need to namespace every single internal variable/procedure in an elisp module. Maybe that isn't a goal, but I wish it were
- yes, i have some ideas for adapting the CL package system for that. it'd have to be opt-in, maybe some tools for automated refactoring
- And... the part where we need more bandwidth for any core runtime efforts to be viable
- my-special-module--loop-variable-3
- How can I get involved with this? if I want to contribute
- hang out in #guile-emacs and/or subscribe to the mailing lists https://guile-emacs.org/
- an embryonic re-implementation by ramin of emacs in guile, with their own new elisp interpreter that should be r7rs compatible
- would love robin's guile-emacs and ramin's efforts to somehow share some of their efforts
- Robin's talk mentions developing a better elisp in Scheme. Why can't your project leverage it?
- guile-elisp is part of guile's compiler system (elisp -> tree-il -> cps -> bytecode), unless another scheme sets out to be guile-compatible in that respect it won't be portable at all
- To be fair, I've been screwing around with chicken, guile, and racket. I haven't found any 2 scheme implementations to be compatible, even within SRFI implementations
- Just the basic syntax and semantics, nothing else
- yeah, scheme's lack of portable libraries in practice motivated me to suggest something that's sure to piss everyone off: Javascript
- YouTube comment: Cool, great, amazing. Extremely ambitious. A clone like this project might be significant harder than for example Lem, which does not care about backward compatibility with Emacs. A clone also is somewhat harder to create a unique selling point for. Especially if Guile-Emacs is also going to be a thing now. Just my 2 cents up to discussions. Anyway I am excited and will follow it:)
[[!inline pages="internal(2024/info/gypsum-after)" raw="yes"]]
[[!inline pages="internal(2024/info/gypsum-nav)" raw="yes"]]
