path: root/2024/talks/gypsum.md

[[!meta title="Gypsum: my clone of Emacs and ELisp written in Scheme"]]
[[!meta copyright="Copyright © 2024 Ramin Honary"]]
[[!inline pages="internal(2024/info/gypsum-nav)" raw="yes"]]

<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->


# Gypsum: my clone of Emacs and ELisp written in Scheme
Ramin Honary (he/him)

 - Source code: <https://codeberg.org/ramin_hal9001/gypsum>
 - E-mail: <mailto:ramin.honary@gmail.com>
 - ActivityPub: [@ramin_hal9001@fe.disroot.org](https://fe.disroot.org/@ramin_hal9001)
 - Website: <https://tilde.town/~ramin_hal9001>

[[!inline pages="internal(2024/info/gypsum-before)" raw="yes"]]

## Slides

### Introduction

1.  Ramin Honary

    -   Emacs enthusiast since 2017

    -   Software developer (full stack)

    -   I love Haskell, Scheme, functional programming

    -   Started learning Scheme about 2 years ago

2.  My project: an Emacs Clone

    -   Tentative name: "Gypsum"
    -   Its not a great name, open to suggestions.

### Goal: to Clone **Emacs Lisp**

-   Many clones already:

    -   Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile

-   These only clone the key bindings, not Elisp

-   Only XEmacs (a fork of GNU Emacs) provided an alternative Emacs Lisp

### Most people don't use Emacs for the keybindings

-   Anecodtal, but yes really.

-   Use Emacs because of the power of Emacs Lisp

-   Emacs is as powerful as a system shell

-   A good language is what makes it powerful

### Goal: use R7RS Standard Scheme

-   I want it to work on a many Scheme implementations

-   Guile is the reference implementation

-   (more about this later)

### Goal: able to run any `init.el`

-   Should be able to use `init.el` without significant changes

-   Many invest significant time in their configs

-   Suddenly not having your config is disruptive

-   Such an Emacs clone would be more useful

### Why do this?

-   I personally like Scheme's minimalism.

-   Use Scheme as more than just an academic language.

-   Seems to be a lot of interest in a project like this.

-   Talk of "Guile Emacs" for about 30 years

### A long history of Guile Emacs (1/3)

- **Early 90s**: Initial discussion between RMS, Tom Lord,
  Aubrey Jaffer, begin work on replacing Emacs Lisp with Scheme.

- **1999--2009**: Ken Raeburn's [Guile-Based Emacs](https://www.mit.edu/~raeburn/guilemacs/).
  (My project is similar.)

  > "*This project that I (Ken Raeburn) have started is for converting*
  > *GNU Emacs to use Guile as its programming language. Support for*
  > *Emacs Lisp will continue to exist, of course, but it may be*
  > *through translation and/or interpretation; the Lisp engine itself*
  > *may no longer be the core of the program.*"

### A long history of Guile Emacs (2/3)

- **2010**: Andy Wingo and Ludovic Courtes
  take maintainership of Guile project.

- **2009--2011**: Emacs Lisp interpreter implemented in Guile.
  Still ships with Guile.

- **2011**: Guile 2.0 is released

- **2011--2015**: Robin Templeton's GSoC project.
  (Is presenting later today!)

### A long history of Guile Emacs (3/3)

- **2020**: Vasilij Schneidermann published an overview called
  "[The State of Emacs Lisp on Guile](https://emacsninja.com/posts/state-of-emacs-lisp-on-guile.html)".

- **2020 to present**: Guile Emacs is dead? Andrea Corallo, GCC Emacs,
  JIT-compiler for Emacs Lisp based on "libgccjit", brings into
  question any need for combining Guile with Emacs.

### Demo

### GUI is barely working

-   I have almost no experience with Gtk or GObject Introspection

-   Hard to debug, crashes at C-level produce no stack traces

-   Using GDB requires rebuilding all of Gtk, GIO, GLib, etc.

### Emacs Lisp parser based on Guile Emacs Lisp

-   Foked the Guile Emacs Lisp implementation for easier development

-   Have already submitted a patch to the parser upstream

### Emacs Lisp interpter is barely working

-   Implementing my own interpreter in portable Scheme

-   Monadic pattern matcher

### Can parse but not interpret "`subr.el`"

-   "`subr.el`" is the first ELisp file run by Emacs

-   A good way to determine what to work on first

### A call for help

### Latest Emacs has **1,393** built-in functions

-   I could never implement that many functions alone

-   Probably not all are required to create a useful editor

### My job is to make contributing easy

-   Document the build and test process

-   Document the system architecture

-   Prioritize which built-in functions are most essential

-   Find low-hanging fruit, use as means to teach others

### The work for which I will take responsibility

-   Clone enough Elisp to be able to run ERT tests

-   Then use GNU Emacs's own regression tests to test patches

-   Make sure there is a usable GUI

-   (Someday?) be able to contribute a patch from within

### Quick architectural overview

### The editor is based in Scheme, not Emacs Lisp

-   Config, scripting, packages all done in Scheme

-   Use of Emacs Lisp for scripting not encouraged

-   Should still be able to run your `init.el`

-   Ideally should be able to run ELPA packages

### Difference with Robin Templeton's project

-   Guile-Emacs links Guile runtime into Emacs

-   Not a Scheme application

-   An IDE for Schemers

### Emacs Lisp is an "environment"

-   "Environments" are a feature of Scheme

-   Scheme procedures can be called from Emacs Lisp

-   Scheme state can be mutated by Emacs Lisp

-   (See "`./gypsum/elisp-eval.scm`", "`new-env`")

### "Functional Lenses"

-   Because R7RS does not standardize MOP (not even in "large")

-   Inspired by Haskell

-   Composes getters and setters

-   Single source file, easy to port

-   Ported to 3 other Schemes

### A lot of work went into keymaps data structure

-   Keybindings are an important part of Emacs

-   Had to do this well from very beginning

-   Keybindings work correctly in demo

### A lot of work went into separating GUI from Editor logic

-   "Parameters" are a feature of Scheme

-   Platform-specific APIs are always parameterized

    -   Windowing and widgets

    -   Translate key events to bindings

    -   Evaluating Scheme expressions

    -   Text buffering and rendering

-   (See "`./gypsum/editor-impl.scm`")

### Monadic pattern matching

-   Simpler, more portable

-   (Not as feature-rich)

-   Easier than porting SRFI-241 ("Match") to Guile

-   No relation to SRFI-247 ("Syntatic Monads")

-   You can still use pattern matching

### Monad pattern matching

Example program


    (define push-stack (put-with cons))
    (define collatz
      (many
        push-stack
        (either
          (try (check (λ (n) (<= n 1)))
                      (success))
          (try (check odd?)
               (next (λ (n) (+ 1 (* 3 n)))))
          (try (check even?)
               (next (λ (n) (quotient n 2))))
          (fail "not an integer")
          )))

### Conclusion

-   I am just getting the ball rolling

-   Helping others contribute is my top priority

-   ActivityPub :: `ramin_hal9001@fe.disroot.org`

-   E-mail :: <span class="spurious-link"
    target="ramin.honary@gmail.com">*ramin.honary@gmail.com*</span>

-   Homepage :: <https://tilde.town/~ramin_hal9001>

-   Codeberg :: <https://codeberg.org/ramin_hal9001/gypsum>

-   This presentation :: <https://emacsconf.org/2024/talks/gypsum/>


## Original presentation proposal
I  would like  to demonstrate  an  Emacs clone  I have  been
writing  in Guile  Scheme  for  the past  year,  which I  am
tentatively  calling "Gypsum".  Unlike  other editors  which
only clone  the Emacs keybindings (Edwin,  Jed, jEdit, Jove,
Lem, MG,  Yi, Zile), I hope  my Emacs clone will  also fully
clone the  Emacs Lisp programming language  well enough that
many of the packages in ELPA, Non-GNU ELPA, and perhaps even
MELPA, can be used in  "Gypsum" without any modification.  I
would also  like to  talk a  little bit about  the how  I am
implementing  it  (the  software architecture),  and  invite
others to contribute.

I  think my  project  is  of interest  to  many Emacs  users
because, firstly, I have personally spoken with a relatively
large number of people who have expressed interest in making
Emacs  programmable in  Scheme.  Secondly, there  is a  good
amount   of  prior   art  for   Scheme  implementations   of
Emacs. There  are even  builds of Emacs  that link  to Guile
which  provides  a   "scheme-eval"  built-in  function  that
translates  between   Elisp  data  types  and   Scheme  data
types. The  Guile compiler itself  ships with an  Emacs Lisp
compiler as  well, although  it does  not provide  enough of
Emacs's built-in functions to be of much use.

So by  using Guile, we  can make use of  a lot of  the prior
art, in fact  I am currently using the  tokenizer and reader
used  in Guile's  built-in  Elisp  interpreter to  implement
"Gypsum's" Elisp interpreter. That said,  I have gone out of
my way to make my code fully R7RS compliant, so I hope I can
port  it to  other Scheme  implementations like  MIT Scheme,
Gambit, Stklos, and perhaps Chez Scheme with Gwen Weinholt's
R7-to-R6RS  translator.  I  consider the  Guile  version  of
Gypsum to  be the  reference implementation  of what  I hope
will become a fully  cross-platform programming language and
text editor written in portable R7RS Scheme.

The   reference  implementation   of  "Gypsum"   is  a   GUI
application   based   on   Gtk  using   a   library   called
"Guile-GI".   Guile-GI   uses  the   GObject   Introspection
framework to automatically generate Scheme language bindings
to libraries  like Gtk and Glib  which are written in  the C
programming language. There is not yet any terminal-emulator
version of "Gypsum."

The next step of the project  will be to implement enough of
Elisp that we can run  tests written in the Emacs Regression
Testing  (ERT)  framework.  We   can  then  incorporate  the
original GNU  Emacs regression  test suite into  Gypsum. Any
new API added to Gypsum  Elisp will most likely already have
regression tests we can use to  make sure it is working in a
way that is compatible with GNU  Emacs Lisp. I would like to
make it as easy as possible for people to contribute to this
project, and  having a list  of APIs to be  implemented each
with  a set  of regression  tests the  APIs are  expected to
pass, is a very good way to do that.

About the speaker:

My name is Ramin Honary, I have been a professional software
engineer  of  16  years,   lately  mostly  doing  full-stack
software  development. I  have always  been fascinated  with
programming languages,  and especially  functional languages
like   Lisp   and  Haskell.   I   have   been  using   Emacs
since 2017.  But lately it is  with Scheme that I  have been
spending  most  of  my  free   time.  I  am  only  a  Scheme
programming  enthusiast,  I  am  not  involved  with  Scheme
professionally.

You may also like another talk by this speaker:
[EmacsConf - 2022 - talks - Build a Zettelkasten with the Hyperbole Rolodex](https://emacsconf.org/2022/talks/rolodex/)

# Discussion

## Questions and answers

-   Q: Would it be possible to support a GUI toolkit other than GTK? Like how GNU Emacs still supports Lucid
    -   A: Yes this planed by having proper backend: emacs-lisp running
        into a module and the GUI being another module. So normalized
        communication. Currently GTK being standard implementation, also
        done here.
-   Q: Do you plan to provide improvements to Elisp as a language, or is the focus on a compatibility layer to facilitate doing all new extensions, etc. in Scheme?
    -   A: Plan is to keep up-to-date with new releases. So new GNU
        feature should be included with each release. But also intend to
        have support for pure Scheme features.
-   Q: If Emacs Lisp support for Guile was documented
    better, could you be nudged/convinced to (re)start using, and
    contributing to that?
    -   A: Compatibility is the most important things. Documentation not
        sufficient to convince users to switch.
    - IRC: Where do you think elisp documentation should be improved? I've always found the built in documentation to be excellent
      - janneke is referring to Guile's ELisp support, I believe; sorry, i meant the documentation of guile's elisp backend
-   Q: Why is being able to interpret all of \`init.el\` an useful goal?
    Sure, there is a lot of code written in elisp - can we consider a
    translator like utility to convert elisp to scheme, once guile-emacs
    becomes a reality? 
    -   A: Probably, but first step is getting the interpretter working.
        Emacs-lisp basically compiled down to intermediate
        representation of the guile compiler \[this was one of the hard
        things to get to work\]. But unclear how this works for other
        schemes. Best solution probably translation elisp -\> scheme,
        but this is not the approach that was done. Would be very cool
        to have. Feel free to give a PR.
-   Q: What is the plan to handle elisp packages that depend on 3rd party/external libraries? (libgit/magit or rg/ripgrep)? 
    -   A: Will be tricky. If loading directly into the elisp process,
        very hard. Cairo could help, but you need emacs lisp binding on
        top of that. For magit, you can call regular process
        communication via stdin/stdout; so you can reuse existing scheme
        libraries. Dynamic libraries not a goal. Rg/ripgrep probably the
        same with process communication.
-   Q: Why is it not feasible for the Emacs layer that interprets Emacs Lisp (the core in C) ot have a Scheme interpreter, instead of using Guile?
    -   A: Guile is a scheme. Not sure what you mean.
    -   A: Check presentation later of Robin Templeton ("Beguiling
        Emacs: Guile-Emacs relaunched!"): the attempt exists by
        translating elisp to guile. 
        -   thank you

- Q: Not really a question, but how about Schemacs as a name?

-   -   A: Cool name, but did not check if it is already used. Feel free
        to discuss by email.
-   Q: I'm curious to know how the hell guile-emacs deals with all of the dynamically scoped modules out there. Is there any effort to automatically modularize and namespace stuff?

## Notes

- oo neat, i didn't know about that first bit of history
- i've heard rms say that scheme (guile) is just a nicer
  lisp; but didn't know there were concrete talks/attempts to use
  guile for emacs that early
- robin: yes, guile-elisp not being portable might be a
  showstopper for ramin
- I've heard good things about guile from guix people. Never really tried it out
- FWIW, I think there have been various attempts to make an Emacs clone in Common Lisp. I guess lem is currently the most active. https://github.com/lem-project/lem/
- I like how he edited his slide mid-presentation.
- https://www.emacswiki.org/emacs/GuileEmacsHistory has some info on very efforts; i was surprised that there were so many (especially with old-school guile, eww ;))
- joining for all the guiles and all the emacsen
- He's got a long way to go.
- of course there were, RMS had decreed that guile was the GNU scripting language and it was a bit embarrassing that it was so little-used...
  - At least Guix uses it now.
  - oh yes, guile 3 was a great step forward, and I do wonder how much of that was due to the impetus of its having real users :)
- if you're interested in guile emacs, be sure to check out robin's guile talk this afternoon
- nice, my silly https://gitlab.com/janneke/guimax also used guile-gi but this looks much more mature
- developed actively til 2014, but more recent work is on a branch so may not be as obvious...
- so...i guess that some basic documentation on elisp may be very helpful
- wbn if you could join efforts somehow
  - there should definitely be some overlap between the projects
- I have 6500 interactive ones, according to Vertico...
  - I got 8021 interactive ones ;)
  - 7690 here
  - 34557 callables \o/
- working towards a similar goal approached from different directions
  - however, working on guile's elisp backend may be a common ground
- ramin's probably talking about subrs, i.e. primitives not themselves implemented in elisp
- you mean providing modularization for elisp programs? or something else
- Being more specific, removing the need to namespace every single internal variable/procedure in an elisp module. Maybe that isn't a goal, but I wish it were
  - yes, i have some ideas for adapting the CL package system for that. it'd have to be opt-in, maybe some tools for automated refactoring
- And... the part where we need more bandwidth for any core runtime efforts to be viable
- my-special-module--loop-variable-3
- How can I get involved with this? if I want to contribute
  - hang out in #guile-emacs and/or subscribe to the mailing lists https://guile-emacs.org/
- an embryonic re-implementation by ramin of emacs in guile, with their own new elisp interpreter that should be r7rs compatible
- would love robin's guile-emacs and ramin's efforts to somehow share some of their efforts
- Robin's talk mentions developing a better elisp in Scheme. Why can't your project leverage it?
- guile-elisp is part of guile's compiler system (elisp -> tree-il -> cps -> bytecode), unless another scheme sets out to be guile-compatible in that respect it won't be portable at all
- To be fair, I've been screwing around with chicken, guile, and racket. I haven't found any 2 scheme implementations to be compatible, even within SRFI implementations
  - Just the basic syntax and semantics, nothing else
  - yeah, scheme's lack of portable libraries in practice motivated me to suggest something that's sure to piss everyone off: Javascript
- YouTube comment: Cool, great, amazing. Extremely ambitious. A clone like this project might be significant harder than for example Lem, which does not care about backward compatibility with Emacs. A clone also is somewhat harder to create a unique selling point for. Especially if Guile-Emacs is also going to be a thing now. Just my 2 cents up to discussions. Anyway I am excited and will follow it:)

[[!inline pages="internal(2024/info/gypsum-after)" raw="yes"]]

[[!inline pages="internal(2024/info/gypsum-nav)" raw="yes"]]