Okay, so I'm going to look at some of the questions showing up
in the etherpad we got here.
It says, have you considered
using a CRDT to share buffers between threads and merge any
concurrent edits automatically? So I have looked at that.
And the problem with CRDTs is that even though they give you a
mathematically correct answer when you're trying to merge
two conflicts, it's not always a useful answer. Like, it's
not coherent. If you have two things trying to edit the same
thing, there's no good way to resolve that. And so they
really work well when you have two people working live, both
editing the same document, because they can fix any
particular issues like that, like you would with Google
Docs. But you have different packages that aren't aware of
each other, and you're going to run into problems. And so
this is something, if you read from the Xi editor, which was
one of the first ones to use CRDTs, in the retrospective, he
talks about how they had this problem, where the CRDTs They
give you an answer, but it's not always an answer that's
useful. And so I feel like locks at least are going to make it.
It's not going to be as efficient if you have a whole bunch of
packages, but I don't imagine there's going to be a ton of
those. It can actually, I think it'll be more useful in
practice.
I host on GitHub because that's what I know.
If there's a way to host it on somewhere else, I'd be
interested in doing that. If you're interested in setting
patches without using GitHub, you could always send an
email. I'm more than happy to accept email patches.
Do you think it's possible to achieve with the current Emacs
code? I do. I think, I think you can do that. Um, like I said,
there's a couple things inside there that are intentional
breaks with existing Emacs code. And some of those being
like functions are immutable. As well as having data shared
switch between different threads, which means there's
going to be some copying going on. So there's going to be
subtle things that are going to be different. And we've
really got to think about those intentionally, but I'm
really going for bug compatibility with GNU Emacs so that
you can take an existing Elisp package and just run it and it
just works, 'cause I think that's one of the big strengths of
the Emacs ecosystem is the millions of lines of Elisp that
people have written.
Um, So I'm not, okay...
So since you're re-implementing Elisp and
Rust, have you considered using more modern Lisp such as
Scheme? So I'm not re-implementing Elisp and Rust. I'm
re-implementing the C in Rust. In fact, I would like to make
more of the core that's written in C in Elisp instead of C or
Rust, because then it's actually introspectible. There's
a talk by Tom Tromney that he gave a while ago about Emacs
should be Emacs Lisp. I kind of like that philosophy,
that as much of it as should be Elisp as possible, and we
should only have C or Rust or some systems level language for
really low-level stuff. Using a more modern LISP such as
Scheme. I know there's, I mean, there's two talks, I think,
in this conference about using Scheme inside Emacs. And I
looked at this at one point about what if you wrote it inside
Common LISP, because that's also has some really low level
details. And then you could go from Elisp down to Common
Lisp. But the problem is, is under the hood, you still need a
systems language. You can't, you still need either C or Rust
or something underneath the Common Lisp to implement the
primitives. And so it's not going to give you just two
languages, you know, you'll have three. You'll have the
elisp, common lisp, and C under the hood. And so in this case
we just have the two. We have the Elisp and the rest.
All right that's all the questions I see there. Let
me go look at... Okay,
so I see into the chat.
Does it features from the Rust compiler that are missing
that way you would take advantage of? Oh, that is a great
question. Um, there's a handful of them. Uh, I should've
written down a list of these. One of them is Polonius, which
is the new borrow checker because we're trying to be used
lifetime to track our objects. We often run into situations
where we've kind of got a hack around things because the
limitations with the borrow checker. And so I have a whole
bunch of like notes inside there about where. A better
Polonius would help inside there and help ease some of the
issues. Another thing is enum variances types, because
right now we have an object which is defined as a big enum that
had all the possible objects, but if we want to have a subset
of those objects or just pass in one of those objects, we've
got to define a new struct. And so we have a whole bunch of
boilerplate code to define that all out. And if we had
variances types, that would make the code a lot easier.
Another one is the allocator API. Right now we're kind of
working around it, but ultimately we're going to need our
own allocator. And the allocator API is still nightly only.
So there's a couple more. I'll look at that more, but that's a
great question.
Let's see.
Okay. I see a question you might not have noticed just asking
about reuse of Remacs. Oh, yes, so I have reused some of Remacs
code, and some taken, like I mentioned, taken some of their
ideas, but ultimately we're using a different model
because under the hood in Remacs, everything is just defined
as an opaque external type that's defined inside Emacs and
so it just pulls those in interacts with those and passes
them back into C. We're trying to see what we can do if we
say okay we're not going to take the same... So they're bound to
the implementation details of Emacs, and we don't want to do
that. We've re-implemented all the core types
ourselves. So that means that we can't just take the
Remacs code one for one and use it in our project, but we can
take a lot of their ideas. I've spent some time reading
through their documentation, different things about how
they approached strings and GC
and different stuff like that.
Looks like all the questions.
Okay, so another question. What are your thoughts on the GUI
layer? Any plans on how to reimplement it? This is something
I've thought a lot about, but I still don't have a solid plan
for. I'm not really a GUI person. I mostly work with
low-level. And so my two thoughts is you can go the GTK route.
There's Rust bindings for that. That's well understood.
It's got a good support. But there's also some interesting
projects to try and do GUI in Rust, native Rust, and have it
use Rust idioms and stuff like that. And so those are things
like Druid and there's eGUI and a bunch of those that are that
way. And I've never used one of them, but I'd be interested to
try that out first and see how well does this work and how well
supported is this doing a Rust-first GUI.
If money could fix the problem, how much would it cost to ship
this with feature parity before 2026? Ooh, in a year. Uh,
that's a good question. Even if we had the money, it would
take more than just me, even if I was working on it full time.
Um, I don't know. That's a good question.
I would think it would take a team
of at least a handful of people to get this
shipping within one year. Because there's still a lot of
work to do. And even once you have everything implemented,
there's a lot of bug finding and smoothing it out so that it
runs as well as Emacs, which has been, you know, battle
tested for a long time.
Um, okay, so this might be a good moment for me to break in and
just say that we've got about 10 minutes left before we
resume new talks on on both tracks. Of course, we're happy to
keep this. This chat open and keep the recording going here,
which will share after the conference as long as as long as
there's discussion here. Thank you.
GTK project has
automatic binding with a framework called GObject
introspection, which is what I'm using for gypsum project.
Probably Rust has a G object implementation, which you can
use. Yeah, I know it has some GTK bindings. Um, I'm not sure if
it's specifically GObject, but that's a good path
because I feel like the problem
with the Rust GUIs is that they're all
very new. And so, you know, everything works in a demo, but
you need something that can work across all different
devices and all different platforms and have really good
support and good accessibility and stuff like that.
All right, Elisp is implemented in C. So if you're not
implementing Elisp in Rust, we're keeping the C
implementation of Elisp. So let me see if I can do a better job
of explaining this. So inside Emacs, you have about a
million lines of Elisp. And underneath that, you have the C,
which is the primitives everything's implemented yet. And
so we're keeping all of the Elisp, and we're just taking that
C layer and replacing it with Rust. And so when you call a
built-in function, it's calling into a Rust function
instead of a C function under the hood. So all the Elisp stays
Elisp, but the C becomes Rust.
So looking at the IRC chat, it feels to me like maybe there's a
little bit of confusion around what do we mean when we say
rewriting Elisp in Rust, right? I think there are some
people that are like, A, we're reinventing ELISP, and
there's other people that are like, no, we're trying to be
byte-for-byte compatible with Rust.
So some people are questioning your no answer on that.
Aren't you really, maybe, is what I'm reading in there.
Do you want to respond to that?
Yeah, I'm trying to think about how I can make this
clearer. So the Elisp stays Elisp. We're not changing the
Elisp at all, or at least very minimally. We want to be able to
take, like I said, bug compatible. So whatever works inside
existing Emacs, you can take all the Lisp and you can run it in
Rune and it works the same. So the Elisp stays the same. It's
just the under the hood core that is getting replaced. And
this in turn adds some new features such as
multi-threading. So it's not exactly compatible, but
you should be able to use your existing code and the Elisp will
stay Elisp. So the idea is that anything that I've written as
an Emacs user, my config, my custom packages, whatever it
may be, that's all going to be valid code. If you take, you
know, the Elisp implementation as being the C code
and the parts of Elisp written in Elisp
that represent opportunity space
thinking about Rust as an implementation language...
Okay fine. You know, you can make a semantic argument, okay
we're re-implementing, we're creating
an alternate implementation of Elisp
but what Elisp is isn't the problem
space here. That's a fixed, a given, if you will.
Is that all right?
That's a good way of saying it. Okay. Yeah, what you
said makes sense. I was kind of responding to some comments,
like I'm not sure it connected for everybody. Makes a lot of
sense. Yeah, I wasn't sure how much I needed to expand on that
and explain that, but I appreciate you jumping in.
Um, okay. So if I were just going to, but...
Will your Rust implementation also be able to run Emacs bytecode or the
implemented at the Lisp level? So I already have a bytecode
interpreter inside there that runs the existing Elisp
bytecode. And so that was one of the first things I did was
bootstrap the interpreter and then bootstrap the bytecode
engine. And so we compile, we use the... because the bytecode
compiler is written in Emacs Lisp. So you bootstrap that
and it gives you the Emacs bytecode. I have a bytecode
engine that runs the bytecode. So that's already done. And
you can potentially, on top of that, do something like the
data compilation or a JIT. But we have both an
interpreter and a bytecode compiler.
And I'll just break in one more time to say with about five
minutes left in our live time with this Q&A session, which
we're happy to keep going as long as there are questions.
Coming up in five minutes, we'll have a talk on color on the
gen track. And then right here, we'll have the p-search
talk.
Thank you.
Is it possible to bootstrap with just the bytecode
interpreter? So I'll have to put in a link to one of my blog
posts. So that was my original idea was to say, I don't want to
have an interpreter, a bytecode interpreter and a native
compiler. I want to just have just one.
So I'm only going to have the bytecode.
And so that's what I did initially.
The problem with that is, is that a bunch of the early bootstrap
Emacs code is written with the assumption
that it's going to be interpreted.
This is especially true with macros,
where you'll have a function defined,
you'll evaluate part of the function.
The other half of the function has macros in it
that are not defined yet, but it doesn't matter because they
don't get used. But with the bytecode interpreter, it
expands all macros when it gets the function definition.
And so those weren't macros when the function was expanded,
and therefore they got instantiated as functions, but
they're not functions, they're macros. And so I initially
spent a bunch of time trying to work around this, trying to
move code around, work stuff around, refactor the code to
try and get it to work with only bytecode interpreter. And
eventually I just gave up. I said, you know what, I'm just
going to write an actual interpreter to handle this because
trying to handle all these lazy macros is too much work. And
everything in the bootstrap is built with the assumption
that you have lazy macro expansion.
I'm guessing the Emacs bytecode interpreter isn't
complete.
So it's mostly complete. There's a handful of opcodes that
aren't implemented that are pretty easy to add that I
haven't run into. And there's some of them that are
deprecated that aren't implemented, but it's essentially
complete.
We also provide a bytecode JIT compilation via libgcc the
way Emacs currently does it. Eventually I would like to...
I'm more inclined to have a proper runtime JIT than an
ahead-of-time compiler like libgcc, like the current
Emacs native compilation, because it allows you to take
type information and actually apply that to the code, which
can let you do more aggressive optimizations to it.
He said, we may either get a new Emacs with an ancient C core
with a modern Lisp, or an Emacs with modern core, but stuck
with ancient Elisp. So there was another project I was
talking to, one of the Guile implementations, about how we
could potentially, he was like, what would it take to
bootstrap Guile in Rune, where you have both, you could have
Elisp and Guile running inside this project. And so we've
started that discussion, which I think would be, which
would be interesting. But it's tricky too, because
fundamentally Elisp and Guile are two different
languages. They have different semantics. They have
different ways of handling things. You've really got to
consider both of those when you're trying to make them work
together.
How would you do the native module system? What would be
different? We can do the same thing. We have an FFI. So I
haven't looked into it a ton, but I feel like it could be
similar. And I'm actually interested, there's a couple
projects on GitHub right now to have an FFI written in Elisp.
So you don't even need to create a separate C or Rust module,
because you can actually write native modules in Rust or C.
And so you can just have direct bindings to a C FFI written in
Elisp. You don't need any C code inside there. And I think
that would be an interesting approach to look at as well.
Oh, Ramin. Yeah, that's right. We were talking about that,
about bootstrapping Scheme.
And at this point, we have broken away from, uh, from this
talk, but we're continuing to record and this will all be
published. I'd say, go ahead and keep going as long as you'd
like in here. And, um, thanks once again for the awesome
discussion. Thank you so much. So I'll just pay attention to
the ether pad and the chat and see.
That sounds good. I'll keep an eye on IRC. And if there are
more questions here, I'll bounce people toward the ether
pad or this, uh, chat room. All right, thank you.