path: root/2022/captions/emacsconf-2022-wayland--emacs-should-become-a-wayland-compositor--michael-bauer--main.vtt

WEBVTT captioned by bhavin192

NOTE Introduction

00:00.000 --> 00:08.000
Hello EmacsConf and hello fellow Emacs fans.

00:08.000 --> 00:10.920
My name is Michael Bauer, and I'm from Germany.

00:10.920 --> 00:00:13.440
I'm gonna talk to you about "Why and how Emacs

00:00:13.440 --> 00:00:17.200
should become a Wayland compositor."

00:17.200 --> 00:21.740
And it already kinda is a Wayland compositor.

00:21.740 --> 00:25.000
This talk is composed by Wayland and Emacs.

00:25.000 --> 00:00:26.840
If I'm talking about a Wayland compositor

00:00:26.840 --> 00:00:29.360
or Emacs as Wayland compositor,

00:00:29.360 --> 00:00:30.440
I mean it in the sense that

00:00:30.440 --> 00:00:36.000
EXWM is an X window manager. I hope you know EXWM.

NOTE Why

00:36.000 --> 00:41.000
So, why?

00:41.000 --> 00:00:44.120
Emacs can do Wayland now, that was a stopper

00:00:44.120 --> 00:00:49.000
before, and now it's solved with `pgtk` branch.

00:49.000 --> 00:00:53.240
It makes the Emacs toolbox bigger,

00:00:53.240 --> 00:00:55.000
which is always a good thing.

00:55.000 --> 00:00:58.440
And the cool thing about Wayland, which is not

00:00:58.440 --> 00:01:02.280
possible under X is, it can run standalone

00:01:02.280 --> 00:01:06.840
on the Linux kernel interface, or nested under X,

00:01:06.840 --> 00:01:07.120
or even nested under Wayland.

01:09.000 --> 00:01:13.840
The compositor features of Emacs doesn't mean

00:01:13.840 --> 00:01:16.960
it has to take over the whole output.

00:01:16.960 --> 00:01:20.560
It can use them, even if it's just like

00:01:20.560 --> 00:01:23.080
a normal window or normal program.

01:23.000 --> 00:01:25.120
And last reason is,

00:01:25.120 --> 00:01:27.240
I want to keep living inside Emacs

00:01:27.240 --> 00:01:31.000
and Wayland is the future, apparently.

NOTE EXWM use case

01:31.000 --> 01:35.000
EXWM use case is the first use case.

01:35.000 --> 00:01:38.400
You take a Wayland surface and put it inside

00:01:38.400 --> 00:01:41.520
an Emacs window. You see it right below.

00:01:41.520 --> 00:01:45.880
The video of me is a Wayland surface,

00:01:45.880 --> 00:01:50.000
and it's inside an Emacs window managed by Emacs.

01:50.000 --> 00:01:53.480
Emacs does the input, and the clipboard handling,

00:01:53.480 --> 00:01:59.000
and can insert itself here, and do great things.

01:59.000 --> 00:02:03.200
And it's a possibility to Lispify the Linux desktop,

00:02:03.200 --> 00:02:08.000
as Emacs Lispifies the command line.

NOTE XWidget use case

02:08.000 --> 00:02:12.960
The other use case is the XWidget use case.

00:02:12.960 --> 00:02:17.000
I don't know if you know XWidgets.

02:17.000 --> 00:02:19.720
It's embedded X windows inside Emacs.

00:02:19.720 --> 00:02:24.000
There's a web browser available in Emacs.

02:24.000 --> 00:02:27.520
With Wayland, you could embed anything that can

00:02:27.520 --> 00:02:32.000
create a Wayland surface like video, web, or 3D.

02:32.000 --> 00:02:34.880
Think OpenGL, something like

00:02:34.880 --> 00:02:38.000
EmacsGL would be possible.

02:38.000 --> 02:46.000
And we wouldn't have just images like we have so far.

NOTE How

02:46.000 --> 00:02:50.560
So, how to implement this Wayland compositor?

00:02:50.560 --> 00:02:52.560
I'm going to tell you how I did it,

00:02:52.560 --> 00:02:57.000
or I did this demo I'm showing you right now.

02:57.000 --> 00:03:00.680
First of all, how does Wayland work?

00:03:00.680 --> 00:03:04.000
Wayland is a protocol in XML.

03:04.000 --> 00:03:11.120
It's a server and client, and they share a set of

00:03:11.120 --> 00:03:13.960
objects, and the objects have methods.

00:03:13.960 --> 00:03:16.120
They are specified in the protocol,

00:03:16.120 --> 00:03:24.080
and Wayland also says how the server

00:03:24.080 --> 00:03:25.720
and client talk to each other.

00:03:25.720 --> 00:03:33.440
First blocker for Emacs becoming a Wayland

00:03:33.440 --> 00:03:37.000
compositor is that Emacs and Wayland both have

00:03:37.000 --> 00:03:41.880
their own event loop, and you can't merge them too.

00:03:41.880 --> 00:03:45.800
But you don't have to merge them

00:03:45.800 --> 00:03:48.000
because you can just make Emacs speak Wayland.

03:48.000 --> 00:03:50.040
So, Emacs becomes a Wayland client,

00:03:50.040 --> 00:03:53.000
and there's an extra server Emacs is talking to.

03:53.000 --> 00:03:59.160
So, we need a minimal Wayland server that does all

00:03:59.160 --> 00:04:03.000
the stuff Emacs can't do and do the rest in Emacs.

04:03.000 --> 00:04:07.800
---The minimal Wayland server, I did it in wlroots.

00:04:07.800 --> 00:04:13.040
That's the library behind Sway. I think it's

00:04:13.040 --> 00:04:20.000
the Wayland library to do stuff like this.

04:20.000 --> 04:26.000
I implemented four different things to make it work.

04:26.000 --> 00:04:30.680
It's these three letter acronyms on the left.

00:04:30.680 --> 00:04:34.520
It's Emacs, Wayland, and then it's a server,

00:04:34.520 --> 00:04:38.000
a client, a protocol, and buffers.

04:38.000 --> 04:44.000
The server is written in C and it's mostly tinywl.

04:44.000 --> 00:04:46.280
It's the example of wlroots,

00:04:46.280 --> 00:04:52.000
and it's around 1000 lines of code.

04:52.000 --> 00:04:54.960
ewc, the Wayland client in Emacs,

00:04:54.960 --> 00:04:58.560
is the thing I'm most proud of.

00:04:58.560 --> 00:05:02.120
It's 300 lines of code, and it is a

00:05:02.120 --> 00:05:08.000
fully featured Wayland client in Emacs.

05:08.000 --> 00:05:11.640
With this, Emacs can speak Wayland,

00:05:11.640 --> 00:05:18.000
and then I implemented Emacs Wayland protocol.

05:18.000 --> 00:05:21.280
It more or less allows Emacs to become a Wayland

00:05:21.280 --> 00:05:24.640
window manager, so it's not actually the compositor.

00:05:24.640 --> 00:05:27.680
The compositor stays in C, but Emacs is

00:05:27.680 --> 00:05:31.000
now a Wayland window manager!

05:31.000 --> 00:05:34.520
And the last thing is Emacs Wayland buffers.

00:05:34.520 --> 00:05:35.880
It's the window manager part.

00:05:35.880 --> 00:05:38.440
It's around 500 lines of code,

00:05:38.440 --> 00:05:41.680
and it does the buffer management inside

00:05:41.680 --> 00:05:45.680
Emacs windows, or floating right like you see me

00:05:45.680 --> 00:05:48.000
now floating on the right.

05:48.000 --> 00:05:51.000
It works, but it is still buggy,

00:05:51.000 --> 00:05:54.320
and it is also missing input handling,

00:05:54.320 --> 00:06:01.000
so there's more code to come for this to work.

NOTE Caveats

06:01.000 --> 06:05.000
Some caveats about this approach.

06:05.000 --> 00:06:09.640
wlroots is around 60 kilo LoCs (Line of Code)

00:06:09.640 --> 00:06:12.000
and in active development.

06:12.000 --> 00:06:16.640
They have like a slogan 60 kilo locs of code

00:06:16.640 --> 00:06:19.760
you had to write anyway to make a Wayland

00:06:19.760 --> 00:06:22.520
compositor. And no, you don't have to write it.

00:06:22.520 --> 00:06:25.840
But I still remember when it was like 50 kilo locs,

00:06:25.840 --> 00:06:29.680
and now it's 60. And it's like a moving target.

00:06:29.680 --> 00:06:32.480
I think it could be quite a lot of work

00:06:32.480 --> 00:06:34.000
to keep up with it.

06:34.000 --> 06:41.000
Yeah, it could be quite a bit of work.

06:41.000 --> 06:46.000
Some windows don't like to keep the aspect ratios.

06:46.000 --> 00:06:49.560
You tell them and you have to crop them.

00:06:49.560 --> 00:06:53.000
And the interface I use in wlroots for doing this,

06:53.000 --> 00:06:57.280
`wlr_scene`, can't do cropping yet,

00:06:57.280 --> 00:07:01.000
so this doesn't work.

07:01.000 --> 00:07:03.240
Another problem is with GTK.

00:07:03.240 --> 00:07:03.280
Once Wayland is enabled and it stays on.

07:11.000 --> 00:07:12.560
This doesn't make sense.

00:07:12.560 --> 00:07:15.000
Okay, if you kill the Wayland server,

00:07:15.000 --> 00:07:18.000
GTK kills Emacs, that's not a good thing.

07:18.000 --> 00:07:21.000
And it's still a bit of work and fussing needed

00:07:21.000 --> 00:07:23.640
to get this to work reliably.

00:07:23.640 --> 00:07:26.000
It's quite buggy right now.

NOTE Call to action

07:26.000 --> 07:30.000
And that brings me to my call to action.

07:30.000 --> 00:07:34.440
I think making Emacs Wayland capable is

00:07:34.440 --> 00:07:39.000
a further step to make an Emacs OS.

07:39.000 --> 00:07:41.360
It gains output and input handling.

00:07:41.360 --> 00:07:44.120
Output handling is already there,

00:07:44.120 --> 00:07:45.760
input handling is still missing,

00:07:45.760 --> 00:07:49.720
but Emacs can manage monitors, outputs,

00:07:49.720 --> 00:07:53.040
different frames if it's like nested,

00:07:53.040 --> 00:07:57.080
And inputs, keyboards, simulation keys,

00:07:57.080 --> 00:07:58.000
stuff like that.

07:58.000 --> 08:02.000
We could use it in more ways for Emacs display, maybe.

08:02.000 --> 00:08:05.720
Wayland just manages simple pixel buffers,

00:08:05.720 --> 00:08:09.000
so it's a protocol for managing pixel buffers.

08:09.000 --> 00:08:12.560
And in a sense, we could go back to

00:08:12.560 --> 00:08:15.560
the old X ways and maybe even ditch GTK.

00:08:15.560 --> 00:08:18.000
I don't know, but why need it?

08:18.000 --> 00:08:20.600
We can composite without it.

00:08:20.600 --> 00:08:27.000
Let's make buffer menus, buffer world, buffer.

08:27.000 --> 00:08:29.040
Emacs Wayland protocol, like I did it,

00:08:29.040 --> 00:08:34.040
allows a very concise design, and it allows

00:08:34.040 --> 00:08:38.000
to improve on the EXWM code base.

08:38.000 --> 00:08:41.840
And I wrote KISS style because EXWM has

00:08:41.840 --> 00:08:45.000
workspace management integrated.

08:45.000 --> 00:08:50.600
I don't think that's needed, like Emacs does it.

00:08:50.600 --> 00:08:53.640
Why do you have to do something extra?

00:08:53.640 --> 00:08:56.000
So why do it?

08:56.000 --> 00:09:00.040
To finish the call to action,

00:09:00.040 --> 00:09:03.280
if this is the thing you want to see in Emacs,

00:09:03.280 --> 00:09:06.600
maybe you want to get involved, have some ideas,

00:09:06.600 --> 00:09:10.000
so we could discuss it.

09:10.000 --> 00:09:14.960
I'm looking forward to discuss with you

00:09:14.960 --> 00:09:21.000
and hear your questions and ideas.

09:21.000 --> 00:09:24.200
I want to say a big thank you to the

00:09:24.200 --> 00:09:26.600
organizers of EmacsConf and the other speakers

00:09:26.600 --> 00:09:29.000
for making this event possible.

09:29.000 --> 09:39.000
Thank you, and see you.