Hello.
Hopefully everyone is staying safe and
staying home,
I feel very grateful to live in a world
today that technology and free software
can be leveraged to connect people in
such disconnected and difficult times,
and to have an online conference like
this. Hopefully you've all
enjoyed this year's EmacsConf so far.
Many thanks to all the people that made
this possible.
Anyways, welcome to my talk "Extend Emacs
to Modern GUI
Applications with EAF, the Emacs
Application Framework".
This will be my first ever talk, so
apologies for my
inexperience, let us begin.
About me: my name is Matthew Zeng, you can
also call me MT
or Mingde. I'm a Chinese Canadian living
in Toronto,
Ontario. Offline: I'm an undergrad
studying mathematics at the University
of Waterloo.
Online: I'm one of the admins of the
Emacs China
— the largest Emacs forum in China. So,
to all Chinese listening to my talk right
now, feel free to check it out.
And this is a link to my GitHub profile,
(and) to my projects I'm involved in.
One's M-EMACS which is
I'm the author of — a user-friendly
full-featured Emacs configuration
distribution,
it is what I'm using right now, as well
as the Emacs Application Framework
which I help to maintain along with the
other author
lazycat, which of course, is today's
topic.
So, as you all might have already
noticed I'm currently using Emacs
and opening navigating closing
all these websites that are rendered
properly
all within Emacs, it's all thanks to the
EAF project.
So, we're living in a society that's
heavily dependent on the internet
and multimedia, it is unavoidable to run
to some occasion that you need to
open a fancy website that uses
JavaScript and CSS,
or you need to watch some videos. However,
due to the nature and history of Emacs,
it cannot
render all these modern graphics
effectively and efficiently.
Emacs is solely a text-based editing
environment,
and I argue that this is not a bad thing,
in fact, it is one of the reasons that me
and I believe many of you as well
are attracted to Emacs in the first
place.
Unfortunately, this results in us having
to open a dedicated web browser to
browse the internet,
open a dedicated video player to watch
some videos, or a PDF renderer to read some
documents.
So far Emacs cannot do all these tasks on
its own
but can only be achieved using other
external applications.
So, the other author manateelazycat,
or lazycat in short, didn't want to use
all these external applications,
he wanted to have an uninterrupted Emacs
experience,
he wanted to truly live in Emacs.
However, it would be a lot of work to
build this
modern application from scratch, there's
simply no time or research to do that.
So, lazycat thought of utilizing existing
applications
and to try to make it collaborate with
Emacs, there are many solutions available,
one of it
is the Emacs X Windows Manager, and I'm
sure a lot of you already know that —
the EXWM. However, it didn't work for him,
because although EXWM opens the
door to use other applications within
Emacs,
it as a fine window manager cannot
modify,
customize, or extend other software from
Emacs.
For example, it cannot modify the
behavior when you press a key in
Chromium or
PDF viewer, therefore it cannot utilize
the rich Emacs ecosystem that's been
growing for almost 40 years.
On the other hand, in the EAF browser, so,
if you M-x eaf-open-browser-with-history,
you can see
on the lower half of my screen — a list of
histories sorted by my personal
most visited sites, and you can search
for a site that you've been
to or search for some keyword in a
search engine.
So, this is all achieved by utilizing the
popular completion framework in the
Emacs ecosystem — ivy.
So, lazycat decided to develop a
solution of his own in 2018,
namely the EAF project, so, I joined the
development last year, 2019.
EAF is
a highly customizable and extensible
GUI application framework that extends
Emacs to graphical capabilities using
PyQt5, and it is not a window manager.
Alright. So, in the README, you can see a
list of GIFs
showcasing all the available EAF
applications,
a browser, a markdown previewer, a video
player, a PDF viewer, and more.
Today I don't have
enough time to demonstrate each one of
them,
but I will select a couple applications
to show you.
So, since we are already using EAF browser,
we'll start with this. Besides using the
classic Control n (C-n), Control p (C-p)
you can also use the Vim style hjkl to
move up or down.
Also, Meta Shift comma (M-<) or g (moves) to the
beginning of page,
Meta Shift period (M->) or capital g (moves) to
the end of page.
Vimium and Surfingkeys
are popular keyboard-based
browsing techniques in Chrome,
and they've imported here as well. You
can press f to toggle markers pointing
to
all the links in the current page, say, I
want to visit the wiki —
which comes very very handy when you
want to configure EAF to your liking,
so you see the marker on top of wiki is
dd,
press dd and Enter (RET), and now
you are navigated to this link, so you
don't need to use your mouse at all.
So, a full list of key bindings can be
found when you (press)
Control h m (C-h m), just as any other Emacs major
mode,
so you don't have to remember everything…
all the key bindings I said to you.
So, this is a global binding application
to every other EAF application as well.
You can find it under the wiki in the
keybindings
section, so press f again and use
ns, press Enter (RET), now you're in the
keybindings web page.
You can see all of the keybindings
available in every
EAF application, and you can try them out,
and you can customize your key bindings
using eaf-bind-key,
you can customize Control n (C-n) as
in the web page
to scroll up in the EAF PDF viewer, or you
can unbind
an existing binding using
eaf-bind-key,
bind it to nil, so it doesn't bind to
anything.
Okay, so, here comes the important part,
if you want to customize EAF, you should
visit the customization page in the wiki.
Now, I press Meta b to go back in
history, and
go to the customization page, press f,
press ad,
Enter, and now we're in the customization
page.
So, the first customization option you
see is dark mode,
let's say, if you want to turn on the dark
mode for EAF browser,
and you don't want to use your mouse to
do all this stuff.
You press c, and you can select
c to toggle the caret browsing, you can
see a lot of markers available,
poped up again, but they're not
on top of links but instead of
paragraphs.
You select the paragraph of your choice,
in this case you want
ls which comes here,
and then you just move the
cursor like what you always do
in Emacs, and now you select everything
and use Meta w (M-w) to copy the text.
Now, we (run) Meta Shift colon (M-:) to evaluate
what we just copied,
and set that to true, and
press r or F5 to refresh the page,
voilà we have the dark mode enabled.
So, there are…, well, let's toggle
it back off for now.
Now, we (run) Meta Shift colon (M-:) again,
and we find the one we just used, and
change it back to false,
and refresh the page, back in the light
mode.
So, there are many other customization
options available, you can either
evaluate like what we just did or add it
to your Emacs configuration file.
So, in this wiki…,
you can make the EAF browser to
continue where you left off
similar to the Chromium setting.
You can make EAF the default browser
(in) Emacs by
aliasing browse-web to
eaf-open-browser,
or set the browse-url-browser-function to
eaf-open-browser, there's just some tricks.
And there (is) also
an experimental adblocker currently
in place, therefore it can
block some elements but not all, so
we really encourage people to help us
test out and
add more conditions in.
So, you can…, the EAF Browser is able to
download
any files from the internet, and it will
be downloaded using Aria2.
You can also customize
the eaf-browser-download-path
using eaf-setq, it's a function that we
defined similar to setq,
the normal setq we know.
So, by default the download file is
stored in your home directory slash
downloads,
and you can change that whenever you
want. You can also disable
saving browsing history, so, remember
when I press…,
when I use M-x eaf-open-browser's
history, I see all the
histories here, but if you want more
privacy, you don't want that to be
available at all. You can turn it off
easily with eaf-setq,
and set that remember-history to false.
You can also set your default search
engine.
Right now we have Google, although
not really good but… Google and also
DuckDuckGo which is a better search engine,
well, yeah, ethically better search
engine.
So, you can also configure
the zoom. The default zoom of your
browser is 1.0, you can convert
default-zoom to 1.25, so when
you open any web page, it will be
zoomed by default. You can
also disable JavaScript, although I
personally don't really suggest you to
do, because it will
basically break a lot of our features,
because a lot of the browser
related features
must be implemented using JavaScript,
but yeah, you can do it if you really
want to. And there's also some
customization on EAF Camera
you can do as well.
Let's move on to EAF PDF Viewer.
Now, let's open
the PDF file using EAF.
So, that's one something already here, but
let's open it here.
So, eaf-open, and
select "Introduction to Programming in
Emacs Lisp".
I have it already open, but it's okay.
So, you have the file,
you have other files displayed…
you have all the pages display, sorry.
There are 273 pages in total, but notice
how fast it is to browse
all the pages, it is blazingly fast,
that's all thanks to Python and
MuPDF which you don't really get from
Emacs Lisp.
So, let's say if I want to jump to page
50. We press p and Enter 50.
And here we are, we are at page 50.
You can look at the lower right to
verify the page you're on.
You can use i to toggle dark mode
as expected. Let's say you want to
find
table of contents, so use Control s —
the Emacs default binding for I-search,
and search for a "table of contents",
here we are, it is highlighted for you,
and you can Control s for more but
there's only one match,
you (press) Control g (C-g) to
disable the highlight,
and you see a lot of options for you to
go.
Okay. Let's say, if you want to go to the
preface.
That is, you press f which is
also similar to
EAF browser, you press f for Vimium,
and you see the marker, now change to wn,
press wn, and then you can go to the
preface.
Now, we are at the preface.
So, now you finish reading, you want to
save your progress?
No worries, it is already saved for you
by EAF. You can safely
close the document using x,
and opening again, eaf-open,
and the file…, see you are at
preface again. You're right at where
you left of.
You can also use M-x org-store-link, or
Control c l (C-c l) —
which I prefer, if you want to save a
particular page in
a Org mode file. Now,
I go back to my presentation doc, I don't
need this anymore.
So, you just (press) Control c Control l (C-c C-l),
or I think M-x org-insert-link.
You can find the file right here, and
you press Enter (RET),
and you press Enter (RET) for the description
again, and now it's right here,
and Control c Control o (C-c C-o) to open it. Voilà!
You're back.
Let's now demonstrate the EAF Video
Player.
So, M-x eaf-open, you use eaf-open whenever
you want to
open some file. You use eaf-open-browser
if you want to use some
actual application that's not really
related to a file.
So, eaf-open, and select the video
you want, so,
video-demo, I already have a video
demo
ready, because I recorded a video of
the demo
of the EAF Camera, have a look.
Let's move to the beginning, "Hello people
from the future!
This is a demo of the EAF Video Player
that demos the EAF Camera feature,
so, as you can see on the screen of me
inside my camera,
and the screen is actually with all
within Emacs.
(Right, the video itself is as well, haha.)
You can open this
using eaf-open-camera
which I'm already into, and you can
press
p to capture a photo,
so, the photo is by default stored at
your $HOME/Downloads directory,
and you can modify it
freely. If you go here, and you can see
the camera stored
right here."
So, you press Space (SPC) to pause, what I used
here is
the eaf-open-this-from-dired.
Basically, in dired
you select
the file that should be opened by
EAF, and I used that. It detects that
it wants to use the EAF Image Viewer, so
I accidentally tested EAF Image Viewer
before I noticed. That gives the
image of the photo I just took
using EAF Camera. As you can see, you
can
use hl — the Vim binding to navigate
in the timestamp in the video,
and I can use
jk to change the volumes of the video.
Alright. Now, you've seen all the basic
usages
of the EAF project, it comes the question
of what is the magic behind it.
All right. Let's open the hacking page in
the wiki,
the design is laid out in a diagram here.
Let's put it
side by side along with my text, so
we can follow through.
Right, okay. Let me…,
sorry, let me drink some water.
This page in the wiki went into a lot
of detail,
due to the time constraint I will just
rephrase some of the ideas here, so for
anyone interested, please have a look at
the wiki yourself.
The easiest way to think about EAF is
that
the actual GUI application is started in
the background,
then the frame of the application is
attached to the appropriate location on
the
Emacs window. So,
EAF linked Qt5 with Emacs using
Elisp and Python.
On the Python side which is colored
yellow in the image,
we have QGraphicsView and
QGraphicsScene objects.
These are used to simulate the Emacs
window buffer design
where QGraphicsScene is similar to
buffers in Emacs,
it controls the state and the content
details of the application
where QGraphicsView is similar to Emacs
window. It populates the buffer
(QGraphicsScene) to the foreground at
the appropriate
position.
Whenever an EAF mode buffer
brings to a background….
Whenever an EAF mode buffer brings to the
foreground, sorry,
a QGraphicsView instance is
created, and whenever the buffer goes to
the background
the QGraphicsView instance is then
deleted,
while QGraphicsScene — the actual
process — remains running in the
background until the EAF mode buffer is
killed.
GPU compositing is used to ensure that
QGraphicsView and
QGraphicsScene is synchronized real time.
Using QWindow::setParent function
the QGraphicsView is attached to the
appropriate location on the Emacs frame,
so that although GUI applications are
not running within Emacs,
they look as if they were.
When user types on the keyboard it is
first received by the Emacs
EAF mode buffer, and then Elisp sends
the event to QGraphicsScene using
D-Bus.
When user clicks on the GUI application
it is received by the QGraphicsView
and processed in Python. Elisp can
communicate with Python through D-Bus,
in other words you can
customize and extend Emacs not just
using
Elisp, and now you can use Python, this way
one can leverage
all the Python properties like
multi-threading or some other stuff,
the entire Python ecosystem can be
utilized as well,
such as the Qt web engine that is the
basis for
our EAF Browser, and PyMuPDF is the
basis for the EAF PDF Viewer.
This really opens the window to many
many new possibilities to extend Emacs
using EAF.
All right, back here. We are always
looking for people to join the
development, there are many many
more work that needs to be done,
such as testing and debug EAF on
more Linux distros and window managers
such as i3
and stuff, you can also add new EAF
applications,
or debug and enhance existing EAF
applications,
or you can port EAF to native Wayland
which I just discussed with
the emacs-webkit author
Akira Kyle, and he told me that
EAF doesn't really work on
native Wayland, because it uses
XWayland, so it doesn't work on the pgtk
port of Emacs.
And we also need people to port EAF to
non-free operating systems
including Windows and macOS,
and that's because, like, D-Bus is a Linux
specific feature, so it doesn't really
work on other platform.
We need to check, replace it with some
alternative, and
QGraphicsScene somehow doesn't really
work on macOS,
and there are many other to-do lists
available, so please have a look
and see if there is anything you want to
work on.
All right. So, since this is a
pre-recorded talk I won't be able to do
the Q & A real time in the video.
However, I will be around on the
collaborative pad
and the IRC #emacsconf,
#emacsconf-questions
to answer any questions when it pops up,
and you can also submit an issue
on the repo, and you can check the wiki
for some other guides and tricks.
All right. Thank you guys, and hopefully
you find
this EAF project very interesting, and
enjoy the rest of EmacsConf 2020.