Hello EmacsConf!
Today I'm here to introduce you to the Emacs Reader.
It is a general-purpose document viewer
that lives inside our beloved Emacs.
It tries to prioritize memory
and performance efficiency as much as possible
even when you're using a lower-end hardware.
And, most importantly,
it tries to do things in an Emacs manner.
That is, it tries to integrate
with existing packages as much as possible
instead of reinventing the wheel.
And architecturally, it tries to take the advantage
of dynamic or native modules
which were introduced back in 2015 into Emacs.
You would ask, why exactly do we need
another document viewer in Emacs?
Don't we already have the built-in DocView
and the notorious pdf-tools?
Well, the built-in DocView has unusable latency,
and I'm going to show you this later
when I compare this with Emacs Reader.
The famous pdf-tools has actually multiple issues.
One, it is extremely memory-hungry
regardless of what kind of PDFs you're reading.
And, well, it can only read PDFs.
Poppler, the library which pdf-tools uses,
is actually sub-optimal,
especially relative to MuPDF,
which is what Emacs Reader is based on.
pdf-tools is also extremely painful to install.
If you've ever installed pdf-tools,
you know that it has a bunch of dependencies,
including a server that is supposedly packaged.
across package managers, system package managers.
It's extremely difficult to install
and painful to install.
And of course, pdf-tools
since the last couple of years
has not been maintained as much.
There's huge PRs that have been unnoticed and unmerged.
Architecturally, Emacs Reader takes a distance
from both DocView and pdf-tools.
So how DocView works is that
it basically wraps around
a tool called mutool.
mutool is actually
a command line tool from MuPDF itself.
It relies on mutool and a bunch
of other similar command line tools,
and basically makes process calls
from Elisp to the CLI tools.
That's how DocView works,
and that's why it sort of has latency issues
because that's the best you can do
by literally calling CLI tools
and outputting the images into Emacs.
How pdf-tools works is that it tries
to have a server-client model.
So the client is Emacs
and the server is basically
something they call epdfinfo.
It's supposed to render the images using Poppler
and then send the images to Emacs
which then tries to display.
I think the server client model is terrible.
One, for latency purposes,
and two, it makes things
unnecessarily more complicated.
Here is where we come
and introduce dynamic modules.
So Emacs Reader is based on
the concept of dynamic modules
which I'm going to talk about in a bit.
But how it works is that we have C modules.
So we have the emacs-module.h,
that's the dynamic module header
which every dynamic module package must have.
And then we have our C files.
And these C files essentially define functions
that are going to be used in Emacs but in C.
We then load these C modules
using simple (require ...) in our Elisp modules.
And then whenever we call
something in the Emacs runtime,
say I'm going to open
PDF files in (find-file) or (reader-open-doc),
what it does is that
it tries to use one of the functions
that is wrapped in Elisp,
but actually tries to call a function in C.
And then the C module is actually
going to make calls to the MuPDF.
Here the MuPDF system package,
this is actually a system package
that is dynamically linked to the C modules.
So we're basically
just using it as a shared library.
So you have the fz_load_page, for example,
it's a MuPDF function
that we're going to be using in the C modules.
So it's going to make
a shared dynamic call to MuPDF
and then render the page
and then show this to Emacs.
This pipeline, I argue,
is much better and leaner and efficient
than a server-client model.
One, because we don't really need
the server-client model.
So back when Politza
first introduced pdf-tools,
that was like 10 years ago in 2015,
the concept of dynamic modules
were not integrated into Emacs.
I think they came around
like one or two years late, 2017.
So that's the best he could go with.
We don't really have to, today,
because, since we can use MuPDF
as a shared library
which can render things in real-time
and just give us the rendered images
which we can then display,
there's no reason for a server to do things for us.
So that's the main architectural difference
that Emacs Reader introduces
compared to pdf-tools and DocView.
What exactly are dynamic modules?
Well, I can't really give you
a full-fledged explanation,
but essentially dynamic modules
let you evaluate
native compiled code
in other languages like C, C++, Rust
that behaves like regular Emacs Lisp.
So when our Emacs C modules,
the render-core.c or render-theme.c,
when all of these are compiled,
and they're called from the Elisp modules.
They behave like Elisp even though
they're as fast as a C function
because they're compiled C code.
But you essentially call them
just like Elisp functions.
You can find them using C-h f and so on.
So you can call any function
from any language that supports
the C ABI, which is virtually everything,
without leaving Emacs
and without losing any performance.
This is extremely helpful
when you want to use
existing libraries like MuPDF
or any other cryptographic library
that is written in C
and you don't want to rewrite
the entire thing in Elisp,
but you can just use it as a native library.
You can read more
on how dynamic modules work
and how you can write one in this blog.
This is something that I wrote myself
just after starting this package
and it will give you a bit more guidance
on how to use dynamic modules more efficiently.
I think dynamic modules
should be used more and more in Emacs
and I think their advantages
have not been exploited
as much as they should.
Now we're going to talk a bit about
the core features of Emacs Reader.
And these are the following features
that we're going to talk about.
And finally, to talk about
some challenges that we faced.
First is memory efficiency.
I already told you that
Emacs Reader's first priority
is to make sure that we are not slow
and we are not taking
a bunch of memory unnecessarily.
So here's a graph of the heap memory size
as it grows for DocView.
So this is again in emacs -Q.
So this is a fresh Emacs session
with just DocView.
It grows up to 900MB
for a very small PDF that is a LaTeX PDF.
No scanned huge PDF. It's a 2MB PDF.
But when I scrolled from the beginning
of the PDF to the end,
it went up to 900MB.
That's the memory heap size.
Does pdf-tools make this any better?
It actually doesn't.
So, pdf-tools pretty much
does the same thing.
if you look at it here
just so if you're going to ask me
are they two different graphs,
or are you just showing me the same graph,
they're actually two different graphs,
because if you look at the DocView graph
it uses cairo and it uses librsvg
because docview by default
converts the images into SVG.
The rendered images are SVGs.
pdf-tools doesn't, so you don't see
any librsvg calls here or anything
So this is pdf-tools
and it basically takes up
the same amount of memory, 900MB,
and exactly the same operation,
exactly the same PDF,
exactly scrolling from first to the last.
Where do we stand?
Well, we actually do much better.
So let me zoom in this.
So if you see, we stand within
at a peak of 72MB.
Exactly the same PDF,
exactly the same operation
from the beginning to the end,
around 285 pages scrolled.
We take much less than 80 MB.
And actually, to be very frank,
the only memory that we're storing in Emacs,
oh, sorry, not in Emacs,
in the MuPDF heap is just about 30 MB.
It's this dark red one.
That's the cache that we're storing.
That's the memory that we're interacting with
in real time.
This is stuff that Emacs adds on top of it
and a bit of libmupdf.
So you can see, in terms of memory,
we're saving...
we're literally down,
what, a fraction of 10!
This was a priority for us
since the beginning,
because when I was starting to use pdf-tools,
it was unusable for me
because I was on a lower-end hardware
and I thought it should not be
really that difficult
for a document reader
to not take a gigabyte of memory.
It really shouldn't because
you're not really doing that much,
you're just displaying images.
So that's how efficient
we are in terms of memory.
Let's see how efficient
we are in terms of speed.
So Emacs Reader is actually
as fast as pdf-tools,
and it is actually
way more faster than DocView.
In some cases,
it actually beats existing
standalone document readers and browsers.
So let's actually see this in action.
So here we are with
a few emacs -Q sessions.
I'm using emacs -Q so as to give you...
that this is actually
as less overhead possible.
So we have first DocView.
All of these tests
are going to be done on the same PDF.
It's the documentation manual from MuPDF.
So if I scroll, this is fine.
I'm just pressing n
and it seems to work fine.
If I press and hold n,
I have pressed n and I'm holding.
And Emacs is stuck.
And it's going to stay stuck
because it's making calls
to the CLI tool that I said, mutool.
And after it's done getting stuck,
it is going to get back.
As you can see, if you go back,
you're able to go back fine.
It does not get stuck
because what Emacs does
is it basically calls mutool,
like fetches a bunch of pages,
essentially all the pages
that you asked for it,
and it puts them into the memory.
And that's it.
It puts them into the memory
and then scrolls through it.
So going back, you will most likely
not have any stuck issues.
Sometimes you do
because some images do get GC'd.
But that's the idea.
Whenever there's no image in memory,
it gets stuck.
And it gets stuck good.
That's DocView.
pdf-tools is actually
not problematic here.
pdf-tools is extremely efficient
and extremely fast.
So we can go through the pages
without any issues.
We can zoom.
The zoom did get stuck a bit,
but that's relatively fine.
Emacs Reader is exactly as fast
as pdf-tools here.
So this is pdf-view,
this is Emacs Reader.
Let's scroll through the pages.
As you can see, nothing is getting stuck
because we're not really waiting
for any tool to send us any images.
We just have a little cache
and we're scrolling through them
and rendering images in real time.
Zooming also works fine.
So, with regards to this,
we're in parity with pdf-tools.
Now, where pdf-tools and actually
a lot of readers have issues
is when they're dealing with scanned PDF.
So, we have this PDF which is notorious
for being really difficult to render
because this is entirely built
with scanned images.
This is the kind of PDF
that you get from Internet Archive.
This is essentially someone
took photos of the book in a camera
and literally turned them into a PDF.
Emacs Reader actually does not have
any issues rendering this.
As you can see, it renders it smoothly
and fine without any halts.
I can change Emacs even while it's doing so,
and it does not have any issues.
pdf-tools are the same.
PDF also does not have any issues.
Sorry. Click pdf-view-mode.
pdf-view (pdf-tools) is a bit slower
but does not have any issues. It works.
Here, actually, pdf-tools and Emacs Reader
are more efficient than even browsers.
So, if I try to open
the same page in a browser,
I'm trying to scroll.
And after I've scrolled and I leave,
scrolling is going to load
for a bunch of seconds
to give me the page.
It's more than five seconds,
as you can see,
and this is actually totally not usable.
If you're going to read this book,
an electromagnetics book,
you're going to have a terrible time
reading this in a browser,
which is supposed to be
the fastest thing alive.
You sort of have the same experience
in Okular. So this is Okular.
If I try to scroll through this,
it will do the same thing.
And while it is better than the browser,
it still takes a while
and it still has, like, if you zoom,
you're going to have a bit of a delay.
You don't really face that in Emacs Reader.
We zoom in and out just fine.
And even with using mouse,
you can zoom in and out just fine.
So this is how Emacs Reader performs
in terms of speed with these other tools.
Now we will go back to the original presentation.
Now, how exactly is Emacs Reader
able to do a lot of this?
I wish I could sort of spend
an entire session
just talking about this, but I can't.
So I'm just going to make this short.
When you load Emacs Reader,
in the standard output,
it's going to say this:
that eight threads have been initialized.
Now, what we did with Emacs here
is that we enabled
system-level multithreading.
Now, Emacs is not multithreaded.
We all know that notoriously.
It is single-threaded.
But we don't really
need Emacs to be multithreaded, though.
Emacs does not need to be multithreaded.
What needs to be multithreaded
is the rendering part
because that's the most expensive part.
In Emacs, we're only just displaying images.
Emacs itself does not have a PDF engine
that is rendering stuff.
MuPDF is supposed to take care of that.
So if I can do multithreading
in the rendering pipeline,
that is when I'm rendering pages
instead of displaying them,
that's fine for me because
the rendering part most of the time,
especially in scanned PDFs,
is the most expensive part.
So if you look at this graph,
we have two parts here.
We have the display pipeline
and we have the rendering pipeline.
In the display pipeline,
we have just the Emacs session
which has the reader loaded
and that's the main thread.
Then we have the rendering pipeline
which has the MuPDF system package
dynamically linked.
So when you load Emacs Reader,
we initialize a thread pool with eight threads.
Now what you do is let's say we are at page 50.
At page 50, the Emacs Reader
maintains a cache.
It's like a stack of pages
that we keep in memory all the time.
This cache is entirely outside of Emacs.
It is not inside Emacs environment.
It is in the C memory heap,
in the MuPDF memory heap
that is outside of Emacs environment.
It does not make any calls to Emacs anything.
It does not have a single Elisp line.
So this cache is stored outside.
Now when I want to retrieve
anything from this cache,
let's say, so I have cached
up until 55, from 45 to 55.
So what happens is that
when you're at page 50,
you always have a cache
that's n + 5 and n - 5.
So you have cache of 5 pages forward
and 5 pages backward.
But let's say I want to go to page 56.
So I will ask an Emacs render page 56.
And I'm not going to ask it
to MuPDF directly.
I'm going to ask it
to the thread pool that do this job.
And thread pool is going to
assign one thread to it.
Let's say the thread 1
which is going to render page 56.
So this thread is going to make calls to MuPDF
through our code dynamic module.
And MuPDF after rendering it
is going to store it in the cache.
So we're going to add another 56 page to this.
Now, while this is happening,
Emacs Reader does not, like Emacs itself,
the session is not going to be stuck
because we just made a call to the thread.
We just asked the thread.
So like this, this call, like it's done.
So you just assign something to a thread
and then this is fine.
Like, you're not waiting for the thread
to complete or anything.
Emacs is not waiting for the thread to complete.
The dynamic module or the C side
might wait to complete
but that is entirely different from
the Emacs session.
So Emacs viewer can continue to
display the page 50
while the rendering pipeline
is still rendering the 56th page.
And when Emacs asks to display page 56,
it's going to ask it to a thread pool.
Then thread pool is going to assign
another thread, let's say this one,
to retrieve page 56 from the memory cache.
And then the 56 page is going to be sent
to the Emacs to be displayed.
Again, the retrieval part
is entirely independent of Emacs.
Emacs does not have to wait for it.
Emacs only needs to wait to display it.
So, the displaying part
and the rendering pipeline
are entirely asynchronous, so to speak.
And in the diagram, if you see,
all the arrows that are
magenta in color,
they are native to the Emacs runtime.
That is, they are single-threaded.
They are connected to Emacs.
And all the arrows that are red in color,
they are totally asynchronous.
They can be multi-threaded if you want.
They are multi-threaded by default
because they interact
only with the MuPDF shared library
and the C heap.
They do not touch anything
in the Emacs runtime.
This is how we're able to switch quickly
between these huge scanned PDFs
that have huge images
in each of their pages
because we don't really wait for
each page to be rendered.
And Emacs does not wait for that.
So that's another architectural feature
of Emacs Reader
that we are system-level multithreaded.
Now Emacs viewer also supports
almost all document formats.
It supports PDF, EPUB, MOBI, XPS, CPZ comics,
and it even supports
other non-ebook formats
like document format,
so you can open
LibreOffice documents in it,
and even stuff like PPT and Excel in it,
even though they're not going to be
supported in a as nice manner.
And we can do that because MuPDF does this.
MuPDF has support for all of this
and it treats them just as it treats PDF.
Nothing special.
The only thing that we don't support right now
is DejaVu, so that is not supported right now.
I'm going to work on making it supported
at the upstream MuPDF.
That's going to take a long time,
but it's in the plans.
Now with Emacs Reader,
we also integrate
with existing Emacs packages
as much as possible.
So bookmarks, C-x r b,
you can do it natively.
So you can save a page as a bookmark
just as you save anything else in Emacs
as a bookmark.
There's also saveplace integration.
So you can scroll a PDF, close it,
and then come back to it
at the same page that you saved it at.
Sorry, that you closed it at.
And it's going to work just out of the box
because of the saveplace
package in Emacs that is built in.
We also have imenu integration
for table of contents.
So if you see this, this is imenu
and you can scroll through the contents
just like you scroll through any imenu.
You can also do it in the menu bar by clicking.
It works just as nice.
We also have something like
the outline mode that pdf-tools has.
So if you press O in a document,
it's going to give you this outline.
And these are buttons that are clickable.
You can click them.
You can press Enter at them.
And this is the menu bar item that I was looking at.
If you click here, index,
it's going to show you
the exact same thing
but in a different interface.
We also have a naive dark mode,
which is not really as nice as
we would like it to be,
and dark mode fanatics
I'm sure will have issues with it,
but we're going to improve it in time.
For now, this is what we have.
And it can be enabled per document,
so you can have one, like,
one document that is in dark mode,
but another one that is not.
That is nice to have.
Eventually we're going to work on more themes.
You should be able to actually integrate it
with Emacs themes as much as possible.
You can make it default so that
it inherits colors from the Emacs theme.
That is one of the things
that we also have planned.
We did face a bunch of challenges
while trying to implement these features.
One of the initial challenges was that
SVGs were actually a bad idea.
They're huge, especially in scanned PDFs,
and they make things much slower.
So we chose to actually have PPMs,
which is the simplest image format ever possible.
Now, it was also very difficult
to make reader-mode be window-specific.
So, you know, while you're scrolling
the same document in one window,
the other window with the same document
should not change.
We should be able to have multiple pages
in different windows of the same document.
That was very difficult
because as I told you about the cache,
the cache works in an idiosyncratic manner
and we needed to make it so that each window
will have its own cache
instead of having a global cache for each file.
That took some rewrite.
And now, because we needed to do
this sort of multithreading,
system-level multithreading,
we needed to use
a specific package of MuPDF
that had a bug for this which got fixed.
And that's 1.26.0.
Because we did that,
a lot of the GNU/Linux distributions did not
really have this latest package.
So we had to actually
package it in-tree.
as a git sub-module.
That was a horror! But eventually... now
I think most GNU/Linux distributions
already have this [version].
The upcoming features that we have planned
are the first one is that we need to rewrite
the display mechanism entirely from scratch
to use a tiled rendering approach.
So right now we just take an image
and display it inside an Emacs buffer
just like that.
But it will be changed so that the image
will be displayed in the tiled manner
so there will be multiple tiles
but it'll be pixel perfect
so you won't really see a difference.
The reason to do this is to implement features
for text selection, actually.
So we can't really do text selection
without running into a bunch of memory
and other issues latency issues
if we don't do tiling.
So we need to do those two things,
they are at the highest priority right now.
And then, once we're done with that,
we're going to support annotations,
highlighting, everything that you're used to
in pdf-tools and org-noter.
And once we're done with that,
we're going to also integrate with AucTeX and SyncTeX.
Because right now, when a PDF gets updated,
especially a LaTeX PDF,
since there is no SyncTeX integration,
it can't really do it nicely
and it sometimes even crashes Emacs.
So that's something that
we will be planning to implement.
Now, from this experiment,
what exactly can Emacs,
the Emacs core devs and others
who are building packages can learn?
Well, the first thing is that all of this
should not be really this difficult
because all we're asking from Emacs
is to display images in real-time
and update them in real-time.
That should not be that difficult
of a thing to do, but apparently it is.
And that's why Emacs's graphical interface
needs to be more modular, more composable,
and flexible for real-time graphics.
If it is supposed to have things like,
again, a document reader,
something like a video editor,
and something like that,
Emacs's graphical interface
needs to grow and be more mature.
One of the things
that's stopping it from doing that
is actually Emacs's overlay functionality.
So right now, the way we display
an image in a buffer
is using an overlay,
actually multiple overlays.
Overlays are static in the sense that
if I attach to one image to one overlay,
I need to have an entirely different image
updated for that overlay.
So I need to create another different image,
change it in the memory,
and then display it to update it.
I can't change the image data
in real time of the overlay.
And that is a big issue.
I've actually made an emacs-devel
mailing list thread about it.
I talked to Eli about it as well.
And he said there's a possibility
that this can be changed,
but it's going to take
a certain amount of rewrite.
There's also issues with Emacs GC.
Emacs GC sometimes leaks memory
when you update images too quickly.
That is, when you have a bunch of images
that are getting churned out too quickly,
Emacs GC starts leaking
and it just goes up to
a huge number of gigabytes in RAM.
That's also a huge problem.
The dynamic module API,
the emacs-module.h header,
needs to have more helpers.
It's really bare bones,
and I like that it is bare bones
so that other languages can use it,
but at the same time, I think
it'll be really good
if we can have some helpers
that can do better memory interaction,
like strings and so on,
which we also faced some issues with.
Emacs's fractional scaling system
seems to be broken across different toolkits.
We have bug reports that say in pgtk in Wayland,
something seems to render differently
because they have fractional scaling enabled.
So that's something
that I think Emacs, overall,
I think Emacs needs to focus on improving
the graphical interface pipeline
to be a much more mature one.
And finally, how can you contribute
to the development of Emacs Reader?
Well, we are on Codeberg.
We are not on GitHub, sorry.
You can go there,
you can look through the issues
and send us a PR if you're interested.
The next major release
is going to go to GNU ELPA.
Finally, we are not yet at GNU ELPA,
so you can't really do M-x package-install
and install our package.
you would need to install it
through use-package :vc.
And since we're going to go to GNU ELPA,
we request you to assign
your copyright to Emacs
because GNU ELPA is essentially part of GNU Emacs.
So you would need to do copyright assignment
if you make non-trivial contribution.
You can join us at IRC
at #phi-mu-lambda.
And I also stream the development
of this package
bi-weekly on Sundays
at PeerTube at the following channel.
Feel free to join us.
Finally, I want to thank Tushar,
who has been persistently contributing
to the project since 0.1.0,
and I'm very, very thankful for him,
for his suggestions,
and for his code contributions as well.
I would also like to thank Prom,
who fixed a major bug
in the Windows build,
since I don't really use Windows anymore,
so that was really nice,
and for Teeoius, for fixing a pthread bug.
I would also like to thank others
who helped fix little things,
who come to the stream to chat,
who sort of see me bang my head
across these C memory bugs.
So thank you to all of those.
And thank you finally to the viewers
and to EmacsConf organizers as well.
This is a splendid opportunity.
Thank you.