WEBVTT captioned by jay_bird and sachac NOTE An introduction to the Emacs reader 00:00:00.720 --> 00:00:02.879 Hello EmacsConf! 00:00:02.880 --> 00:00:06.639 Today I'm here to introduce you to the Emacs Reader. 00:00:06.640 --> 00:00:08.759 It is a general-purpose document viewer 00:00:08.760 --> 00:00:12.319 that lives inside our beloved Emacs. 00:00:12.320 --> 00:00:14.159 It tries to prioritize memory 00:00:14.160 --> 00:00:17.159 and performance efficiency as much as possible 00:00:17.160 --> 00:00:20.519 even when you're using a lower-end hardware. 00:00:20.520 --> 00:00:22.119 And, most importantly, 00:00:22.120 --> 00:00:25.439 it tries to do things in an Emacs manner. 00:00:25.440 --> 00:00:26.999 That is, it tries to integrate 00:00:27.000 --> 00:00:29.719 with existing packages as much as possible 00:00:29.720 --> 00:00:32.239 instead of reinventing the wheel. 00:00:32.240 --> 00:00:36.119 And architecturally, it tries to take the advantage 00:00:36.120 --> 00:00:38.479 of dynamic or native modules 00:00:38.480 --> 00:00:44.759 which were introduced back in 2015 into Emacs. NOTE Yet another document viewer in Emacs? 00:00:44.760 --> 00:00:46.759 You would ask, why exactly do we need 00:00:46.760 --> 00:00:49.199 another document viewer in Emacs? 00:00:49.200 --> 00:00:51.839 Don't we already have the built-in DocView 00:00:51.840 --> 00:00:55.199 and the notorious pdf-tools? 00:00:55.200 --> 00:00:59.439 Well, the built-in DocView has unusable latency, 00:00:59.440 --> 00:01:01.399 and I'm going to show you this later 00:01:01.400 --> 00:01:04.599 when I compare this with Emacs Reader. 00:01:04.600 --> 00:01:08.079 The famous pdf-tools has actually multiple issues. 00:01:08.080 --> 00:01:10.639 One, it is extremely memory-hungry 00:01:10.640 --> 00:01:14.399 regardless of what kind of PDFs you're reading. 00:01:14.400 --> 00:01:17.939 And, well, it can only read PDFs. 00:01:17.940 --> 00:01:22.199 Poppler, the library which pdf-tools uses, 00:01:22.200 --> 00:01:23.879 is actually sub-optimal, 00:01:23.880 --> 00:01:25.799 especially relative to MuPDF, 00:01:25.800 --> 00:01:28.559 which is what Emacs Reader is based on. 00:01:28.560 --> 00:01:31.919 pdf-tools is also extremely painful to install. 00:01:31.920 --> 00:01:34.279 If you've ever installed pdf-tools, 00:01:34.280 --> 00:01:38.479 you know that it has a bunch of dependencies, 00:01:38.480 --> 00:01:42.319 including a server that is supposedly packaged. 00:01:42.320 --> 00:01:45.061 across package managers, system package managers. 00:01:45.062 --> 00:01:47.737 It's extremely difficult to install 00:01:47.738 --> 00:01:50.279 and painful to install. 00:01:50.280 --> 00:01:52.839 And of course, pdf-tools 00:01:52.840 --> 00:01:54.559 since the last couple of years 00:01:54.560 --> 00:01:56.559 has not been maintained as much. 00:01:56.560 --> 00:02:05.759 There's huge PRs that have been unnoticed and unmerged. NOTE Architecture of Emacs Reader 00:02:05.760 --> 00:02:08.999 Architecturally, Emacs Reader takes a distance 00:02:09.000 --> 00:02:12.559 from both DocView and pdf-tools. 00:02:12.560 --> 00:02:15.399 So how DocView works is that 00:02:15.400 --> 00:02:18.679 it basically wraps around 00:02:18.680 --> 00:02:20.879 a tool called mutool. 00:02:20.880 --> 00:02:22.319 mutool is actually 00:02:22.320 --> 00:02:26.119 a command line tool from MuPDF itself. 00:02:26.120 --> 00:02:28.199 It relies on mutool and a bunch 00:02:28.200 --> 00:02:30.579 of other similar command line tools, 00:02:30.580 --> 00:02:34.199 and basically makes process calls 00:02:34.200 --> 00:02:36.519 from Elisp to the CLI tools. 00:02:36.520 --> 00:02:38.639 That's how DocView works, 00:02:38.640 --> 00:02:41.319 and that's why it sort of has latency issues 00:02:41.320 --> 00:02:42.519 because that's the best you can do 00:02:42.520 --> 00:02:45.019 by literally calling CLI tools 00:02:45.020 --> 00:02:50.679 and outputting the images into Emacs. 00:02:50.680 --> 00:02:55.039 How pdf-tools works is that it tries 00:02:55.040 --> 00:02:57.479 to have a server-client model. 00:02:57.480 --> 00:02:58.999 So the client is Emacs 00:02:59.000 --> 00:03:00.559 and the server is basically 00:03:00.560 --> 00:03:02.999 something they call epdfinfo. 00:03:03.000 --> 00:03:07.240 It's supposed to render the images using Poppler 00:03:07.241 --> 00:03:10.919 and then send the images to Emacs 00:03:10.920 --> 00:03:13.279 which then tries to display. 00:03:13.280 --> 00:03:16.279 I think the server client model is terrible. 00:03:16.280 --> 00:03:18.079 One, for latency purposes, 00:03:18.080 --> 00:03:19.839 and two, it makes things 00:03:19.840 --> 00:03:21.799 unnecessarily more complicated. 00:03:21.800 --> 00:03:24.199 Here is where we come 00:03:24.200 --> 00:03:26.679 and introduce dynamic modules. 00:03:26.680 --> 00:03:30.579 So Emacs Reader is based on 00:03:30.580 --> 00:03:32.279 the concept of dynamic modules 00:03:32.280 --> 00:03:34.279 which I'm going to talk about in a bit. 00:03:34.280 --> 00:03:37.159 But how it works is that we have C modules. 00:03:37.160 --> 00:03:39.039 So we have the emacs-module.h, 00:03:39.040 --> 00:03:40.679 that's the dynamic module header 00:03:40.680 --> 00:03:43.159 which every dynamic module package must have. 00:03:43.160 --> 00:03:45.479 And then we have our C files. 00:03:45.480 --> 00:03:52.579 And these C files essentially define functions 00:03:52.580 --> 00:03:56.439 that are going to be used in Emacs but in C. 00:03:56.440 --> 00:03:59.319 We then load these C modules 00:03:59.320 --> 00:04:03.799 using simple (require ...) in our Elisp modules. 00:04:03.800 --> 00:04:05.079 And then whenever we call 00:04:05.080 --> 00:04:07.119 something in the Emacs runtime, 00:04:07.120 --> 00:04:09.159 say I'm going to open 00:04:09.160 --> 00:04:13.559 PDF files in (find-file) or (reader-open-doc), 00:04:13.560 --> 00:04:15.799 what it does is that 00:04:15.800 --> 00:04:19.039 it tries to use one of the functions 00:04:19.040 --> 00:04:20.999 that is wrapped in Elisp, 00:04:21.000 --> 00:04:24.839 but actually tries to call a function in C. 00:04:24.840 --> 00:04:26.839 And then the C module is actually 00:04:26.840 --> 00:04:29.279 going to make calls to the MuPDF. 00:04:29.280 --> 00:04:31.599 Here the MuPDF system package, 00:04:31.600 --> 00:04:33.399 this is actually a system package 00:04:33.400 --> 00:04:35.839 that is dynamically linked to the C modules. 00:04:35.840 --> 00:04:36.919 So we're basically 00:04:36.920 --> 00:04:39.799 just using it as a shared library. 00:04:39.800 --> 00:04:43.359 So you have the fz_load_page, for example, 00:04:43.360 --> 00:04:44.839 it's a MuPDF function 00:04:44.840 --> 00:04:47.399 that we're going to be using in the C modules. 00:04:47.400 --> 00:04:50.079 So it's going to make 00:04:50.080 --> 00:04:53.279 a shared dynamic call to MuPDF 00:04:53.280 --> 00:04:55.119 and then render the page 00:04:55.120 --> 00:04:59.179 and then show this to Emacs. 00:04:59.180 --> 00:05:01.839 This pipeline, I argue, 00:05:01.840 --> 00:05:05.599 is much better and leaner and efficient 00:05:05.600 --> 00:05:07.639 than a server-client model. 00:05:07.640 --> 00:05:09.479 One, because we don't really need 00:05:09.480 --> 00:05:10.839 the server-client model. 00:05:10.840 --> 00:05:12.359 So back when Politza 00:05:12.360 --> 00:05:14.759 first introduced pdf-tools, 00:05:14.760 --> 00:05:19.759 that was like 10 years ago in 2015, 00:05:19.760 --> 00:05:21.240 the concept of dynamic modules 00:05:21.241 --> 00:05:23.279 were not integrated into Emacs. 00:05:23.280 --> 00:05:24.359 I think they came around 00:05:24.360 --> 00:05:28.079 like one or two years late, 2017. 00:05:28.080 --> 00:05:31.219 So that's the best he could go with. 00:05:31.220 --> 00:05:33.079 We don't really have to, today, 00:05:33.080 --> 00:05:35.719 because, since we can use MuPDF 00:05:35.720 --> 00:05:36.999 as a shared library 00:05:37.000 --> 00:05:39.479 which can render things in real-time 00:05:39.480 --> 00:05:41.759 and just give us the rendered images 00:05:41.760 --> 00:05:43.599 which we can then display, 00:05:43.600 --> 00:05:49.659 there's no reason for a server to do things for us. 00:05:49.660 --> 00:05:53.359 So that's the main architectural difference 00:05:53.360 --> 00:05:55.479 that Emacs Reader introduces 00:05:55.480 --> 00:06:00.279 compared to pdf-tools and DocView. NOTE A word on dynamic modules 00:06:00.280 --> 00:06:02.479 What exactly are dynamic modules? 00:06:02.480 --> 00:06:04.119 Well, I can't really give you 00:06:04.120 --> 00:06:06.199 a full-fledged explanation, 00:06:06.200 --> 00:06:08.639 but essentially dynamic modules 00:06:08.640 --> 00:06:10.519 let you evaluate 00:06:10.520 --> 00:06:12.039 native compiled code 00:06:12.040 --> 00:06:15.119 in other languages like C, C++, Rust 00:06:15.120 --> 00:06:18.519 that behaves like regular Emacs Lisp. 00:06:18.520 --> 00:06:23.639 So when our Emacs C modules, 00:06:23.640 --> 00:06:26.039 the render-core.c or render-theme.c, 00:06:26.040 --> 00:06:28.299 when all of these are compiled, 00:06:28.300 --> 00:06:30.839 and they're called from the Elisp modules. 00:06:30.840 --> 00:06:34.439 They behave like Elisp even though 00:06:34.440 --> 00:06:37.039 they're as fast as a C function 00:06:37.040 --> 00:06:39.359 because they're compiled C code. 00:06:39.360 --> 00:06:41.399 But you essentially call them 00:06:41.400 --> 00:06:42.759 just like Elisp functions. 00:06:42.760 --> 00:06:47.819 You can find them using C-h f and so on. 00:06:47.820 --> 00:06:49.679 So you can call any function 00:06:49.680 --> 00:06:51.719 from any language that supports 00:06:51.720 --> 00:06:53.519 the C ABI, which is virtually everything, 00:06:53.520 --> 00:06:54.919 without leaving Emacs 00:06:54.920 --> 00:06:56.759 and without losing any performance. 00:06:56.760 --> 00:06:58.479 This is extremely helpful 00:06:58.480 --> 00:06:59.919 when you want to use 00:06:59.920 --> 00:07:02.119 existing libraries like MuPDF 00:07:02.120 --> 00:07:04.079 or any other cryptographic library 00:07:04.080 --> 00:07:06.039 that is written in C 00:07:06.040 --> 00:07:07.037 and you don't want to rewrite 00:07:07.038 --> 00:07:08.537 the entire thing in Elisp, 00:07:08.538 --> 00:07:11.739 but you can just use it as a native library. 00:07:11.740 --> 00:07:13.039 You can read more 00:07:13.040 --> 00:07:14.679 on how dynamic modules work 00:07:14.680 --> 00:07:17.759 and how you can write one in this blog. 00:07:17.760 --> 00:07:19.479 This is something that I wrote myself 00:07:19.480 --> 00:07:22.239 just after starting this package 00:07:22.240 --> 00:07:25.439 and it will give you a bit more guidance 00:07:25.440 --> 00:07:27.519 on how to use dynamic modules more efficiently. 00:07:27.520 --> 00:07:28.679 I think dynamic modules 00:07:28.680 --> 00:07:32.299 should be used more and more in Emacs 00:07:32.300 --> 00:07:34.519 and I think their advantages 00:07:34.520 --> 00:07:36.079 have not been exploited 00:07:36.080 --> 00:07:39.559 as much as they should. NOTE Features of Emacs Reader 00:07:39.560 --> 00:07:42.319 Now we're going to talk a bit about 00:07:42.320 --> 00:07:46.719 the core features of Emacs Reader. 00:07:46.720 --> 00:07:48.879 And these are the following features 00:07:48.880 --> 00:07:50.399 that we're going to talk about. 00:07:50.400 --> 00:07:51.959 And finally, to talk about 00:07:51.960 --> 00:07:56.759 some challenges that we faced. NOTE Memory efficiency 00:07:56.760 --> 00:07:58.519 First is memory efficiency. 00:07:58.520 --> 00:08:00.819 I already told you that 00:08:00.820 --> 00:08:03.239 Emacs Reader's first priority 00:08:03.240 --> 00:08:06.439 is to make sure that we are not slow 00:08:06.440 --> 00:08:07.959 and we are not taking 00:08:07.960 --> 00:08:10.319 a bunch of memory unnecessarily. 00:08:10.320 --> 00:08:14.439 So here's a graph of the heap memory size 00:08:14.440 --> 00:08:17.919 as it grows for DocView. 00:08:17.920 --> 00:08:20.637 So this is again in emacs -Q. 00:08:20.638 --> 00:08:22.399 So this is a fresh Emacs session 00:08:22.400 --> 00:08:25.279 with just DocView. 00:08:25.280 --> 00:08:27.819 It grows up to 900MB 00:08:27.820 --> 00:08:31.559 for a very small PDF that is a LaTeX PDF. 00:08:31.560 --> 00:08:36.779 No scanned huge PDF. It's a 2MB PDF. 00:08:36.780 --> 00:08:39.679 But when I scrolled from the beginning 00:08:39.680 --> 00:08:41.619 of the PDF to the end, 00:08:41.620 --> 00:08:43.639 it went up to 900MB. 00:08:43.640 --> 00:08:46.819 That's the memory heap size. 00:08:46.820 --> 00:08:49.699 Does pdf-tools make this any better? 00:08:49.700 --> 00:08:51.919 It actually doesn't. 00:08:51.920 --> 00:08:55.039 So, pdf-tools pretty much 00:08:55.040 --> 00:08:57.219 does the same thing. 00:08:57.220 --> 00:08:58.439 if you look at it here 00:08:58.440 --> 00:09:01.359 just so if you're going to ask me 00:09:01.360 --> 00:09:02.939 are they two different graphs, 00:09:02.940 --> 00:09:04.839 or are you just showing me the same graph, 00:09:04.840 --> 00:09:06.119 they're actually two different graphs, 00:09:06.120 --> 00:09:08.779 because if you look at the DocView graph 00:09:08.780 --> 00:09:11.559 it uses cairo and it uses librsvg 00:09:11.560 --> 00:09:13.439 because docview by default 00:09:13.440 --> 00:09:16.119 converts the images into SVG. 00:09:16.120 --> 00:09:17.999 The rendered images are SVGs. 00:09:18.000 --> 00:09:20.559 pdf-tools doesn't, so you don't see 00:09:20.560 --> 00:09:24.039 any librsvg calls here or anything 00:09:24.040 --> 00:09:25.439 So this is pdf-tools 00:09:25.440 --> 00:09:27.079 and it basically takes up 00:09:27.080 --> 00:09:29.079 the same amount of memory, 900MB, 00:09:29.080 --> 00:09:30.919 and exactly the same operation, 00:09:30.920 --> 00:09:32.479 exactly the same PDF, 00:09:32.480 --> 00:09:36.139 exactly scrolling from first to the last. 00:09:36.140 --> 00:09:37.719 Where do we stand? 00:09:37.720 --> 00:09:40.559 Well, we actually do much better. 00:09:40.560 --> 00:09:42.599 So let me zoom in this. 00:09:42.600 --> 00:09:46.319 So if you see, we stand within 00:09:46.320 --> 00:09:49.259 at a peak of 72MB. 00:09:49.260 --> 00:09:51.279 Exactly the same PDF, 00:09:51.280 --> 00:09:53.039 exactly the same operation 00:09:53.040 --> 00:09:54.559 from the beginning to the end, 00:09:54.560 --> 00:09:57.599 around 285 pages scrolled. 00:09:57.600 --> 00:10:03.239 We take much less than 80 MB. 00:10:03.240 --> 00:10:05.071 And actually, to be very frank, 00:10:05.072 --> 00:10:09.204 the only memory that we're storing in Emacs, 00:10:09.205 --> 00:10:12.439 oh, sorry, not in Emacs, 00:10:12.440 --> 00:10:16.599 in the MuPDF heap is just about 30 MB. 00:10:16.600 --> 00:10:19.119 It's this dark red one. 00:10:19.120 --> 00:10:22.559 That's the cache that we're storing. 00:10:22.560 --> 00:10:24.759 That's the memory that we're interacting with 00:10:24.760 --> 00:10:25.479 in real time. 00:10:25.480 --> 00:10:29.199 This is stuff that Emacs adds on top of it 00:10:29.200 --> 00:10:32.919 and a bit of libmupdf. 00:10:32.920 --> 00:10:35.199 So you can see, in terms of memory, 00:10:35.200 --> 00:10:37.239 we're saving... 00:10:37.240 --> 00:10:41.119 we're literally down, 00:10:41.120 --> 00:10:45.359 what, a fraction of 10! 00:10:45.360 --> 00:10:48.519 This was a priority for us 00:10:48.520 --> 00:10:49.279 since the beginning, 00:10:49.280 --> 00:10:51.999 because when I was starting to use pdf-tools, 00:10:52.000 --> 00:10:53.359 it was unusable for me 00:10:53.360 --> 00:10:55.159 because I was on a lower-end hardware 00:10:55.160 --> 00:10:57.599 and I thought it should not be 00:10:57.600 --> 00:10:58.959 really that difficult 00:10:58.960 --> 00:11:00.879 for a document reader 00:11:00.880 --> 00:11:04.099 to not take a gigabyte of memory. 00:11:04.100 --> 00:11:05.919 It really shouldn't because 00:11:05.920 --> 00:11:07.359 you're not really doing that much, 00:11:07.360 --> 00:11:10.919 you're just displaying images. 00:11:10.920 --> 00:11:12.239 So that's how efficient 00:11:12.240 --> 00:11:13.639 we are in terms of memory. 00:11:13.640 --> 00:11:15.371 Let's see how efficient 00:11:15.372 --> 00:11:18.719 we are in terms of speed. NOTE Performance and speed 00:11:18.720 --> 00:11:21.099 So Emacs Reader is actually 00:11:21.100 --> 00:11:23.119 as fast as pdf-tools, 00:11:23.120 --> 00:11:24.079 and it is actually 00:11:24.080 --> 00:11:27.239 way more faster than DocView. 00:11:27.240 --> 00:11:28.559 In some cases, 00:11:28.560 --> 00:11:31.679 it actually beats existing 00:11:31.680 --> 00:11:34.859 standalone document readers and browsers. 00:11:34.860 --> 00:11:41.119 So let's actually see this in action. 00:11:41.120 --> 00:11:42.319 So here we are with 00:11:42.320 --> 00:11:46.039 a few emacs -Q sessions. 00:11:46.040 --> 00:11:50.719 I'm using emacs -Q so as to give you... 00:11:50.720 --> 00:11:52.159 that this is actually 00:11:52.160 --> 00:11:55.139 as less overhead possible. 00:11:55.140 --> 00:11:57.359 So we have first DocView. 00:11:57.360 --> 00:12:01.137 All of these tests 00:12:01.138 --> 00:12:03.039 are going to be done on the same PDF. 00:12:03.040 --> 00:12:07.199 It's the documentation manual from MuPDF. 00:12:07.200 --> 00:12:10.559 So if I scroll, this is fine. 00:12:10.560 --> 00:12:12.859 I'm just pressing n 00:12:12.860 --> 00:12:15.159 and it seems to work fine. 00:12:15.160 --> 00:12:19.519 If I press and hold n, 00:12:19.520 --> 00:12:21.799 I have pressed n and I'm holding. 00:12:21.800 --> 00:12:26.419 And Emacs is stuck. 00:12:26.420 --> 00:12:27.559 And it's going to stay stuck 00:12:27.560 --> 00:12:28.799 because it's making calls 00:12:28.800 --> 00:12:31.279 to the CLI tool that I said, mutool. 00:12:31.280 --> 00:12:35.519 And after it's done getting stuck, 00:12:35.520 --> 00:12:40.179 it is going to get back. 00:12:40.180 --> 00:12:43.039 As you can see, if you go back, 00:12:43.040 --> 00:12:45.079 you're able to go back fine. 00:12:45.080 --> 00:12:46.199 It does not get stuck 00:12:46.200 --> 00:12:48.439 because what Emacs does 00:12:48.440 --> 00:12:51.519 is it basically calls mutool, 00:12:51.520 --> 00:12:53.239 like fetches a bunch of pages, 00:12:53.240 --> 00:12:54.919 essentially all the pages 00:12:54.920 --> 00:12:56.199 that you asked for it, 00:12:56.200 --> 00:12:59.159 and it puts them into the memory. 00:12:59.160 --> 00:12:59.879 And that's it. 00:12:59.880 --> 00:13:01.199 It puts them into the memory 00:13:01.200 --> 00:13:03.139 and then scrolls through it. 00:13:03.140 --> 00:13:05.839 So going back, you will most likely 00:13:05.840 --> 00:13:07.239 not have any stuck issues. 00:13:07.240 --> 00:13:07.839 Sometimes you do 00:13:07.840 --> 00:13:10.919 because some images do get GC'd. 00:13:10.920 --> 00:13:13.599 But that's the idea. 00:13:13.600 --> 00:13:16.639 Whenever there's no image in memory, 00:13:16.640 --> 00:13:18.739 it gets stuck. 00:13:18.740 --> 00:13:21.239 And it gets stuck good. 00:13:21.240 --> 00:13:23.579 That's DocView. 00:13:23.580 --> 00:13:25.199 pdf-tools is actually 00:13:25.200 --> 00:13:27.359 not problematic here. 00:13:27.360 --> 00:13:29.039 pdf-tools is extremely efficient 00:13:29.040 --> 00:13:30.199 and extremely fast. 00:13:30.200 --> 00:13:32.839 So we can go through the pages 00:13:32.840 --> 00:13:34.479 without any issues. 00:13:34.480 --> 00:13:37.159 We can zoom. 00:13:37.160 --> 00:13:39.879 The zoom did get stuck a bit, 00:13:39.880 --> 00:13:44.039 but that's relatively fine. 00:13:44.040 --> 00:13:46.959 Emacs Reader is exactly as fast 00:13:46.960 --> 00:13:49.199 as pdf-tools here. 00:13:49.200 --> 00:13:50.279 So this is pdf-view, 00:13:50.280 --> 00:13:51.279 this is Emacs Reader. 00:13:51.860 --> 00:13:55.759 Let's scroll through the pages. 00:13:55.760 --> 00:13:59.159 As you can see, nothing is getting stuck 00:13:59.160 --> 00:14:00.919 because we're not really waiting 00:14:00.920 --> 00:14:06.359 for any tool to send us any images. 00:14:06.360 --> 00:14:08.299 We just have a little cache 00:14:08.300 --> 00:14:09.399 and we're scrolling through them 00:14:09.400 --> 00:14:13.959 and rendering images in real time. 00:14:13.960 --> 00:14:17.279 Zooming also works fine. 00:14:17.280 --> 00:14:19.519 So, with regards to this, 00:14:19.520 --> 00:14:23.679 we're in parity with pdf-tools. NOTE Scanned PDFs 00:14:23.680 --> 00:14:26.319 Now, where pdf-tools and actually 00:14:26.320 --> 00:14:28.079 a lot of readers have issues 00:14:28.080 --> 00:14:32.499 is when they're dealing with scanned PDF. 00:14:32.500 --> 00:14:36.839 So, we have this PDF which is notorious 00:14:36.840 --> 00:14:40.599 for being really difficult to render 00:14:40.600 --> 00:14:42.599 because this is entirely built 00:14:42.600 --> 00:14:43.479 with scanned images. 00:14:43.480 --> 00:14:44.619 This is the kind of PDF 00:14:44.620 --> 00:14:46.519 that you get from Internet Archive. 00:14:46.520 --> 00:14:47.839 This is essentially someone 00:14:47.840 --> 00:14:50.919 took photos of the book in a camera 00:14:50.920 --> 00:14:56.659 and literally turned them into a PDF. 00:14:56.660 --> 00:14:58.719 Emacs Reader actually does not have 00:14:58.720 --> 00:15:01.079 any issues rendering this. 00:15:01.080 --> 00:15:05.119 As you can see, it renders it smoothly 00:15:05.120 --> 00:15:09.679 and fine without any halts. 00:15:09.680 --> 00:15:13.959 I can change Emacs even while it's doing so, 00:15:13.960 --> 00:15:17.139 and it does not have any issues. 00:15:17.140 --> 00:15:20.071 pdf-tools are the same. 00:15:20.072 --> 00:15:21.759 PDF also does not have any issues. 00:15:21.760 --> 00:15:26.579 Sorry. Click pdf-view-mode. 00:15:26.580 --> 00:15:29.859 pdf-view (pdf-tools) is a bit slower 00:15:29.860 --> 00:15:35.619 but does not have any issues. It works. 00:15:35.620 --> 00:15:40.700 Here, actually, pdf-tools and Emacs Reader 00:15:40.701 --> 00:15:46.099 are more efficient than even browsers. 00:15:46.100 --> 00:15:47.199 So, if I try to open 00:15:47.200 --> 00:15:50.839 the same page in a browser, 00:15:50.840 --> 00:15:52.919 I'm trying to scroll. 00:15:52.920 --> 00:15:54.919 And after I've scrolled and I leave, 00:15:54.920 --> 00:15:58.119 scrolling is going to load 00:15:58.120 --> 00:15:59.839 for a bunch of seconds 00:15:59.840 --> 00:16:03.139 to give me the page. 00:16:03.140 --> 00:16:04.679 It's more than five seconds, 00:16:04.680 --> 00:16:05.479 as you can see, 00:16:05.480 --> 00:16:08.639 and this is actually totally not usable. 00:16:08.640 --> 00:16:10.199 If you're going to read this book, 00:16:10.200 --> 00:16:11.999 an electromagnetics book, 00:16:12.000 --> 00:16:13.599 you're going to have a terrible time 00:16:13.600 --> 00:16:14.759 reading this in a browser, 00:16:14.760 --> 00:16:15.479 which is supposed to be 00:16:15.480 --> 00:16:17.159 the fastest thing alive. 00:16:17.160 --> 00:16:19.119 You sort of have the same experience 00:16:19.120 --> 00:16:20.559 in Okular. So this is Okular. 00:16:20.560 --> 00:16:22.439 If I try to scroll through this, 00:16:22.440 --> 00:16:25.419 it will do the same thing. 00:16:25.420 --> 00:16:28.519 And while it is better than the browser, 00:16:28.520 --> 00:16:31.119 it still takes a while 00:16:31.120 --> 00:16:34.119 and it still has, like, if you zoom, 00:16:34.120 --> 00:16:36.799 you're going to have a bit of a delay. 00:16:36.800 --> 00:16:41.579 You don't really face that in Emacs Reader. 00:16:41.580 --> 00:16:45.259 We zoom in and out just fine. 00:16:45.260 --> 00:16:47.239 And even with using mouse, 00:16:47.240 --> 00:16:51.839 you can zoom in and out just fine. 00:16:51.840 --> 00:16:54.799 So this is how Emacs Reader performs 00:16:54.800 --> 00:17:01.119 in terms of speed with these other tools. 00:17:01.120 --> 00:17:08.959 Now we will go back to the original presentation. NOTE System-level multi-threading 00:17:08.960 --> 00:17:11.919 Now, how exactly is Emacs Reader 00:17:11.920 --> 00:17:14.079 able to do a lot of this? 00:17:14.080 --> 00:17:17.839 I wish I could sort of spend 00:17:17.840 --> 00:17:18.999 an entire session 00:17:19.000 --> 00:17:21.239 just talking about this, but I can't. 00:17:21.240 --> 00:17:22.919 So I'm just going to make this short. 00:17:22.920 --> 00:17:24.799 When you load Emacs Reader, 00:17:24.800 --> 00:17:26.319 in the standard output, 00:17:26.320 --> 00:17:27.439 it's going to say this: 00:17:27.440 --> 00:17:29.279 that eight threads have been initialized. 00:17:29.280 --> 00:17:32.679 Now, what we did with Emacs here 00:17:32.680 --> 00:17:33.799 is that we enabled 00:17:33.800 --> 00:17:35.039 system-level multithreading. 00:17:35.040 --> 00:17:36.639 Now, Emacs is not multithreaded. 00:17:36.640 --> 00:17:38.199 We all know that notoriously. 00:17:38.200 --> 00:17:39.519 It is single-threaded. 00:17:39.520 --> 00:17:41.479 But we don't really 00:17:41.480 --> 00:17:43.819 need Emacs to be multithreaded, though. 00:17:43.820 --> 00:17:45.759 Emacs does not need to be multithreaded. 00:17:45.760 --> 00:17:47.199 What needs to be multithreaded 00:17:47.200 --> 00:17:48.519 is the rendering part 00:17:48.520 --> 00:17:50.759 because that's the most expensive part. 00:17:50.760 --> 00:17:53.519 In Emacs, we're only just displaying images. 00:17:53.520 --> 00:17:56.479 Emacs itself does not have a PDF engine 00:17:56.480 --> 00:17:57.919 that is rendering stuff. 00:17:57.920 --> 00:18:00.559 MuPDF is supposed to take care of that. 00:18:00.560 --> 00:18:03.199 So if I can do multithreading 00:18:03.200 --> 00:18:05.079 in the rendering pipeline, 00:18:05.080 --> 00:18:07.119 that is when I'm rendering pages 00:18:07.120 --> 00:18:08.719 instead of displaying them, 00:18:08.720 --> 00:18:10.279 that's fine for me because 00:18:10.280 --> 00:18:11.679 the rendering part most of the time, 00:18:11.680 --> 00:18:12.959 especially in scanned PDFs, 00:18:12.960 --> 00:18:14.679 is the most expensive part. 00:18:14.680 --> 00:18:16.439 So if you look at this graph, 00:18:16.440 --> 00:18:17.959 we have two parts here. 00:18:17.960 --> 00:18:19.679 We have the display pipeline 00:18:19.680 --> 00:18:22.279 and we have the rendering pipeline. 00:18:22.280 --> 00:18:23.639 In the display pipeline, 00:18:23.640 --> 00:18:26.519 we have just the Emacs session 00:18:26.520 --> 00:18:29.359 which has the reader loaded 00:18:29.360 --> 00:18:31.579 and that's the main thread. 00:18:31.580 --> 00:18:33.319 Then we have the rendering pipeline 00:18:33.320 --> 00:18:35.559 which has the MuPDF system package 00:18:35.560 --> 00:18:38.459 dynamically linked. 00:18:38.460 --> 00:18:40.399 So when you load Emacs Reader, 00:18:40.400 --> 00:18:45.159 we initialize a thread pool with eight threads. 00:18:45.160 --> 00:18:48.759 Now what you do is let's say we are at page 50. 00:18:48.760 --> 00:18:51.759 At page 50, the Emacs Reader 00:18:51.760 --> 00:18:53.999 maintains a cache. 00:18:54.000 --> 00:18:56.519 It's like a stack of pages 00:18:56.520 --> 00:18:58.479 that we keep in memory all the time. 00:18:58.480 --> 00:19:02.519 This cache is entirely outside of Emacs. 00:19:02.520 --> 00:19:04.559 It is not inside Emacs environment. 00:19:04.560 --> 00:19:07.570 It is in the C memory heap, 00:19:07.571 --> 00:19:09.119 in the MuPDF memory heap 00:19:09.120 --> 00:19:11.119 that is outside of Emacs environment. 00:19:11.120 --> 00:19:13.839 It does not make any calls to Emacs anything. 00:19:13.840 --> 00:19:15.799 It does not have a single Elisp line. 00:19:15.800 --> 00:19:20.119 So this cache is stored outside. 00:19:20.120 --> 00:19:22.079 Now when I want to retrieve 00:19:22.080 --> 00:19:23.439 anything from this cache, 00:19:23.440 --> 00:19:26.199 let's say, so I have cached 00:19:26.200 --> 00:19:29.359 up until 55, from 45 to 55. 00:19:29.360 --> 00:19:31.079 So what happens is that 00:19:31.080 --> 00:19:32.759 when you're at page 50, 00:19:32.760 --> 00:19:34.359 you always have a cache 00:19:34.360 --> 00:19:36.719 that's n + 5 and n - 5. 00:19:36.720 --> 00:19:39.719 So you have cache of 5 pages forward 00:19:39.720 --> 00:19:41.959 and 5 pages backward. 00:19:41.960 --> 00:19:44.399 But let's say I want to go to page 56. 00:19:45.140 --> 00:19:50.079 So I will ask an Emacs render page 56. 00:19:50.080 --> 00:19:51.399 And I'm not going to ask it 00:19:51.400 --> 00:19:53.079 to MuPDF directly. 00:19:53.080 --> 00:19:54.399 I'm going to ask it 00:19:54.400 --> 00:19:56.719 to the thread pool that do this job. 00:19:56.720 --> 00:19:58.119 And thread pool is going to 00:19:58.120 --> 00:19:59.719 assign one thread to it. 00:19:59.720 --> 00:20:00.959 Let's say the thread 1 00:20:00.960 --> 00:20:03.239 which is going to render page 56. 00:20:03.240 --> 00:20:06.559 So this thread is going to make calls to MuPDF 00:20:06.560 --> 00:20:08.919 through our code dynamic module. 00:20:08.920 --> 00:20:11.839 And MuPDF after rendering it 00:20:11.840 --> 00:20:13.439 is going to store it in the cache. 00:20:13.440 --> 00:20:18.059 So we're going to add another 56 page to this. 00:20:18.060 --> 00:20:21.759 Now, while this is happening, 00:20:21.760 --> 00:20:24.679 Emacs Reader does not, like Emacs itself, 00:20:24.680 --> 00:20:27.379 the session is not going to be stuck 00:20:27.380 --> 00:20:30.239 because we just made a call to the thread. 00:20:30.240 --> 00:20:32.279 We just asked the thread. 00:20:32.280 --> 00:20:35.359 So like this, this call, like it's done. 00:20:35.360 --> 00:20:38.159 So you just assign something to a thread 00:20:38.160 --> 00:20:40.959 and then this is fine. 00:20:40.960 --> 00:20:42.479 Like, you're not waiting for the thread 00:20:42.480 --> 00:20:43.719 to complete or anything. 00:20:43.720 --> 00:20:46.519 Emacs is not waiting for the thread to complete. 00:20:46.520 --> 00:20:48.519 The dynamic module or the C side 00:20:48.520 --> 00:20:49.479 might wait to complete 00:20:49.480 --> 00:20:51.279 but that is entirely different from 00:20:51.280 --> 00:20:52.159 the Emacs session. 00:20:52.160 --> 00:20:54.839 So Emacs viewer can continue to 00:20:54.840 --> 00:20:56.279 display the page 50 00:20:56.280 --> 00:20:58.599 while the rendering pipeline 00:20:58.600 --> 00:21:01.979 is still rendering the 56th page. 00:21:01.980 --> 00:21:05.759 And when Emacs asks to display page 56, 00:21:05.760 --> 00:21:09.619 it's going to ask it to a thread pool. 00:21:09.620 --> 00:21:11.536 Then thread pool is going to assign 00:21:11.537 --> 00:21:13.319 another thread, let's say this one, 00:21:13.320 --> 00:21:16.999 to retrieve page 56 from the memory cache. 00:21:17.000 --> 00:21:20.039 And then the 56 page is going to be sent 00:21:20.040 --> 00:21:24.559 to the Emacs to be displayed. 00:21:24.560 --> 00:21:26.039 Again, the retrieval part 00:21:26.040 --> 00:21:28.519 is entirely independent of Emacs. 00:21:28.520 --> 00:21:30.159 Emacs does not have to wait for it. 00:21:30.160 --> 00:21:34.719 Emacs only needs to wait to display it. 00:21:34.720 --> 00:21:36.619 So, the displaying part 00:21:36.620 --> 00:21:37.919 and the rendering pipeline 00:21:37.920 --> 00:21:41.559 are entirely asynchronous, so to speak. 00:21:41.560 --> 00:21:43.639 And in the diagram, if you see, 00:21:43.640 --> 00:21:46.399 all the arrows that are 00:21:46.400 --> 00:21:48.839 magenta in color, 00:21:48.840 --> 00:21:51.639 they are native to the Emacs runtime. 00:21:51.640 --> 00:21:53.959 That is, they are single-threaded. 00:21:53.960 --> 00:21:55.679 They are connected to Emacs. 00:21:55.680 --> 00:21:58.759 And all the arrows that are red in color, 00:21:58.760 --> 00:22:01.859 they are totally asynchronous. 00:22:01.860 --> 00:22:03.519 They can be multi-threaded if you want. 00:22:03.520 --> 00:22:05.759 They are multi-threaded by default 00:22:05.760 --> 00:22:07.679 because they interact 00:22:07.680 --> 00:22:09.519 only with the MuPDF shared library 00:22:09.520 --> 00:22:11.399 and the C heap. 00:22:11.400 --> 00:22:12.719 They do not touch anything 00:22:12.720 --> 00:22:14.639 in the Emacs runtime. 00:22:14.640 --> 00:22:18.959 This is how we're able to switch quickly 00:22:18.960 --> 00:22:22.519 between these huge scanned PDFs 00:22:22.520 --> 00:22:23.959 that have huge images 00:22:23.960 --> 00:22:25.359 in each of their pages 00:22:25.360 --> 00:22:28.079 because we don't really wait for 00:22:28.080 --> 00:22:31.379 each page to be rendered. 00:22:31.380 --> 00:22:35.359 And Emacs does not wait for that. 00:22:35.360 --> 00:22:39.239 So that's another architectural feature 00:22:39.240 --> 00:22:40.319 of Emacs Reader 00:22:40.320 --> 00:22:43.199 that we are system-level multithreaded. 00:22:43.200 --> 00:22:47.399 Now Emacs viewer also supports 00:22:47.400 --> 00:22:49.319 almost all document formats. 00:22:49.320 --> 00:22:54.759 It supports PDF, EPUB, MOBI, XPS, CPZ comics, 00:22:54.760 --> 00:22:56.439 and it even supports 00:22:56.440 --> 00:22:59.970 other non-ebook formats 00:22:59.971 --> 00:23:00.839 like document format, 00:23:00.840 --> 00:23:01.839 so you can open 00:23:01.840 --> 00:23:04.799 LibreOffice documents in it, 00:23:04.800 --> 00:23:07.079 and even stuff like PPT and Excel in it, 00:23:07.080 --> 00:23:08.759 even though they're not going to be 00:23:08.760 --> 00:23:13.859 supported in a as nice manner. 00:23:13.860 --> 00:23:16.239 And we can do that because MuPDF does this. 00:23:16.240 --> 00:23:18.079 MuPDF has support for all of this 00:23:18.080 --> 00:23:22.679 and it treats them just as it treats PDF. 00:23:22.680 --> 00:23:24.539 Nothing special. 00:23:24.540 --> 00:23:26.519 The only thing that we don't support right now 00:23:26.520 --> 00:23:30.159 is DejaVu, so that is not supported right now. 00:23:30.160 --> 00:23:33.319 I'm going to work on making it supported 00:23:33.320 --> 00:23:35.079 at the upstream MuPDF. 00:23:36.020 --> 00:23:38.439 That's going to take a long time, 00:23:38.440 --> 00:23:44.239 but it's in the plans. NOTE Native Emacs integrations 00:23:44.240 --> 00:23:45.439 Now with Emacs Reader, 00:23:45.440 --> 00:23:46.679 we also integrate 00:23:46.680 --> 00:23:48.619 with existing Emacs packages 00:23:48.620 --> 00:23:50.039 as much as possible. 00:23:50.040 --> 00:23:52.999 So bookmarks, C-x r b, 00:23:53.000 --> 00:23:54.359 you can do it natively. 00:23:54.360 --> 00:23:57.559 So you can save a page as a bookmark 00:23:57.560 --> 00:23:59.639 just as you save anything else in Emacs 00:23:59.640 --> 00:24:00.519 as a bookmark. 00:24:00.520 --> 00:24:02.599 There's also saveplace integration. 00:24:02.600 --> 00:24:06.159 So you can scroll a PDF, close it, 00:24:06.160 --> 00:24:07.599 and then come back to it 00:24:07.600 --> 00:24:10.159 at the same page that you saved it at. 00:24:10.160 --> 00:24:12.879 Sorry, that you closed it at. 00:24:12.880 --> 00:24:14.919 And it's going to work just out of the box 00:24:14.920 --> 00:24:16.399 because of the saveplace 00:24:16.400 --> 00:24:18.999 package in Emacs that is built in. 00:24:19.000 --> 00:24:20.919 We also have imenu integration 00:24:20.920 --> 00:24:22.479 for table of contents. 00:24:22.480 --> 00:24:26.719 So if you see this, this is imenu 00:24:26.720 --> 00:24:28.679 and you can scroll through the contents 00:24:28.680 --> 00:24:30.559 just like you scroll through any imenu. 00:24:30.560 --> 00:24:39.499 You can also do it in the menu bar by clicking. 00:24:39.500 --> 00:24:40.679 It works just as nice. 00:24:40.680 --> 00:24:42.739 We also have something like 00:24:42.740 --> 00:24:44.799 the outline mode that pdf-tools has. 00:24:44.800 --> 00:24:48.039 So if you press O in a document, 00:24:48.040 --> 00:24:49.959 it's going to give you this outline. 00:24:49.960 --> 00:24:53.399 And these are buttons that are clickable. 00:24:53.400 --> 00:24:54.439 You can click them. 00:24:54.440 --> 00:24:56.519 You can press Enter at them. 00:24:56.520 --> 00:25:00.359 And this is the menu bar item that I was looking at. 00:25:00.360 --> 00:25:01.999 If you click here, index, 00:25:02.000 --> 00:25:03.279 it's going to show you 00:25:03.280 --> 00:25:05.339 the exact same thing 00:25:05.340 --> 00:25:10.339 but in a different interface. NOTE (Naive) dark mode 00:25:10.340 --> 00:25:15.259 We also have a naive dark mode, 00:25:15.260 --> 00:25:17.799 which is not really as nice as 00:25:17.800 --> 00:25:18.599 we would like it to be, 00:25:18.600 --> 00:25:20.799 and dark mode fanatics 00:25:20.800 --> 00:25:22.199 I'm sure will have issues with it, 00:25:22.200 --> 00:25:24.199 but we're going to improve it in time. 00:25:24.200 --> 00:25:27.379 For now, this is what we have. 00:25:27.380 --> 00:25:30.359 And it can be enabled per document, 00:25:30.360 --> 00:25:33.099 so you can have one, like, 00:25:33.100 --> 00:25:34.879 one document that is in dark mode, 00:25:34.880 --> 00:25:36.439 but another one that is not. 00:25:36.440 --> 00:25:39.279 That is nice to have. 00:25:39.280 --> 00:25:42.679 Eventually we're going to work on more themes. 00:25:42.680 --> 00:25:46.479 You should be able to actually integrate it 00:25:46.480 --> 00:25:49.439 with Emacs themes as much as possible. 00:25:49.440 --> 00:25:52.679 You can make it default so that 00:25:52.680 --> 00:25:54.839 it inherits colors from the Emacs theme. 00:25:54.840 --> 00:25:56.359 That is one of the things 00:25:56.360 --> 00:26:01.139 that we also have planned. NOTE Challenges and further improvements 00:26:01.140 --> 00:26:03.439 We did face a bunch of challenges 00:26:03.440 --> 00:26:05.519 while trying to implement these features. 00:26:05.520 --> 00:26:07.519 One of the initial challenges was that 00:26:07.520 --> 00:26:09.319 SVGs were actually a bad idea. 00:26:09.320 --> 00:26:12.159 They're huge, especially in scanned PDFs, 00:26:12.160 --> 00:26:14.679 and they make things much slower. 00:26:14.680 --> 00:26:18.119 So we chose to actually have PPMs, 00:26:18.120 --> 00:26:24.099 which is the simplest image format ever possible. 00:26:24.100 --> 00:26:26.439 Now, it was also very difficult 00:26:26.440 --> 00:26:29.559 to make reader-mode be window-specific. 00:26:29.560 --> 00:26:31.559 So, you know, while you're scrolling 00:26:31.560 --> 00:26:34.279 the same document in one window, 00:26:34.280 --> 00:26:36.199 the other window with the same document 00:26:36.200 --> 00:26:37.039 should not change. 00:26:37.040 --> 00:26:39.079 We should be able to have multiple pages 00:26:39.080 --> 00:26:42.319 in different windows of the same document. 00:26:42.320 --> 00:26:44.679 That was very difficult 00:26:44.680 --> 00:26:46.679 because as I told you about the cache, 00:26:46.680 --> 00:26:50.599 the cache works in an idiosyncratic manner 00:26:50.600 --> 00:26:54.079 and we needed to make it so that each window 00:26:54.080 --> 00:26:56.559 will have its own cache 00:26:56.560 --> 00:27:01.199 instead of having a global cache for each file. 00:27:01.200 --> 00:27:03.799 That took some rewrite. 00:27:03.800 --> 00:27:06.879 And now, because we needed to do 00:27:06.880 --> 00:27:07.799 this sort of multithreading, 00:27:07.800 --> 00:27:08.999 system-level multithreading, 00:27:09.000 --> 00:27:10.919 we needed to use 00:27:10.920 --> 00:27:13.039 a specific package of MuPDF 00:27:13.040 --> 00:27:16.439 that had a bug for this which got fixed. 00:27:16.440 --> 00:27:20.719 And that's 1.26.0. 00:27:20.720 --> 00:27:23.336 Because we did that, 00:27:23.337 --> 00:27:26.462 a lot of the GNU/Linux distributions did not 00:27:26.463 --> 00:27:28.871 really have this latest package. 00:27:28.872 --> 00:27:30.771 So we had to actually 00:27:30.772 --> 00:27:33.804 package it in-tree. 00:27:33.805 --> 00:27:36.971 as a git sub-module. 00:27:36.972 --> 00:27:40.737 That was a horror! But eventually... now 00:27:40.738 --> 00:27:43.604 I think most GNU/Linux distributions 00:27:43.605 --> 00:27:46.340 already have this [version]. 00:27:46.341 --> 00:27:48.639 The upcoming features that we have planned 00:27:48.640 --> 00:27:52.799 are the first one is that we need to rewrite 00:27:52.800 --> 00:27:55.359 the display mechanism entirely from scratch 00:27:55.360 --> 00:27:57.559 to use a tiled rendering approach. 00:27:57.560 --> 00:27:59.999 So right now we just take an image 00:28:00.000 --> 00:28:02.959 and display it inside an Emacs buffer 00:28:02.960 --> 00:28:03.959 just like that. 00:28:03.960 --> 00:28:08.759 But it will be changed so that the image 00:28:08.760 --> 00:28:10.759 will be displayed in the tiled manner 00:28:10.760 --> 00:28:12.479 so there will be multiple tiles 00:28:12.480 --> 00:28:14.719 but it'll be pixel perfect 00:28:14.720 --> 00:28:16.399 so you won't really see a difference. 00:28:16.400 --> 00:28:19.839 The reason to do this is to implement features 00:28:19.840 --> 00:28:20.999 for text selection, actually. 00:28:21.000 --> 00:28:24.239 So we can't really do text selection 00:28:24.240 --> 00:28:27.079 without running into a bunch of memory 00:28:27.080 --> 00:28:29.999 and other issues latency issues 00:28:30.000 --> 00:28:33.019 if we don't do tiling. 00:28:33.020 --> 00:28:35.679 So we need to do those two things, 00:28:35.680 --> 00:28:38.879 they are at the highest priority right now. 00:28:38.880 --> 00:28:40.279 And then, once we're done with that, 00:28:40.280 --> 00:28:42.279 we're going to support annotations, 00:28:42.280 --> 00:28:45.439 highlighting, everything that you're used to 00:28:45.440 --> 00:28:47.319 in pdf-tools and org-noter. 00:28:47.320 --> 00:28:50.119 And once we're done with that, 00:28:50.120 --> 00:28:55.019 we're going to also integrate with AucTeX and SyncTeX. 00:28:55.020 --> 00:28:58.519 Because right now, when a PDF gets updated, 00:28:58.520 --> 00:29:00.239 especially a LaTeX PDF, 00:29:00.240 --> 00:29:03.437 since there is no SyncTeX integration, 00:29:03.438 --> 00:29:05.771 it can't really do it nicely 00:29:05.772 --> 00:29:08.660 and it sometimes even crashes Emacs. 00:29:08.661 --> 00:29:11.537 So that's something that 00:29:11.538 --> 00:29:14.271 we will be planning to implement. NOTE What Emacs can learn? 00:29:14.272 --> 00:29:16.159 Now, from this experiment, 00:29:16.160 --> 00:29:17.919 what exactly can Emacs, 00:29:17.920 --> 00:29:20.519 the Emacs core devs and others 00:29:20.520 --> 00:29:22.399 who are building packages can learn? 00:29:22.400 --> 00:29:24.919 Well, the first thing is that all of this 00:29:24.920 --> 00:29:27.159 should not be really this difficult 00:29:27.160 --> 00:29:30.359 because all we're asking from Emacs 00:29:30.360 --> 00:29:32.439 is to display images in real-time 00:29:32.440 --> 00:29:36.279 and update them in real-time. 00:29:36.280 --> 00:29:37.759 That should not be that difficult 00:29:37.760 --> 00:29:40.279 of a thing to do, but apparently it is. 00:29:40.280 --> 00:29:43.279 And that's why Emacs's graphical interface 00:29:43.280 --> 00:29:47.959 needs to be more modular, more composable, 00:29:47.960 --> 00:29:50.999 and flexible for real-time graphics. 00:29:51.000 --> 00:29:54.219 If it is supposed to have things like, 00:29:54.220 --> 00:29:56.179 again, a document reader, 00:29:56.180 --> 00:29:57.279 something like a video editor, 00:29:57.280 --> 00:29:58.239 and something like that, 00:29:58.980 --> 00:30:00.479 Emacs's graphical interface 00:30:00.480 --> 00:30:05.239 needs to grow and be more mature. 00:30:05.240 --> 00:30:06.239 One of the things 00:30:06.240 --> 00:30:08.079 that's stopping it from doing that 00:30:08.080 --> 00:30:10.319 is actually Emacs's overlay functionality. 00:30:10.320 --> 00:30:13.939 So right now, the way we display 00:30:13.940 --> 00:30:16.519 an image in a buffer 00:30:16.520 --> 00:30:18.900 is using an overlay, 00:30:18.901 --> 00:30:22.019 actually multiple overlays. 00:30:22.020 --> 00:30:25.839 Overlays are static in the sense that 00:30:25.840 --> 00:30:29.739 if I attach to one image to one overlay, 00:30:29.740 --> 00:30:34.039 I need to have an entirely different image 00:30:34.040 --> 00:30:37.199 updated for that overlay. 00:30:37.200 --> 00:30:39.639 So I need to create another different image, 00:30:39.640 --> 00:30:41.179 change it in the memory, 00:30:41.180 --> 00:30:43.639 and then display it to update it. 00:30:43.640 --> 00:30:46.639 I can't change the image data 00:30:46.640 --> 00:30:49.239 in real time of the overlay. 00:30:49.240 --> 00:30:53.999 And that is a big issue. 00:30:54.000 --> 00:30:56.259 I've actually made an emacs-devel 00:30:56.260 --> 00:30:58.279 mailing list thread about it. 00:30:58.280 --> 00:31:01.119 I talked to Eli about it as well. 00:31:01.120 --> 00:31:04.639 And he said there's a possibility 00:31:04.640 --> 00:31:05.359 that this can be changed, 00:31:05.360 --> 00:31:06.959 but it's going to take 00:31:06.960 --> 00:31:09.919 a certain amount of rewrite. 00:31:09.920 --> 00:31:12.319 There's also issues with Emacs GC. 00:31:12.320 --> 00:31:14.639 Emacs GC sometimes leaks memory 00:31:14.640 --> 00:31:16.439 when you update images too quickly. 00:31:16.440 --> 00:31:18.599 That is, when you have a bunch of images 00:31:18.600 --> 00:31:21.359 that are getting churned out too quickly, 00:31:21.360 --> 00:31:23.039 Emacs GC starts leaking 00:31:23.040 --> 00:31:25.159 and it just goes up to 00:31:25.160 --> 00:31:29.679 a huge number of gigabytes in RAM. 00:31:29.680 --> 00:31:32.399 That's also a huge problem. 00:31:32.400 --> 00:31:33.759 The dynamic module API, 00:31:33.760 --> 00:31:37.139 the emacs-module.h header, 00:31:37.140 --> 00:31:38.799 needs to have more helpers. 00:31:38.800 --> 00:31:41.719 It's really bare bones, 00:31:41.720 --> 00:31:43.439 and I like that it is bare bones 00:31:43.440 --> 00:31:44.999 so that other languages can use it, 00:31:45.000 --> 00:31:46.959 but at the same time, I think 00:31:46.960 --> 00:31:47.879 it'll be really good 00:31:47.880 --> 00:31:49.839 if we can have some helpers 00:31:49.840 --> 00:31:53.879 that can do better memory interaction, 00:31:53.880 --> 00:31:57.259 like strings and so on, 00:31:57.260 --> 00:32:00.379 which we also faced some issues with. 00:32:00.380 --> 00:32:02.319 Emacs's fractional scaling system 00:32:02.320 --> 00:32:05.359 seems to be broken across different toolkits. 00:32:05.360 --> 00:32:10.999 We have bug reports that say in pgtk in Wayland, 00:32:11.000 --> 00:32:13.559 something seems to render differently 00:32:13.560 --> 00:32:17.259 because they have fractional scaling enabled. 00:32:17.260 --> 00:32:18.439 So that's something 00:32:18.440 --> 00:32:21.239 that I think Emacs, overall, 00:32:21.240 --> 00:32:24.359 I think Emacs needs to focus on improving 00:32:24.360 --> 00:32:28.239 the graphical interface pipeline 00:32:28.240 --> 00:32:32.299 to be a much more mature one. NOTE Contributing to the development 00:32:32.300 --> 00:32:34.239 And finally, how can you contribute 00:32:34.240 --> 00:32:35.799 to the development of Emacs Reader? 00:32:35.800 --> 00:32:37.359 Well, we are on Codeberg. 00:32:37.360 --> 00:32:40.279 We are not on GitHub, sorry. 00:32:40.280 --> 00:32:41.639 You can go there, 00:32:41.640 --> 00:32:43.079 you can look through the issues 00:32:43.080 --> 00:32:45.279 and send us a PR if you're interested. 00:32:45.280 --> 00:32:46.879 The next major release 00:32:46.880 --> 00:32:49.839 is going to go to GNU ELPA. 00:32:49.840 --> 00:32:52.259 Finally, we are not yet at GNU ELPA, 00:32:52.260 --> 00:32:54.439 so you can't really do M-x package-install 00:32:54.440 --> 00:32:56.119 and install our package. 00:32:56.120 --> 00:32:58.199 you would need to install it 00:32:58.200 --> 00:33:04.939 through use-package :vc. 00:33:04.940 --> 00:33:07.499 And since we're going to go to GNU ELPA, 00:33:07.500 --> 00:33:09.119 we request you to assign 00:33:09.120 --> 00:33:10.519 your copyright to Emacs 00:33:10.520 --> 00:33:13.959 because GNU ELPA is essentially part of GNU Emacs. 00:33:13.960 --> 00:33:16.719 So you would need to do copyright assignment 00:33:16.720 --> 00:33:20.579 if you make non-trivial contribution. 00:33:20.580 --> 00:33:22.479 You can join us at IRC 00:33:22.480 --> 00:33:24.359 at #phi-mu-lambda. 00:33:24.360 --> 00:33:27.199 And I also stream the development 00:33:27.200 --> 00:33:28.039 of this package 00:33:28.040 --> 00:33:29.839 bi-weekly on Sundays 00:33:29.840 --> 00:33:31.639 at PeerTube at the following channel. 00:33:31.640 --> 00:33:35.519 Feel free to join us. NOTE Acknowledgements 00:33:35.520 --> 00:33:38.499 Finally, I want to thank Tushar, 00:33:38.500 --> 00:33:40.639 who has been persistently contributing 00:33:40.640 --> 00:33:42.839 to the project since 0.1.0, 00:33:42.840 --> 00:33:46.519 and I'm very, very thankful for him, 00:33:46.520 --> 00:33:47.759 for his suggestions, 00:33:47.760 --> 00:33:50.879 and for his code contributions as well. 00:33:50.880 --> 00:33:53.319 I would also like to thank Prom, 00:33:53.320 --> 00:33:55.799 who fixed a major bug 00:33:55.800 --> 00:33:56.859 in the Windows build, 00:33:56.860 --> 00:33:58.839 since I don't really use Windows anymore, 00:33:58.840 --> 00:33:59.919 so that was really nice, 00:33:59.920 --> 00:34:05.459 and for Teeoius, for fixing a pthread bug. 00:34:05.460 --> 00:34:06.919 I would also like to thank others 00:34:06.920 --> 00:34:09.559 who helped fix little things, 00:34:09.560 --> 00:34:13.179 who come to the stream to chat, 00:34:13.180 --> 00:34:16.599 who sort of see me bang my head 00:34:16.600 --> 00:34:19.239 across these C memory bugs. 00:34:19.240 --> 00:34:21.599 So thank you to all of those. 00:34:21.600 --> 00:34:24.399 And thank you finally to the viewers 00:34:24.400 --> 00:34:28.079 and to EmacsConf organizers as well. 00:34:28.080 --> 00:34:31.939 This is a splendid opportunity. 00:34:31.940 --> 00:34:37.280 Thank you.