diff options
Diffstat (limited to '')
20 files changed, 14835 insertions, 0 deletions
diff --git a/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main--chapters.vtt b/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main--chapters.vtt new file mode 100644 index 00000000..a32fdf09 --- /dev/null +++ b/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main--chapters.vtt @@ -0,0 +1,77 @@ +WEBVTT + + +00:00:05.760 --> 00:01:20.079 +Introduction + +00:01:20.080 --> 00:02:10.319 +Scope: A complete multi-media content processing framework + +00:02:10.320 --> 00:03:02.419 +Prior art and similar art + +00:03:02.420 --> 00:03:57.159 +LaTeX-Beamer + Reveal.js with Blee and BISOS + +00:03:57.160 --> 00:05:12.519 +Blee-LCNT novel concepts + +00:05:12.520 --> 00:06:32.559 +Part of a bigger picture - part of a series + +00:06:32.560 --> 00:12:52.639 +Nature of polyexistentials + +00:12:52.640 --> 00:14:23.119 +Content processing - a ByStar/BISOS/Blee Capability Bundle (BCB) + +00:14:23.120 --> 00:14:31.279 +ByStar containment hierarchy and ByStar capability bundles + +00:14:31.280 --> 00:15:21.999 +Aggregated conviviality of ByStar capabilities + +00:15:22.000 --> 00:15:47.867 +Parts list: integrated components + +00:15:47.868 --> 00:18:45.719 +Resulting contents - output forms and formats + +00:18:45.720 --> 00:20:31.979 +reveal.js + +00:20:31.980 --> 00:21:33.479 +Generating the video + +00:21:33.480 --> 00:22:39.179 +A unified single input -- a sequencef of frames + +00:22:39.180 --> 00:23:16.199 +Abstractions to keep in mind + +00:23:16.200 --> 00:24:24.359 +Frame control types + +00:24:24.360 --> 00:26:25.199 +How outputs are generate from the inputs + +00:26:25.200 --> 00:27:46.479 +Context for unified source walkthrough + +00:27:46.480 --> 00:29:24.079 +One slide + +00:29:24.080 --> 00:31:05.799 +Dynamic blocks + +00:31:05.800 --> 00:33:42.279 +Internationalization - a non-Americanist perspective + +00:33:42.280 --> 00:35:07.719 +Autonomous self-publication and federated re-publications + +00:35:07.720 --> 00:36:02.559 +Ingredients of BISOS platforms and their progression + +00:36:02.560 --> 00:36:41.640 +Moving forward diff --git a/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main.vtt b/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main.vtt new file mode 100644 index 00000000..90f7b470 --- /dev/null +++ b/2025/captions/emacsconf-2025-blee-lcnt--bleelcnt-an-emacscentered-content-production-and-selfpublication-framework--mohsen-banan--main.vtt @@ -0,0 +1,2058 @@ +WEBVTT captioned by mohsen + +NOTE Introduction + +00:00:05.760 --> 00:00:08.159 +Greetings. Salaam. + +00:00:08.160 --> 00:00:10.159 +This is Mohsen Banan. + +00:00:10.160 --> 00:00:12.839 +I am a software and internet engineer. + +00:00:12.840 --> 00:00:14.679 +The title of this presentation + +00:00:14.680 --> 00:00:18.839 +is "Blee-LCNT: An Emacs Centered + +00:00:18.840 --> 00:00:23.659 +Content Production and Self-Publication Framework". + +00:00:23.660 --> 00:00:25.559 +Blee stands for + +00:00:25.560 --> 00:00:29.279 +ByStar Libre-Halaal Emacs Environment. + +00:00:29.280 --> 00:00:31.799 +In last year's EmacsConf, + +00:00:31.800 --> 00:00:36.079 +I introduced Blee, BISOS and ByStar + +00:00:36.080 --> 00:00:39.439 +as concepts and as foundations. + +00:00:39.440 --> 00:00:41.079 +This year I want to focus + +00:00:41.080 --> 00:00:43.879 +on one concrete capability. + +00:00:43.880 --> 00:00:47.959 +Content Production and Self-Publication + +00:00:47.960 --> 00:00:54.119 +is a foundational Blee and BISOS Capability Bundle. + +00:00:54.120 --> 00:00:55.759 +Both this presentation + +00:00:55.760 --> 00:00:59.079 +and the Nature of Polyexistentials book + +00:00:59.080 --> 00:01:02.879 +were developed with Blee-LCNT. + +00:01:02.880 --> 00:01:06.759 +In this presentation I want to look at Emacs + +00:01:06.760 --> 00:01:08.519 +as a central ingredient + +00:01:08.520 --> 00:01:10.959 +for a usage environment + +00:01:10.960 --> 00:01:14.919 +that we can use to orchestrate production of + +00:01:14.920 --> 00:01:20.079 +quite fancy multi-media presentations. + +NOTE Scope: A complete multi-media content processing framework + +00:01:20.080 --> 00:01:23.079 +Let's consider two different scopes. + +00:01:23.080 --> 00:01:27.919 +First, the scope of Blee-LCNT Capabilities Bundle, + +00:01:27.920 --> 00:01:29.919 +which is that of a complete + +00:01:29.920 --> 00:01:32.599 +multi-media content authorship, + +00:01:32.600 --> 00:01:34.799 +generation, publication + +00:01:34.800 --> 00:01:37.639 +and distribution framework. + +00:01:37.640 --> 00:01:40.999 +That complete scope is presented in this slide + +00:01:41.000 --> 00:01:44.239 +and it spans both black ink + +00:01:44.240 --> 00:01:46.639 +and violet ink. + +00:01:46.640 --> 00:01:49.799 +Second, the scope of this presentation, + +00:01:49.800 --> 00:01:52.119 +which is more limited. + +00:01:52.120 --> 00:01:54.919 +In this presentation I confine myself + +00:01:54.920 --> 00:01:58.519 +to the bullets is violet ink. + +00:01:58.520 --> 00:02:01.159 +Here, I focus on presentation + +00:02:01.160 --> 00:02:03.599 +and video as content types + +00:02:03.600 --> 00:02:05.999 +and their authorship and generation + +00:02:06.000 --> 00:02:10.319 +and their federated re-publication. + +NOTE Prior art and similar art + +00:02:10.320 --> 00:02:12.559 +This is a common topic. + +00:02:12.560 --> 00:02:14.839 +It makes good sense for us to start with + +00:02:14.840 --> 00:02:19.079 +a review of prior art and similar art. + +00:02:19.080 --> 00:02:21.959 +I went through the past EmacsConf talks + +00:02:21.960 --> 00:02:23.919 +and found a good number of them + +00:02:23.920 --> 00:02:25.999 +that also deal with the topic + +00:02:26.000 --> 00:02:28.839 +of content generation. + +00:02:28.840 --> 00:02:30.319 +A few of these are included + +00:02:30.320 --> 00:02:33.359 +in black ink in this slide. + +00:02:33.360 --> 00:02:35.599 +Many of these have chosen the Babel, + +00:02:35.600 --> 00:02:40.719 +in other words Org-Mode+LaTeX as primary input. + +00:02:40.720 --> 00:02:43.599 +I prefer the inverse of that. + +00:02:43.600 --> 00:02:45.839 +I also looked for past talks + +00:02:45.840 --> 00:02:49.999 +which have used Reveal.js and LaTeX-Beamer. + +00:02:50.000 --> 00:02:53.399 +For example, Sacha's use of Reveal.js + +00:02:53.400 --> 00:02:56.959 +is shown in violet inK. + +00:02:56.960 --> 00:03:02.419 +And Ihor's use of Beamer is in teal ink. + +NOTE LaTeX-Beamer + Reveal.js with Blee and BISOS + +00:03:02.420 --> 00:03:05.399 +This presentation is about a combination + +00:03:05.400 --> 00:03:08.639 +of Reveal.js and LaTeX-Beamer. + +00:03:08.640 --> 00:03:10.599 +For those who may not be familiar + +00:03:10.600 --> 00:03:12.619 +with Beamer and Reveal, + +00:03:12.620 --> 00:03:14.799 +here is a quick intro. + +00:03:14.800 --> 00:03:19.039 +Among academics, LaTeX-Beamer is the go-to tool + +00:03:19.040 --> 00:03:22.159 +for producing presentations. + +00:03:22.160 --> 00:03:24.239 +Reveal.js is recognized + +00:03:24.240 --> 00:03:25.919 +as the best of breed + +00:03:25.920 --> 00:03:29.919 +for dispensing HTML slide decks. + +00:03:29.920 --> 00:03:32.439 +For many, Reveal and Beamer + +00:03:32.440 --> 00:03:35.959 +live in different universes. + +00:03:35.960 --> 00:03:38.679 +Beamer is pdf oriented + +00:03:38.680 --> 00:03:42.019 +and Reveal is html oriented. + +00:03:42.020 --> 00:03:44.519 +Combining two powerful tools + +00:03:44.520 --> 00:03:48.359 +makes for an even more powerful tool. + +00:03:48.360 --> 00:03:51.879 +This Blee-LCNT Presentations combines + +00:03:51.880 --> 00:03:57.159 +the best of LaTeX-Beamer with Reveal.js. + +NOTE Blee-LCNT novel concepts + +00:03:57.160 --> 00:04:00.679 +Beamer primarily functions as producer + +00:04:00.680 --> 00:04:03.099 +and Reveal functions as dispenser + +00:04:03.100 --> 00:04:05.579 +and multi-media enhancer. + +00:04:05.580 --> 00:04:08.299 +Here is how the combination works. + +00:04:08.300 --> 00:04:10.439 +LaTeX Beamer pdf result + +00:04:10.440 --> 00:04:13.839 +is dissected into named frame images + +00:04:13.840 --> 00:04:18.799 +which can then be inserted in Reveal.js. + +00:04:18.800 --> 00:04:21.239 +LaTeX Beamer frames can also be + +00:04:21.240 --> 00:04:24.799 +translated into html with HeVeA + +00:04:24.800 --> 00:04:28.999 +which can also be inserted in Reveal.js. + +00:04:29.000 --> 00:04:31.119 +Voice-overs for Beamer frames + +00:04:31.120 --> 00:04:34.039 +can be correlated to frame names + +00:04:34.040 --> 00:04:37.119 +and applied to image or html frames. + +00:04:37.120 --> 00:04:42.079 +Screen captures and image narrations as videos + +00:04:42.080 --> 00:04:44.359 +can be directly dispensed + +00:04:44.360 --> 00:04:46.379 +through Reveal. + +00:04:46.380 --> 00:04:49.439 +There are various additional novel concepts + +00:04:49.440 --> 00:04:50.599 +with regard to the way + +00:04:50.600 --> 00:04:54.559 +that we have integrated all of this together. + +00:04:54.560 --> 00:04:57.599 +Instead of Org-Mode+LaTeX, + +00:04:57.600 --> 00:05:00.999 +we do LaTeX+Org-Mode. + +00:05:01.000 --> 00:05:03.999 +Instead of Babel, we do COMEEGA, + +00:05:04.000 --> 00:05:05.999 +instead of the Literate model + +00:05:06.000 --> 00:05:08.839 +we introduce the Surrounded model. + +00:05:08.840 --> 00:05:10.839 +You shall see various examples + +00:05:10.840 --> 00:05:12.519 +of these shortly. + +NOTE Part of a bigger picture - part of a series + +00:05:12.520 --> 00:05:15.639 +All of this is part of a bigger picture. + +00:05:15.640 --> 00:05:17.619 +A much bigger picture. + +00:05:17.620 --> 00:05:23.599 +My talks at EmacsConf 2021, 2022 + +00:05:23.600 --> 00:05:26.519 +and 2024 are related. + +00:05:26.520 --> 00:05:31.399 +This 2025 talk builds on those. + +00:05:31.400 --> 00:05:34.719 +Last year's talk "About Blee: + +00:05:34.720 --> 00:05:36.839 +enveloping our own autonomy + +00:05:36.840 --> 00:05:38.999 +directed digital ecosystem + +00:05:39.000 --> 00:05:42.199 +with Emacs" in particular, + +00:05:42.200 --> 00:05:44.979 +lays the foundations for this talk. + +00:05:44.980 --> 00:05:47.119 +If you have not seen that, + +00:05:47.120 --> 00:05:51.159 +it would make good sense to review it. + +00:05:51.160 --> 00:05:54.279 +In my previous talks I have been criticized + +00:05:54.280 --> 00:05:58.359 +of having a "prophetic" style. + +00:05:58.360 --> 00:06:02.059 +The scope of ByStar is lofty and immense. + +00:06:02.060 --> 00:06:04.879 +In many ways it is unbelievable. + +00:06:04.880 --> 00:06:09.139 +And EmacsConf talks are meant to be short. + +00:06:09.140 --> 00:06:11.839 +So, as a result, sometimes + +00:06:11.840 --> 00:06:13.959 +I end up being cryptic. + +00:06:13.960 --> 00:06:17.499 +Having accepted the "prophetic" criticism + +00:06:17.500 --> 00:06:19.399 +as legitimate, + +00:06:19.400 --> 00:06:23.599 +I now need to put a book on the table. + +00:06:23.600 --> 00:06:26.839 +With that book in place, moving forward, + +00:06:26.840 --> 00:06:29.339 +when needing to be cryptic, + +00:06:29.340 --> 00:06:32.559 +I shall cite Chapter and Verse. + +NOTE Nature of polyexistentials + +00:06:32.560 --> 00:06:34.879 +I am delighted to announce + +00:06:34.880 --> 00:06:37.559 +the availability of my recent book, + +00:06:37.560 --> 00:06:40.199 +"Nature of Polyexistentials". + +00:06:40.200 --> 00:06:42.959 +The full title of my book is: + +00:06:42.960 --> 00:06:45.039 +Nature Of Polyexistentials--- + +00:06:45.040 --> 00:06:48.239 +Basis For Abolishment Of The Western + +00:06:48.240 --> 00:06:51.219 +Intellectual Property Rights Regime--- + +00:06:51.220 --> 00:06:53.899 +And Introduction Of The Libre-Halaal + +00:06:53.900 --> 00:06:56.999 +ByStar Digital Ecosystem. + +00:06:57.000 --> 00:06:59.199 +Knowledge, know-how, uses of know-how, + +00:06:59.200 --> 00:07:02.879 +ideas, formulas, software and information + +00:07:02.880 --> 00:07:05.519 +are inherently non-scarce. + +00:07:05.520 --> 00:07:08.439 +They are *polyexistentials*. + +00:07:08.440 --> 00:07:10.239 +Unlike monoexistentials + +00:07:10.240 --> 00:07:12.259 +which exist in singular, + +00:07:12.260 --> 00:07:17.539 +polyexistentials naturally exist in multiples. + +00:07:17.540 --> 00:07:19.559 +What is abundant in nature + +00:07:19.560 --> 00:07:22.599 +is being made artificially scarce + +00:07:22.600 --> 00:07:25.399 +through man-made ownership rules + +00:07:25.400 --> 00:07:28.599 +called copyright and patents. + +00:07:28.600 --> 00:07:31.239 +These mistaken ownership rules, + +00:07:31.240 --> 00:07:34.959 +the so called Western IPR regime, + +00:07:34.960 --> 00:07:37.319 +has immense ramifications + +00:07:37.320 --> 00:07:38.839 +on the shape and the direction + +00:07:38.840 --> 00:07:42.619 +of the American Digital Ecosystem. + +00:07:42.620 --> 00:07:45.119 +It would be an understatement to say + +00:07:45.120 --> 00:07:47.779 +that the American Digital Ecosystem + +00:07:47.780 --> 00:07:50.599 +has put humanity in danger. + +00:07:50.600 --> 00:07:53.099 +Two parts of the book, in particular + +00:07:53.100 --> 00:07:55.679 +are of immediate relevance. + +00:07:55.680 --> 00:07:58.219 +Part III, the ethics layer, + +00:07:58.220 --> 00:08:01.119 +focuses on contours of cures. + +00:08:01.120 --> 00:08:02.839 +Having dismissed the Western + +00:08:02.840 --> 00:08:06.119 +intellectual property rights (IPR) regime + +00:08:06.120 --> 00:08:11.739 +as an erroneous governance model for polyexistentials, + +00:08:11.740 --> 00:08:14.319 +I propose the Libre-Halaal model + +00:08:14.320 --> 00:08:17.199 +of governance of polyexistentials + +00:08:17.200 --> 00:08:22.779 +towards facilitating conviviality of tools. + +00:08:22.780 --> 00:08:25.359 +Part IV, the engineering layer, + +00:08:25.360 --> 00:08:29.599 +introduces the Libre-Halaal ByStar Digital Ecosystem. + +00:08:29.600 --> 00:08:32.399 +as an ethical alternative + +00:08:32.400 --> 00:08:34.239 +to the prevailing proprietary + +00:08:34.240 --> 00:08:37.499 +American digital ecosystem. + +00:08:37.500 --> 00:08:40.479 +The book also provides additional details + +00:08:40.480 --> 00:08:42.919 +about the content generation + +00:08:42.920 --> 00:08:44.919 +and publication facilities + +00:08:44.920 --> 00:08:46.839 +that I am presenting here. + +00:08:46.840 --> 00:08:50.079 +And the book itself, as content, + +00:08:50.080 --> 00:08:53.439 +was generated and published + +00:08:53.440 --> 00:08:55.319 +using the facilities + +00:08:55.320 --> 00:08:57.239 +that I am presenting here. + +00:08:57.240 --> 00:08:59.199 +You can think of this book + +00:08:59.200 --> 00:09:01.159 +as being in two volumes. + +00:09:01.160 --> 00:09:05.919 +Our focus are Blee and BISOS in Volume II. + +00:09:05.920 --> 00:09:10.239 +Volume I deals with the general concept + +00:09:10.240 --> 00:09:13.879 +of polyexistence and invalidity + +00:09:13.880 --> 00:09:18.679 +of IPR and our terminoloy of Libre-Halaal--- + +00:09:18.680 --> 00:09:23.519 +instead of the common but ill directed vocabulary + +00:09:23.520 --> 00:09:28.239 +of Free Software and Open-Source and FOSS. + +00:09:28.240 --> 00:09:31.239 +In Chapter 11, I introduce + +00:09:31.240 --> 00:09:34.759 +the very sensitive and potent vocabulary + +00:09:34.760 --> 00:09:37.719 +of Halaal and Libre-Halaal. + +00:09:37.720 --> 00:09:39.079 +The contents of this book + +00:09:39.080 --> 00:09:41.659 +belong to all of humanity + +00:09:41.660 --> 00:09:45.519 +and verbatim copying of it is unrestricted. + +00:09:45.520 --> 00:09:49.479 +If you want to read it, this book is yours. + +00:09:49.480 --> 00:09:51.839 +The "Nature of Polyexistentials" book + +00:09:51.840 --> 00:09:56.659 +is available both online and in print. + +00:09:56.660 --> 00:09:59.439 +This book is available as two editions. + +00:09:59.440 --> 00:10:03.819 +The US Edition and the International edition. + +00:10:03.820 --> 00:10:05.959 +The US Edition is written + +00:10:05.960 --> 00:10:10.079 +with a slightly milder Western unfriendly tone, + +00:10:10.080 --> 00:10:12.399 +while the International Edition + +00:10:12.400 --> 00:10:17.619 +includes additional original content in Farsi. + +00:10:17.620 --> 00:10:20.399 +I consider the International Edition + +00:10:20.400 --> 00:10:22.979 +to be the authoritative version. + +00:10:22.980 --> 00:10:25.319 +However, many readers in + +00:10:25.320 --> 00:10:27.319 +the US and Western countries + +00:10:27.320 --> 00:10:31.199 +may prefer the US Edition. + +00:10:31.200 --> 00:10:33.999 +I maintain separate Git repositories + +00:10:34.000 --> 00:10:36.039 +for each edition on GitHub: + +00:10:36.040 --> 00:10:42.839 +US Edition is at bxplpc/120033 + +00:10:42.840 --> 00:10:51.419 +and International Edition: bxplpc/120074 + +00:10:51.420 --> 00:10:53.679 +Cloning these repositories + +00:10:53.680 --> 00:10:56.399 +will give you access to the book + +00:10:56.400 --> 00:11:00.039 +in PDF format (suitable for both + +00:11:00.040 --> 00:11:04.039 +A4 and US Letter printing) + +00:11:04.040 --> 00:11:06.379 +and in EPUB format. + +00:11:06.380 --> 00:11:08.559 +Alternatively, the content + +00:11:08.560 --> 00:11:12.039 +can be downloaded directly from your browser + +00:11:12.040 --> 00:11:17.259 +without needing to clone the repositories. + +00:11:17.260 --> 00:11:19.079 +To ensure broader online + +00:11:19.080 --> 00:11:21.899 +availability and stability, + +00:11:21.900 --> 00:11:26.159 +I have also published the book on Zenodo, + +00:11:26.160 --> 00:11:31.779 +complete with a DOI (Digital Object Identifier). + +00:11:31.780 --> 00:11:34.439 +You can download both the A4 + +00:11:34.440 --> 00:11:39.639 +and 8.5 x 11 PDFs from there as well. + +00:11:39.640 --> 00:11:44.119 +The book is also available in print on Amazon + +00:11:44.120 --> 00:11:46.239 +and at most major bookstores + +00:11:46.240 --> 00:11:49.379 +in the US and Western regions. + +00:11:49.380 --> 00:11:51.519 +The ISBNs for both editions + +00:11:51.520 --> 00:11:54.139 +are included in this slide. + +00:11:54.140 --> 00:11:56.319 +Additionally, I have published + +00:11:56.320 --> 00:12:00.719 +this book in Iran through Jangal Publishers. + +00:12:00.720 --> 00:12:03.079 +I did not write this book for profit. + +00:12:03.080 --> 00:12:05.359 +My aim is to share my thoughts + +00:12:05.360 --> 00:12:10.599 +and encourage readers to engage with my views and ideas. + +00:12:10.600 --> 00:12:12.499 +Your feedback is welcome, + +00:12:12.500 --> 00:12:14.119 +and I am genuinely interested + +00:12:14.120 --> 00:12:17.199 +in hearing your perspectives. + +00:12:17.200 --> 00:12:20.879 +In Western markets, I have priced the print edition + +00:12:20.880 --> 00:12:24.339 +somewhat above production costs. + +00:12:24.340 --> 00:12:26.639 +If you find value in the book + +00:12:26.640 --> 00:12:28.599 +and the ByStar project, + +00:12:28.600 --> 00:12:32.759 +purchasing a copy will help support my work. + +00:12:32.760 --> 00:12:37.459 +Thanks in advance for your support. + +00:12:37.460 --> 00:12:39.479 +And here are the same links + +00:12:39.480 --> 00:12:42.179 +as a native Reveal slide. + +00:12:42.180 --> 00:12:43.839 +If instead of a video, + +00:12:43.840 --> 00:12:47.759 +you are viewing this presentation as a Reveal web page, + +00:12:47.760 --> 00:12:52.639 +you can just click on the pointers and URLs. + +NOTE Content processing - a ByStar/BISOS/Blee Capability Bundle (BCB) + +00:12:52.640 --> 00:12:55.079 +Instead of the traditional model + +00:12:55.080 --> 00:12:59.559 +of giving you recipes in a DIY context + +00:12:59.560 --> 00:13:01.479 +towards the goal of creating + +00:13:01.480 --> 00:13:04.559 +content processing capabilities + +00:13:04.560 --> 00:13:07.659 +on top of what you may already have, + +00:13:07.660 --> 00:13:09.959 +I am doing the opposite. + +00:13:09.960 --> 00:13:15.159 +I am saying: take this whole BISOS and Blee thing, + +00:13:15.160 --> 00:13:17.559 +and in there you will also have + +00:13:17.560 --> 00:13:20.239 +the content processing capabilities + +00:13:20.240 --> 00:13:22.579 +that I am speaking of here. + +00:13:22.580 --> 00:13:24.919 +So, at the top level we have + +00:13:24.920 --> 00:13:27.519 +our own autonomy and privacy + +00:13:27.520 --> 00:13:30.199 +directed digital ecosystem, + +00:13:30.200 --> 00:13:32.839 +which in contrast to the center oriented + +00:13:32.840 --> 00:13:35.659 +American digital ecosystem, + +00:13:35.660 --> 00:13:38.479 +is edge oriented. + +00:13:38.480 --> 00:13:40.919 +We call it: "The Libre-Halaal + +00:13:40.920 --> 00:13:43.919 +ByStar Digital Ecosystem". + +00:13:43.920 --> 00:13:45.799 +All the systems in ByStar, + +00:13:45.800 --> 00:13:50.699 +run BISOS (By* Internet Services OS), + +00:13:50.700 --> 00:13:53.759 +which is a layer on top of Debian. + +00:13:53.760 --> 00:13:58.199 +The usage environment of ByStar and BISOS is Blee + +00:13:58.200 --> 00:14:01.579 +which is a layer on top of Emacs. + +00:14:01.580 --> 00:14:04.919 +With those in place, we then create + +00:14:04.920 --> 00:14:10.139 +a capability bundle called Blee-LCNT. + +00:14:10.140 --> 00:14:13.039 +So, when you buy into Blee and BISOS, + +00:14:13.040 --> 00:14:15.199 +you will naturally also get + +00:14:15.200 --> 00:14:18.719 +these content processing capabilities--- + +00:14:18.720 --> 00:14:23.119 +without a need for any recipies or DIY effort. + +NOTE ByStar containment hierarchy and ByStar capability bundles + +00:14:23.120 --> 00:14:24.879 +If you were to look at the model + +00:14:24.880 --> 00:14:29.119 +that I introduced as containment hierarchies, + +00:14:29.120 --> 00:14:31.279 +it would look like this. + +NOTE Aggregated conviviality of ByStar capabilities + +00:14:31.280 --> 00:14:33.779 +We love Emacs and we love Unix + +00:14:33.780 --> 00:14:36.759 +because their design is convivial. + +00:14:36.760 --> 00:14:39.199 +By convivial, I am referring + +00:14:39.200 --> 00:14:40.759 +to Ivan Illich's concept + +00:14:40.760 --> 00:14:45.319 +and terminology of "Tools for Conviviality". + +00:14:45.320 --> 00:14:48.679 +It was first published in 1973. + +00:14:48.680 --> 00:14:50.959 +It's a must read. + +00:14:50.960 --> 00:14:52.639 +A goal of the design + +00:14:52.640 --> 00:14:54.799 +of the ByStar Digital Ecosystem + +00:14:54.800 --> 00:14:57.479 +is to enlarge the aggregated + +00:14:57.480 --> 00:15:01.719 +conviviality of its capabilities. + +00:15:01.720 --> 00:15:04.719 +What distinguishes Blee-LCNT + +00:15:04.720 --> 00:15:08.959 +from other content processing tools and frameworks, + +00:15:08.960 --> 00:15:12.439 +is our emphasis on enhancing + +00:15:12.440 --> 00:15:15.659 +the aggregated conviviality. + +00:15:15.660 --> 00:15:19.259 +These tools let you express yourself. + +00:15:19.260 --> 00:15:21.999 +They let you be in charge. + +NOTE Parts list: integrated components + +00:15:22.000 --> 00:15:24.499 +Here is our parts list. + +00:15:24.500 --> 00:15:25.839 +These are the components + +00:15:25.840 --> 00:15:27.959 +that we have chosen to bring together + +00:15:27.960 --> 00:15:32.779 +towards our goal of creating convivial tools. + +00:15:32.780 --> 00:15:36.039 +In this slide, we are using black ink + +00:15:36.040 --> 00:15:38.519 +to denote exisiting tools + +00:15:38.520 --> 00:15:41.339 +and we use violet ink + +00:15:41.340 --> 00:15:44.419 +to denote pieces that we have developed + +00:15:44.420 --> 00:15:47.100 +towards cohesive integration. + +00:15:46.560 --> 00:15:47.867 +[This] video, + +NOTE Resulting contents - output forms and formats + +00:15:47.868 --> 00:15:51.479 +the video is just one of the outputs. + +00:15:51.480 --> 00:15:54.499 +There are other outputs as well. + +00:15:54.500 --> 00:15:56.359 +In this figure, the outputs + +00:15:56.360 --> 00:15:58.859 +are shown in the top layer. + +00:15:58.860 --> 00:16:02.279 +Using this video as an example, + +00:16:02.280 --> 00:16:05.599 +this presentation's output also include + +00:16:05.600 --> 00:16:07.599 +the "Presentation Form" + +00:16:07.600 --> 00:16:10.999 +and the "Article-Presentation Form". + +00:16:11.000 --> 00:16:13.719 +Let's look at these more closely. + +00:16:13.720 --> 00:16:17.259 +For Presentations, there are 3 different forms. + +00:16:17.260 --> 00:16:19.559 +The Video Form, the Presentation From + +00:16:19.560 --> 00:16:22.819 +and the Article-Presentation Form. + +00:16:22.820 --> 00:16:27.439 +The Presentation Form produces both a pdf output + +00:16:27.440 --> 00:16:29.079 +and Reveal output. + +00:16:29.080 --> 00:16:32.879 +Next we will walkthrough some of the benefits + +00:16:32.880 --> 00:16:35.519 +that availability of these forms + +00:16:35.520 --> 00:16:38.099 +and formats provide. + +00:16:38.100 --> 00:16:41.959 +The video presentation that you are watching + +00:16:41.960 --> 00:16:44.599 +is just one of the outputs + +00:16:44.600 --> 00:16:48.479 +of the Blee-LCNT machinery. + +00:16:48.480 --> 00:16:52.679 +There are two PDF format outputs + +00:16:52.680 --> 00:16:56.439 +and two HTML outputs + +00:16:56.440 --> 00:16:58.859 +that are also quite useful. + +00:16:58.860 --> 00:17:02.119 +The primary output of Beamer + +00:17:02.120 --> 00:17:04.239 +is a set of slides + +00:17:04.240 --> 00:17:10.439 +that people use to give their talks with. + +00:17:10.440 --> 00:17:12.479 +Typically that's done live. + +00:17:12.480 --> 00:17:19.179 +In my case I dissect the images of each frame + +00:17:19.180 --> 00:17:21.639 +and do a voiceover on it + +00:17:21.640 --> 00:17:28.839 +and then dispense it through reveal. + +00:17:28.840 --> 00:17:33.379 +In a second, you will see that as well. + +00:17:33.380 --> 00:17:36.959 +This PDF output is very useful. + +00:17:36.960 --> 00:17:39.279 +You get the table of contents, of course, + +00:17:39.280 --> 00:17:42.207 +and in addition to that, + +00:17:42.208 --> 00:17:46.319 +Beamer generates navigations for you + +00:17:46.320 --> 00:17:49.599 +where on any part you get + +00:17:49.600 --> 00:17:51.839 +a small table of content as well. + +00:17:51.840 --> 00:17:57.119 +This is heavily used amongst academics, + +00:17:57.120 --> 00:18:00.959 +and it's a good output on its own, + +00:18:00.960 --> 00:18:03.319 +and I'm augmenting it + +00:18:03.320 --> 00:18:05.399 +in a variety of ways. + +00:18:05.400 --> 00:18:09.719 +In addition to the presentation PDF format, + +00:18:09.720 --> 00:18:15.359 +there is also an article-presentation PDF format + +00:18:15.360 --> 00:18:18.799 +which gives you the same content, + +00:18:18.800 --> 00:18:25.159 +but it gives it to you in a textual form + +00:18:25.160 --> 00:18:30.939 +with the table of content and the rest. + +00:18:30.940 --> 00:18:34.759 +This is a good form to use + +00:18:34.760 --> 00:18:39.919 +when you are giving, for example, class lectures, + +00:18:39.920 --> 00:18:45.719 +and the students often prefer this format. + +NOTE reveal.js + +00:18:45.720 --> 00:18:51.839 +Now for the HTML format output, the most relevant, + +00:18:51.840 --> 00:18:55.599 +of course, is the reveal itself. + +00:18:55.600 --> 00:19:05.679 +If you have not used reveal before, + +00:19:05.680 --> 00:19:10.559 +in my view, it's a HTML slide dispenser. + +00:19:10.560 --> 00:19:15.479 +I don't look at it as a presentation framework. + +00:19:15.480 --> 00:19:22.599 +I use, as you are seeing, we use Beamer to feed into it + +00:19:22.600 --> 00:19:25.759 +and we use it to dispense the information. + +00:19:25.760 --> 00:19:33.439 +It has all the typical navigation + +00:19:33.440 --> 00:19:39.959 +capabilities that you would expect, + +00:19:39.960 --> 00:19:44.319 +and most of what I have as slides are images, + +00:19:44.320 --> 00:19:48.239 +but occasionally, particularly when there is a need + +00:19:48.240 --> 00:19:52.999 +to provide pointers, HTML pointers, + +00:19:53.000 --> 00:20:01.439 +I then also include a textual output. + +00:20:01.440 --> 00:20:05.559 +This is also produced + +00:20:05.560 --> 00:20:09.839 +from the Beamer LaTeX source, + +00:20:09.840 --> 00:20:14.959 +but it's HTML through textual HTML, + +00:20:14.960 --> 00:20:19.019 +through HeVeA, not the image. + +00:20:19.020 --> 00:20:22.499 +You can... you get a table of contents. + +00:20:22.500 --> 00:20:24.574 +You can navigate + +00:20:24.575 --> 00:20:28.079 +and there are a whole lot of other features + +00:20:28.080 --> 00:20:31.979 +that reveal also provides. + +NOTE Generating the video + +00:20:31.980 --> 00:20:35.879 +So to generate the video, + +00:20:35.880 --> 00:20:40.980 +what I do is I come to + +00:20:40.981 --> 00:20:49.459 +the very beginning of the presentation. + +00:20:49.460 --> 00:20:51.519 +I turn on the screen capture recorder, + +00:20:51.520 --> 00:20:54.159 +and then I start playing + +00:20:54.160 --> 00:20:58.239 +the voiceover for each slide + +00:20:58.240 --> 00:21:02.519 +and at the very end, you get a video, + +00:21:02.520 --> 00:21:08.759 +but what you just did is you dispensed every frame, + +00:21:08.760 --> 00:21:11.279 +one at a time, through reveal. + +00:21:11.280 --> 00:21:15.319 +In addition to this HTML form, + +00:21:15.320 --> 00:21:22.239 +you also get an article presentation form of it, + +00:21:22.240 --> 00:21:24.159 +with a full table of contents + +00:21:24.160 --> 00:21:27.759 +and the videos are there, and the notes are there, + +00:21:27.760 --> 00:21:33.479 +and this is also quite useful. + +NOTE A unified single input -- a sequencef of frames + +00:21:33.480 --> 00:21:36.519 +Now, let's look at the one single input file + +00:21:36.520 --> 00:21:38.879 +that produced all of the outputs + +00:21:38.880 --> 00:21:39.879 +that we just saw. + +00:21:39.880 --> 00:21:43.079 +I have put both the input file + +00:21:43.080 --> 00:21:45.119 +and some of the output files + +00:21:45.120 --> 00:21:48.299 +for this presentation on Github. + +00:21:48.300 --> 00:21:49.839 +Here are some links + +00:21:49.840 --> 00:21:51.679 +to these repos and files. + +00:21:51.680 --> 00:21:54.679 +And here are the same links + +00:21:54.680 --> 00:21:57.119 +as a native Reveal slide. + +00:21:57.120 --> 00:21:59.879 +This figure gives us an overview + +00:21:59.880 --> 00:22:02.759 +of how one set of inputs + +00:22:02.760 --> 00:22:04.959 +encapsulted in a single file + +00:22:04.960 --> 00:22:08.759 +can produce all of the outputs that we saw. + +00:22:08.760 --> 00:22:11.439 +The main TeX file shown at the bottom + +00:22:11.440 --> 00:22:15.659 +is processed by both XeLaTeX and by HeVeA. + +00:22:15.660 --> 00:22:18.279 +That main TeX file, in addition + +00:22:18.280 --> 00:22:19.679 +to LaTeX syntax, + +00:22:19.680 --> 00:22:22.999 +also include org-mode constructs + +00:22:23.000 --> 00:22:27.039 +that facilitate addition of audio and video files. + +00:22:27.040 --> 00:22:34.879 +Later, I'll walkthrough the bodyPresArtEnFa.tex file + +00:22:34.880 --> 00:22:39.179 +that generated this very presentation with you. + +NOTE Abstractions to keep in mind + +00:22:39.180 --> 00:22:42.679 +When you construct that primary TeX file, + +00:22:42.680 --> 00:22:44.679 +there are several abstractions + +00:22:44.680 --> 00:22:46.899 +that you need to keep in mind. + +00:22:46.900 --> 00:22:49.119 +Is my presentation going to go + +00:22:49.120 --> 00:22:52.739 +from Left-To-Right or from Right-To-Left? + +00:22:52.740 --> 00:22:57.039 +Perso-Arabic presentations go from Right-To-Left. + +00:22:57.040 --> 00:22:59.679 +Another consideration is the types + +00:22:59.680 --> 00:23:03.119 +of forms of results that you want. + +00:23:03.120 --> 00:23:05.019 +Just the presentation + +00:23:05.020 --> 00:23:08.999 +or Article-Presentation as well? + +00:23:09.000 --> 00:23:10.879 +With those choices in place + +00:23:10.880 --> 00:23:13.399 +you can produce condition based text + +00:23:13.400 --> 00:23:16.199 +for each of your desired outputs. + +NOTE Frame control types + +00:23:16.200 --> 00:23:18.919 +Think of this video presentation + +00:23:18.920 --> 00:23:20.879 +as a sequence of frames. + +00:23:20.880 --> 00:23:26.119 +Each frame is controlled by an org-mode dynamic block. + +00:23:26.120 --> 00:23:29.039 +This table lists available dblocks + +00:23:29.040 --> 00:23:31.559 +from which you can choose. + +00:23:31.560 --> 00:23:34.039 +For example, this particular frame + +00:23:34.040 --> 00:23:34.839 +that we are watching + +00:23:34.840 --> 00:23:41.979 +is controlled by b:lcnt:pres:frame/derivedImage. + +00:23:41.980 --> 00:23:44.639 +Beamer creates a pdf file + +00:23:44.640 --> 00:23:47.879 +that includes the image of this slide. + +00:23:47.880 --> 00:23:51.459 +That image is then injected into Reveal. + +00:23:51.460 --> 00:23:55.359 +And in the end, a video of that image is produced + +00:23:55.360 --> 00:23:57.239 +with the narrations + +00:23:57.240 --> 00:23:59.259 +that I am uttering right now. + +00:23:59.260 --> 00:24:02.199 +All of this has similarly been applied + +00:24:02.200 --> 00:24:03.599 +to each and every frame + +00:24:03.600 --> 00:24:05.919 +that you have been watching. + +00:24:05.920 --> 00:24:08.399 +Similar to Frame Controls, + +00:24:08.400 --> 00:24:10.719 +there are org-mode dynamic blocks + +00:24:10.720 --> 00:24:13.519 +for "Frame Body Types". + +00:24:13.520 --> 00:24:15.839 +You can easily insert an image + +00:24:15.840 --> 00:24:19.639 +which is typically created by OpenOffice Draw + +00:24:19.640 --> 00:24:21.619 +into a frame. + +00:24:21.620 --> 00:24:24.359 +Same with say a screen capture video. + +NOTE How outputs are generate from the inputs + +00:24:24.360 --> 00:24:29.319 +Now that we have looked at the "Outputs" and the "Inputs", + +00:24:29.320 --> 00:24:31.679 +let's look at how the Outputs + +00:24:31.680 --> 00:24:35.919 +are generated from the Inputs. + +00:24:35.920 --> 00:24:39.399 +Let's bootstrap Raw-BISOS and Raw-Blee. + +00:24:39.400 --> 00:24:41.719 +Starting from scratch, + +00:24:41.720 --> 00:24:45.799 +get yourself a fresh copy of Debian 12. + +00:24:45.800 --> 00:24:52.719 +Then go to https://github.com/bxGenesis/start . + +00:24:52.720 --> 00:24:55.079 +The README.org file + +00:24:55.080 --> 00:24:57.119 +of that github repo + +00:24:57.120 --> 00:24:58.639 +is same as Chapter 18, + +00:24:58.640 --> 00:25:01.959 +"Engineering Adoption of BISOS and ByStar" of the book. + +00:25:01.960 --> 00:25:05.359 +We will next run "raw-bisos.sh", + +00:25:05.360 --> 00:25:09.959 +but prior to that, let's take a quick look. + +00:25:09.960 --> 00:25:14.759 +This bootstrap scripts will do a lot as root + +00:25:14.760 --> 00:25:16.479 +on your Fresh-Debian. + +00:25:16.480 --> 00:25:18.599 +It is best to first try it + +00:25:18.600 --> 00:25:21.179 +on a disposable VM. + +00:25:21.180 --> 00:25:27.159 +raw-bisos.sh adds the current debian user to sudoers. + +00:25:27.160 --> 00:25:30.399 +Then it installs pipx. + +00:25:30.400 --> 00:25:34.199 +And then with pipx it installs + +00:25:34.200 --> 00:25:37.999 +from PyPI bisos.provision. + +00:25:38.000 --> 00:25:43.279 +bisos.provision includes additional bash scripts + +00:25:43.280 --> 00:25:45.359 +that are then executed. + +00:25:45.360 --> 00:25:48.159 +Full installation involves + +00:25:48.160 --> 00:25:51.039 +setting up various accounts, groups, + +00:25:51.040 --> 00:25:53.279 +various directory hierarchies, + +00:25:53.280 --> 00:25:55.439 +lots of apt packages + +00:25:55.440 --> 00:25:57.979 +and lots of python packages + +00:25:57.980 --> 00:26:01.499 +from the bisos namespace. + +00:26:01.500 --> 00:26:03.879 +If you are ready, copy and paste + +00:26:03.880 --> 00:26:06.599 +this line and run it. + +00:26:06.600 --> 00:26:08.039 +You will be prompted + +00:26:08.040 --> 00:26:09.619 +for the root password. + +00:26:09.620 --> 00:26:11.279 +Then be patient. + +00:26:11.280 --> 00:26:12.559 +Full installation + +00:26:12.560 --> 00:26:14.519 +can take 15 minutes or so. + +00:26:14.520 --> 00:26:17.079 +The logs of this script + +00:26:17.080 --> 00:26:18.519 +are also captured + +00:26:18.520 --> 00:26:25.199 +in ~/raw-bisos-${dateTag}-log.org + +NOTE Context for unified source walkthrough + +00:26:25.200 --> 00:26:28.959 +Now that we have Raw-BISOS and Raw-Blee installed, + +00:26:28.960 --> 00:26:31.039 +we are ready to walk through + +00:26:31.040 --> 00:26:32.319 +the unified source + +00:26:32.320 --> 00:26:34.439 +of the very presentation + +00:26:34.440 --> 00:26:36.259 +that you are watching. + +00:26:36.260 --> 00:26:40.959 +The "bodyPresArtEnFa.tex" file + +00:26:40.960 --> 00:26:42.439 +that we will visit + +00:26:42.440 --> 00:26:45.059 +is in COMEEGA-LaTeX syntax + +00:26:45.060 --> 00:26:47.699 +with lots of org-mode dblocks + +00:26:47.700 --> 00:26:50.479 +which generate Beamer-LaTeX frames + +00:26:50.480 --> 00:26:54.139 +and conditioned LaTeX bodies. + +00:26:54.140 --> 00:26:55.599 +After the walkthrough, + +00:26:55.600 --> 00:27:00.359 +I'll describe dblocks and COMEEGA in more detail. + +00:27:00.360 --> 00:27:02.239 +At the tail end of the walkthrough, + +00:27:02.240 --> 00:27:05.319 +we will also go through the generation process + +00:27:05.320 --> 00:27:10.859 +which runs XeLaTeX and HeVeA and a lot more. + +00:27:10.860 --> 00:27:13.619 +Let's look at our input file. + +00:27:13.620 --> 00:27:17.019 +It's a LaTeX file in LaTeX mode, + +00:27:17.020 --> 00:27:24.279 +and it has org syntax org-mode included in it, + +00:27:24.280 --> 00:27:29.559 +and I can toggle between LaTeX and org-mode. + +00:27:29.560 --> 00:27:33.599 +So, now I'm gonna be in org-mode, + +00:27:33.600 --> 00:27:37.839 +and org-mode gives me everything + +00:27:37.840 --> 00:27:39.399 +that org has to offer, + +00:27:39.400 --> 00:27:46.479 +including a very convenient navigation framework. + +NOTE One slide + +00:27:46.480 --> 00:27:54.279 +Let's take one slide and take a look at how it was done. + +00:27:54.280 --> 00:27:58.679 +So I would come to this scope slide + +00:27:58.680 --> 00:28:03.999 +and while I am there, I'm going to click on N. + +00:28:04.000 --> 00:28:09.759 +N takes me to the native LaTeX form back, + +00:28:09.760 --> 00:28:16.359 +so that I'll be looking at it not in org, but in LaTeX. + +00:28:16.360 --> 00:28:22.906 +So we're back in LaTeX, and as you can see + +00:28:22.907 --> 00:28:25.999 +it uses a dynamic block + +00:28:26.000 --> 00:28:30.799 +starting with the comments and the BEGIN, + +00:28:30.800 --> 00:28:34.839 +and it uses a dynamic block + +00:28:34.840 --> 00:28:38.079 +named a framedDrive image, + +00:28:38.080 --> 00:28:45.399 +which means the content of this frame + +00:28:45.400 --> 00:28:50.439 +will be dispensed as an image, not as text, + +00:28:50.440 --> 00:28:56.899 +and it also automatically creates for me + +00:28:56.900 --> 00:29:00.439 +a name, a label, that can be used + +00:29:00.440 --> 00:29:05.119 +for voiceover augmentation. + +00:29:05.120 --> 00:29:08.119 +So a file in the audio directory + +00:29:08.120 --> 00:29:13.039 +called ScopeOfBleeLcnt.mp3 + +00:29:13.040 --> 00:29:19.319 +is this audio that will come on top of this slide + +00:29:19.320 --> 00:29:24.079 +and then the rest is the LaTeX itself. + +NOTE Dynamic blocks + +00:29:24.080 --> 00:29:29.679 +The concept of "Org Dynamic Blocks" + +00:29:29.680 --> 00:29:31.519 +is very powerful. + +00:29:31.520 --> 00:29:33.599 +I think of them as universal + +00:29:33.600 --> 00:29:35.179 +visible macros. + +00:29:35.180 --> 00:29:41.359 +But, why should they be primarily used in just Org-Mode? + +00:29:41.360 --> 00:29:43.639 +I say, let's generalize them + +00:29:43.640 --> 00:29:46.059 +to "Emacs Dynamic Blocks". + +00:29:46.060 --> 00:29:49.959 +Have defaults for org-dblock-start-re + +00:29:49.960 --> 00:29:52.159 +in every relevant mode + +00:29:52.160 --> 00:29:55.099 +and use them everywhere. + +00:29:55.100 --> 00:29:56.319 +Blee does that. + +00:29:56.320 --> 00:30:01.719 +In COMEEGA-LaTeX, Dynamic Blocks create Frame Controls + +00:30:01.720 --> 00:30:05.519 +and insert Image and Video contents. + +00:30:05.520 --> 00:30:07.519 +Much of Blee and BISOS + +00:30:07.520 --> 00:30:09.959 +are implemented in COMEEGA. + +00:30:09.960 --> 00:30:13.599 +Almost all of our Elisp, Python, Bash + +00:30:13.600 --> 00:30:17.199 +and LaTeX work uses COMEEGA. + +00:30:17.200 --> 00:30:19.299 +COMEEGA stands for Collaborative + +00:30:19.300 --> 00:30:21.679 +Org-Mode + +00:30:21.680 --> 00:30:24.759 +Enhanced Emacs Generalized Authorship. + +00:30:24.760 --> 00:30:27.879 +It is the inverse of org-babel. + +00:30:27.880 --> 00:30:29.999 +COMEEGA adds org-mode + +00:30:30.000 --> 00:30:33.099 +to your programming mode. + +00:30:33.100 --> 00:30:35.079 +Full and proper use of COMEEGA, + +00:30:35.080 --> 00:30:38.299 +requires Polymode. + +00:30:38.300 --> 00:30:41.359 +Let's call that Poly-COMEEGA. + +00:30:41.360 --> 00:30:43.319 +But Emacs's Polymode + +00:30:43.320 --> 00:30:45.679 +is work-in-progress, + +00:30:45.680 --> 00:30:49.199 +particularly now with the new tree-sitter. + +00:30:49.200 --> 00:30:53.199 +So, in the interim, my usage of COMEEGA + +00:30:53.200 --> 00:30:55.919 +has been in the form of Toggle-COMEEGA. + +00:30:55.920 --> 00:30:59.479 +Where I manually switch between + +00:30:59.480 --> 00:31:02.359 +the programming-mode and org-mode. + +00:31:02.360 --> 00:31:04.199 +For me this has proved to be + +00:31:04.200 --> 00:31:05.799 +a fine interim solution. + +NOTE Internationalization - a non-Americanist perspective + +00:31:05.800 --> 00:31:09.679 +Naturally, content processing + +00:31:09.680 --> 00:31:11.239 +should be multi-lingual + +00:31:11.240 --> 00:31:14.159 +and internationalized. + +00:31:14.160 --> 00:31:15.839 +Let's look at that dimension. + +00:31:15.840 --> 00:31:21.019 +I am Iranian and much of what I write is in Farsi. + +00:31:21.020 --> 00:31:23.519 +Getting Perso-Arabic text right + +00:31:23.520 --> 00:31:25.519 +is often a challenge, + +00:31:25.520 --> 00:31:30.059 +as it involves Bi-Directional text (BIDI) + +00:31:30.060 --> 00:31:32.999 +and shaping of characters. + +00:31:33.000 --> 00:31:36.039 +In the context of our content generation + +00:31:36.040 --> 00:31:39.819 +these need to span all relevant tools, + +00:31:39.820 --> 00:31:41.759 +not just emacs. + +00:31:41.760 --> 00:31:43.759 +For emacs, I have created + +00:31:43.760 --> 00:31:46.239 +my own input method + +00:31:46.240 --> 00:31:49.419 +called farsi-transliterate-banan. + +00:31:49.420 --> 00:31:54.139 +My EmacsConf 2021 talk was about that. + +00:31:54.140 --> 00:31:57.199 +Now let's look at some examples + +00:31:57.200 --> 00:32:01.699 +and spice it up a bit with semantics. + +00:32:01.700 --> 00:32:05.279 +As an example of proper BIDI text, + +00:32:05.280 --> 00:32:07.899 +here is the orignal Farsi text + +00:32:07.900 --> 00:32:10.359 +along with English translation + +00:32:10.360 --> 00:32:12.519 +of Imam Khomeini's text + +00:32:12.520 --> 00:32:15.479 +with respect to invalidity + +00:32:15.480 --> 00:32:20.399 +of Western Intellectual Proprty Rights regime. + +00:32:20.400 --> 00:32:23.039 +And as another example + +00:32:23.040 --> 00:32:24.479 +of proper BIDI text, + +00:32:24.480 --> 00:32:29.919 +here is Ayatollah Mothari's take on Western IPR + +00:32:29.920 --> 00:32:35.159 +not being private property. Note that these predate + +00:32:35.160 --> 00:32:36.919 +by more than half a century + +00:32:36.920 --> 00:32:43.239 +Jack Dorsey and Elon Musk's tweets of April 11, 2025 + +00:32:43.240 --> 00:32:47.199 +saying "Delete all IP law". + +00:32:47.200 --> 00:32:49.159 +This topic is too important + +00:32:49.160 --> 00:32:50.399 +and too sensitive + +00:32:50.400 --> 00:32:53.639 +to be left to American billionaires + +00:32:53.640 --> 00:32:55.639 +and their tweets. + +00:32:55.640 --> 00:32:58.199 +Let me again refer you to the logic + +00:32:58.200 --> 00:33:00.599 +of polyexistentials in my book. + +00:33:00.600 --> 00:33:06.359 +Chapter 14 of the book is dedicated to + +00:33:06.360 --> 00:33:08.579 +Ethics and ownership in Religions. + +00:33:08.580 --> 00:33:10.919 +With respect to my preference + +00:33:10.920 --> 00:33:12.719 +for Ethics over Freedom, + +00:33:12.720 --> 00:33:16.519 +let me refer you to Section 12.4 + +00:33:16.520 --> 00:33:19.079 +"A Cynical Perspective + +00:33:19.080 --> 00:33:22.859 +on Freedom Orientation of Americans" + +00:33:22.860 --> 00:33:25.999 +in which I describe where the FOSS labels + +00:33:26.000 --> 00:33:29.039 +and the likes of Stallman, Raymond, + +00:33:29.040 --> 00:33:31.599 +Moglen and Lessig have gone wrong. + +00:33:31.600 --> 00:33:34.239 +If you are one of their followers, + +00:33:34.240 --> 00:33:36.599 +perhaps Chapter 12 is for you. + +00:33:36.600 --> 00:33:42.279 +My emphasis thus far has been on content generation. + +NOTE Autonomous self-publication and federated re-publications + +00:33:42.280 --> 00:33:44.999 +Let's very briefly also look at + +00:33:45.000 --> 00:33:47.159 +Autonomous Self-Publication + +00:33:47.160 --> 00:33:52.279 +and Federated Re-Publications of our content. + +00:33:52.280 --> 00:33:55.759 +From the very beginning the Debian folks + +00:33:55.760 --> 00:33:59.039 +understood the importance of "Universality" + +00:33:59.040 --> 00:34:03.359 +and coined the "Universal Debian" label. + +00:34:03.360 --> 00:34:05.919 +This means that we can base + +00:34:05.920 --> 00:34:08.619 +our entire digital ecosystem + +00:34:08.620 --> 00:34:13.499 +on just the Libre-Halaal Debian distro. + +00:34:13.500 --> 00:34:17.299 +And that is what we have done with ByStar. + +00:34:17.300 --> 00:34:20.039 +In ByStar, everything is based on + +00:34:20.040 --> 00:34:24.119 +just the Universal Debian everywhere. + +00:34:24.120 --> 00:34:26.999 +This has made our Usage Environment + +00:34:27.000 --> 00:34:31.319 +totally harmonious with our Service Environment + +00:34:31.320 --> 00:34:38.059 +allowing for very powerful software-service continuums. + +00:34:38.060 --> 00:34:41.479 +Of course, all of this is immediately applicable + +00:34:41.480 --> 00:34:46.019 +to our ByStar Content Bundle as well. + +00:34:46.020 --> 00:34:50.519 +Some have asked, why don't you also include Ubuntu? + +00:34:50.520 --> 00:34:53.679 +I think the opposite makes more sense. + +00:34:53.680 --> 00:34:56.699 +Ubuntu should converge with Debian. + +00:34:56.700 --> 00:34:59.639 +I tried to explain this to Mark Shuttleworth + +00:34:59.640 --> 00:35:02.479 +in an email a while back. + +00:35:02.480 --> 00:35:04.119 +I have included that email + +00:35:04.120 --> 00:35:07.719 +in Section 12.1.5. + +NOTE Ingredients of BISOS platforms and their progression + +00:35:07.720 --> 00:35:10.439 +In this presentation, we have stopped + +00:35:10.440 --> 00:35:13.159 +at the "Raw-BISOS" stage. + +00:35:13.160 --> 00:35:15.759 +We can further evolve Raw-BISOS + +00:35:15.760 --> 00:35:17.959 +and make it be "Sited" + +00:35:17.960 --> 00:35:22.239 +and provide autonomous publication services. + +00:35:22.240 --> 00:35:25.679 +But here by going through EmacsConf and youtube + +00:35:25.680 --> 00:35:30.959 +we are using the "Federated Re-Publications" model. + +00:35:30.960 --> 00:35:32.479 +Something this large, + +00:35:32.480 --> 00:35:35.479 +should be well documented. + +00:35:35.480 --> 00:35:37.079 +In Emacs, the way that + +00:35:37.080 --> 00:35:39.319 +we have been dealing with documentation + +00:35:39.320 --> 00:35:43.439 +and information retrieval is archaic. + +00:35:43.440 --> 00:35:46.079 +Man-pages, TeXInfo, Helpful-Mode + +00:35:46.080 --> 00:35:51.599 +and convention based Doc-Strings are old and limited. + +00:35:51.600 --> 00:35:55.279 +In BISOS and Blee, we use Blee-Panels + +00:35:55.280 --> 00:35:57.739 +for all kinds of documentation. + +00:35:57.740 --> 00:36:02.559 +Let me show you some examples. + +NOTE Moving forward + +00:36:02.560 --> 00:36:05.199 +So, what next? + +00:36:05.200 --> 00:36:10.599 +If Blee, BISOS, ByStar, Libre-Halaal, Polyexistentials + +00:36:10.600 --> 00:36:14.159 +and these Content Processing capabilities + +00:36:14.160 --> 00:36:16.639 +have piqued your interest, + +00:36:16.640 --> 00:36:19.379 +please feel welcome to contact me. + +00:36:19.380 --> 00:36:22.239 +These Emacs Conferences have proven + +00:36:22.240 --> 00:36:25.379 +to be very useful and productive. + +00:36:25.380 --> 00:36:27.199 +I look forward to your thoughts, + +00:36:27.200 --> 00:36:29.599 +feedback and questions. + +00:36:29.600 --> 00:36:35.359 +I want to thank all the EmacsConf 2025 Organizers + +00:36:35.360 --> 00:36:37.199 +for their great work, + +00:36:37.200 --> 00:36:41.640 +and Sacha in particular. diff --git a/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main--chapters.vtt b/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main--chapters.vtt new file mode 100644 index 00000000..f3af8a6f --- /dev/null +++ b/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main--chapters.vtt @@ -0,0 +1,41 @@ +WEBVTT + + +00:00:03.620 --> 00:02:36.639 +Introduction + +00:02:36.640 --> 00:04:54.279 +What is Calc? + +00:04:54.280 --> 00:06:37.398 +calc-algebraic-entry + +00:06:37.399 --> 00:08:07.759 +calc-roll-down + +00:08:07.760 --> 00:08:58.179 +Advanced functions + +00:08:58.180 --> 00:09:54.719 +Solving equations with calc-solve-for + +00:09:54.720 --> 00:12:00.079 +Systems of equations + +00:12:00.080 --> 00:12:39.959 +calc-find-root + +00:12:39.960 --> 00:14:17.539 +Derivatives and integrals + +00:14:17.540 --> 00:18:12.159 +Programmable functions + +00:18:12.160 --> 00:20:08.799 +Plotting + +00:20:08.800 --> 00:22:38.599 +Wish list + +00:22:38.600 --> 00:23:35.920 +Wrapping up diff --git a/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main.vtt b/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main.vtt new file mode 100644 index 00000000..f0bf2d2a --- /dev/null +++ b/2025/captions/emacsconf-2025-calc--basic-calc-functionality-for-engineering-or-electronics--christopher-howard--main.vtt @@ -0,0 +1,888 @@ +WEBVTT captioned by sachac + +NOTE Introduction + +00:00:03.620 --> 00:00:08.799 +Hello, my name is Christopher Howard and welcome to my talk. + +00:00:08.800 --> 00:00:11.319 +This is basically an introduction + +00:00:11.320 --> 00:00:15.119 +to the built-in Emacs calculator, + +00:00:15.120 --> 00:00:18.319 +properly known as Emacs Calc, + +00:00:18.320 --> 00:00:21.439 +particularly from the perspective of someone + +00:00:21.440 --> 00:00:27.559 +with a technical background such as engineering or electronics. + +00:00:27.560 --> 00:00:32.879 +I will say, though, my personal interest is not really + +00:00:32.880 --> 00:00:37.839 +in digital computing or digital calculators, + +00:00:37.840 --> 00:00:42.519 +but lately has been focused more on analog computing. + +00:00:42.520 --> 00:00:46.799 +I have, for example, been working to master + +00:00:46.800 --> 00:00:50.839 +the venerable slide rule, a mechanical computer + +00:00:50.840 --> 00:00:57.319 +that calculates multiplication powers and logarithms. + +00:00:57.320 --> 00:01:02.199 +Here's a picture of one. + +00:01:02.200 --> 00:01:06.799 +It's a physical tool that was used for hundreds of years + +00:01:06.800 --> 00:01:08.999 +for this sort of thing + +00:01:09.000 --> 00:01:16.679 +before the handheld calculator was made popular. + +00:01:16.680 --> 00:01:18.639 +And I also had a project that I did + +00:01:18.640 --> 00:01:21.119 +for a while to several months + +00:01:21.120 --> 00:01:33.119 +to build an electronic analog computer. + +00:01:33.120 --> 00:01:38.679 +A rudimentary attempt of mine, but it's functional, + +00:01:38.680 --> 00:01:43.399 +and it's basically a 1960s or 1970s style + +00:01:43.400 --> 00:01:48.839 +electronic analog computer built very much on a budget, + +00:01:48.840 --> 00:01:52.559 +but the box in the middle is the computer proper + +00:01:52.560 --> 00:01:55.719 +which has most of the components inside of it + +00:01:55.720 --> 00:02:00.199 +as well as the potentiometers for setting values, + +00:02:00.200 --> 00:02:02.039 +and an operation switch. + +00:02:02.040 --> 00:02:04.399 +There's a patch panel on the left + +00:02:04.400 --> 00:02:07.119 +for connecting the different integrators, + +00:02:07.120 --> 00:02:11.319 +amplifiers, multipliers, and so forth together. + +00:02:11.320 --> 00:02:16.919 +Then the output of the simulation is displayed + +00:02:16.920 --> 00:02:19.799 +on the oscilloscope on the right side, + +00:02:19.800 --> 00:02:25.479 +which is a digital oscilloscope. + +00:02:25.480 --> 00:02:28.439 +To be honest, I think that a talk about analog computing + +00:02:28.440 --> 00:02:30.199 +would be much more interesting + +00:02:30.200 --> 00:02:32.039 +than the talk that I'm about to give, + +00:02:32.040 --> 00:02:36.639 +but unfortunately that would be out of scope for EmacsConf. + +NOTE What is Calc? + +00:02:36.640 --> 00:02:39.919 +So instead I will talk about Emacs Calc, + +00:02:39.920 --> 00:02:43.359 +the digital calculator built into Emacs. + +00:02:43.360 --> 00:02:47.519 +Emacs Calc, while not being a replacement for software + +00:02:47.520 --> 00:02:51.479 +like GNU Octave, does have advanced calculator functionality + +00:02:51.480 --> 00:02:55.039 +that can be useful in engineering, electronics, + +00:02:55.040 --> 00:03:00.759 +or other technical applications. So I don't want to oversell it, + +00:03:00.760 --> 00:03:06.479 +but I think functionality-wise, Calc is somewhere in between + +00:03:06.480 --> 00:03:12.239 +what you'd expect of a decent scientific calculator + +00:03:12.240 --> 00:03:23.939 +and an advanced graphics calculator. + +00:03:23.940 --> 00:03:28.839 +So this talk I'll mention is not intended to be a tutorial + +00:03:28.840 --> 00:03:33.839 +but only a brief introduction to Calc. + +00:03:33.840 --> 00:03:37.439 +Please refer to the built-in Calc info manual + +00:03:37.440 --> 00:03:46.739 +for detailed instructions on how to complete operations. + +00:03:46.740 --> 00:04:01.479 +Turn off my volume here. + +00:04:01.480 --> 00:04:05.719 +The documentation for Emacs Calc is built-in, + +00:04:05.720 --> 00:04:10.439 +although on some distributions you may have to install + +00:04:10.440 --> 00:04:24.479 +the Emacs documentation separately for licensing reasons. + +00:04:24.480 --> 00:04:28.599 +Calc presents itself as a stack-based calculator + +00:04:28.600 --> 00:04:31.599 +where entries are dropped onto a stack + +00:04:31.600 --> 00:04:36.739 +and then an operation is performed on the stack entries. + +00:04:36.740 --> 00:04:42.899 +For example, I can drop 1.23 onto the stack, + +00:04:42.900 --> 00:04:54.279 +and then 8.56, and then multiply them together. + +NOTE calc-algebraic-entry + +00:04:54.280 --> 00:05:01.559 +It may present itself as a stack-based calculator, + +00:05:01.560 --> 00:05:05.399 +but indeed, Calc is also capable of accepting input + +00:05:05.400 --> 00:05:07.739 +in the more well-known algebraic format + +00:05:07.740 --> 00:05:10.759 +by using the calc-algebraic-entry command, + +00:05:10.760 --> 00:05:14.999 +which by default is bound to the apostrophe (') key. + +00:05:15.000 --> 00:05:19.759 +So you type the apostrophe key, enter the algebraic input, + +00:05:19.760 --> 00:05:22.759 +including parentheses as needed. + +00:05:22.760 --> 00:05:28.199 +For example, here's a calculation of the resonance frequency + +00:05:28.200 --> 00:05:35.039 +of a coil which has an inductance of 250 microhenries + +00:05:35.040 --> 00:05:41.059 +and 160 picofarads, taken from one of my electronics handbooks. + +00:05:41.060 --> 00:05:50.019 +The formula for that is 1 over 2 pi + +00:05:50.020 --> 00:05:57.439 +and then the square root of our inductance + +00:05:57.440 --> 00:06:06.279 +which is in this case 250 microfarads - excuse me, microhenries + +00:06:06.280 --> 00:06:19.399 +and then the capacitance is 160 picofarads. + +00:06:19.400 --> 00:06:24.399 +Small typo here. + +00:06:24.400 --> 00:06:26.639 +Now I need to evaluate that one more time, + +00:06:26.640 --> 00:06:30.919 +because pi is a symbol. + +00:06:30.920 --> 00:06:37.398 +I get about 800 kHz resonant frequency. + +NOTE calc-roll-down + +00:06:37.399 --> 00:06:41.679 +The command calc-roll-down, + +00:06:41.680 --> 00:06:44.199 +which by default is bound to the TAB key, + +00:06:44.200 --> 00:06:47.919 +will swap the top two stack entries, + +00:06:47.920 --> 00:06:51.559 +which is sometimes useful if you need to manipulate something + +00:06:51.560 --> 00:06:56.999 +that's further down the stack. + +00:06:57.000 --> 00:07:02.039 +So I can swap this around and say multiply by two + +00:07:02.040 --> 00:07:05.479 +and then put it back where it was. + +00:07:05.480 --> 00:07:14.039 +This command is also capable of rolling the entire stack. + +00:07:14.040 --> 00:07:18.899 +Say I want to shift them all around. + +00:07:18.900 --> 00:07:21.399 +This can be done by passing extra arguments + +00:07:21.400 --> 00:07:23.559 +to the calc-roll-down function. + +00:07:23.560 --> 00:07:28.279 +That's a little bit inconvenient to do manually, + +00:07:28.280 --> 00:07:40.079 +so in my init file, I defined here a key definition + +00:07:40.080 --> 00:07:45.759 +that passes in those arguments correctly. + +00:07:45.760 --> 00:07:49.179 +I attached this to shift-tab, + +00:07:49.180 --> 00:07:52.319 +so this way, I can roll the entire stack. + +00:07:52.320 --> 00:07:56.159 +Then I could change one entry here + +00:07:56.160 --> 00:08:03.459 +and then put it back where it was. + +00:08:03.460 --> 00:08:07.759 +So Calc does algebraic input. + +NOTE Advanced functions + +00:08:07.760 --> 00:08:10.159 +It also does advanced functions + +00:08:10.160 --> 00:08:15.599 +that you would expect any handheld scientific calculator, + +00:08:15.600 --> 00:08:19.159 +including trigonometric functions. + +00:08:19.160 --> 00:08:25.319 +For example, we can get the sine of a number. + +00:08:25.320 --> 00:08:30.719 +Now I'll mention here that Calc has multiple modes. + +00:08:30.720 --> 00:08:32.319 +Right now it's in degree mode. + +00:08:32.320 --> 00:08:38.159 +You can switch over to radian mode if you want. + +00:08:38.160 --> 00:08:42.799 +I'm going to put it back in degrees. + +00:08:42.800 --> 00:08:49.799 +Drop 12 degrees on the stack, and then get the sine of that. + +00:08:49.800 --> 00:08:58.179 +And then with the inverse sine function, I can put it back. + +NOTE Solving equations with calc-solve-for + +00:08:58.180 --> 00:09:07.519 +Calc also has the nifty ability to solve equations for you + +00:09:07.520 --> 00:09:13.919 +so long as the equation is not too complicated. + +00:09:13.920 --> 00:09:19.959 +This is using the calc-solve-for function. + +00:09:19.960 --> 00:09:31.699 +For example, we could enter in an equation algebraically, + +00:09:31.700 --> 00:09:36.679 +then run calc-solve-for, and we just have to tell it + +00:09:36.680 --> 00:09:40.999 +what variable we want to solve for. And there we go. + +00:09:41.000 --> 00:09:43.199 +We can do this manually as well + +00:09:43.200 --> 00:09:54.719 +just so you can see that we get the same result. + +NOTE Systems of equations + +00:09:54.720 --> 00:09:57.959 +Calc is also able to solve systems of equations. + +00:09:57.960 --> 00:10:03.439 +We can put more than one equation on the stack, + +00:10:03.440 --> 00:10:08.959 +and then solve for several variables. + +00:10:08.960 --> 00:10:13.319 +To give a technical example for this, + +00:10:13.320 --> 00:10:30.659 +I'll show you a resistor network scribble that I did recently. + +00:10:30.660 --> 00:10:32.819 +Hopefully you can see that. Basically, + +00:10:32.820 --> 00:10:38.719 +it's fairly simple, a pretty simple resistor network + +00:10:38.720 --> 00:10:42.159 +with 1 kilo ohm and 10 kilo ohm resistors, + +00:10:42.160 --> 00:10:48.959 +and using the loop methods, we are calculating the currents, + +00:10:48.960 --> 00:10:52.759 +the current in each loop, and then that current can be used + +00:10:52.760 --> 00:10:58.839 +to solve for the voltage of each individual resistor + +00:10:58.840 --> 00:11:06.199 +if we want to. So at the bottom there we have the equations + +00:11:06.200 --> 00:11:11.519 +that we come up with as we work through each loop. + +00:11:11.520 --> 00:11:19.579 +And I'm going to paste that into Calc. + +00:11:19.580 --> 00:11:22.719 +To save some time, I'm going to copy and paste that + +00:11:22.720 --> 00:11:34.259 +from my notes instead of typing it out. + +00:11:34.260 --> 00:11:38.259 +So we have two equations there on the stack + +00:11:38.260 --> 00:11:44.719 +in one stack entry. We run that calc-solve-for function again, + +00:11:44.720 --> 00:11:49.899 +and we tell it which variables we want to solve for. + +00:11:49.900 --> 00:11:51.959 +And voila! Those are our currents, + +00:11:51.960 --> 00:11:55.719 +which we can then use to get the voltages + +00:11:55.720 --> 00:12:00.079 +for the individual resistors. + +NOTE calc-find-root + +00:12:00.080 --> 00:12:01.999 +I'll just briefly mention + +00:12:02.000 --> 00:12:05.839 +that if Calc is not able to solve an equation + +00:12:05.840 --> 00:12:07.779 +with calc-solve-for, + +00:12:07.780 --> 00:12:10.279 +then you might be helped by another calc function + +00:12:10.280 --> 00:12:11.559 +called calc-find-root. + +00:12:11.560 --> 00:12:14.439 +This function basically does a manual search + +00:12:14.440 --> 00:12:30.199 +for a numerical solution to the equation. + +00:12:30.200 --> 00:12:39.959 +And there's the documentation page on that. + +NOTE Derivatives and integrals + +00:12:39.960 --> 00:12:44.039 +Calc can also solve or find derivatives of functions, + +00:12:44.040 --> 00:12:47.579 +at least the more straightforward functions. + +00:12:47.580 --> 00:12:49.839 +For a simple example, + +00:12:49.840 --> 00:13:00.559 +we can get the derivative of that + +00:13:00.560 --> 00:13:11.979 +with the derivative function. + +00:13:11.980 --> 00:13:17.159 +On the other hand, Calc is also capable of figuring out + +00:13:17.160 --> 00:13:22.099 +indefinite integrals. + +00:13:22.100 --> 00:13:26.859 +Say we put that function back on the stack, + +00:13:26.860 --> 00:13:32.559 +and this time, we call the integral function. + +00:13:32.560 --> 00:13:35.079 +There you go. Of course, you have to add + +00:13:35.080 --> 00:13:39.819 +your own constant of integration. + +00:13:39.820 --> 00:13:43.399 +For integrals that Calc cannot figure out symbolically, + +00:13:43.400 --> 00:13:46.079 +a numerical integration method is available + +00:13:46.080 --> 00:13:59.998 +through the calc-num-integral command, which is documented... + +00:13:59.999 --> 00:14:17.539 +The function documentation is available here, more or less. + +NOTE Programmable functions + +00:14:17.540 --> 00:14:20.399 +I definitely need to mention + +00:14:20.400 --> 00:14:24.759 +that Calc is capable of doing programmable functions. + +00:14:24.760 --> 00:14:29.619 +That is to say, you can program your own functions into Calc. + +00:14:29.620 --> 00:14:32.239 +There are three separate ways to do this. + +00:14:32.240 --> 00:14:36.279 +One is through a macro method + +00:14:36.280 --> 00:14:41.539 +similar to Emacs's usual keyboard macros. + +00:14:41.540 --> 00:14:46.519 +The second method is to transform an algebraic function + +00:14:46.520 --> 00:14:50.859 +into a stored function definition. + +00:14:50.860 --> 00:14:54.059 +And the third is to use Elisp directly. + +00:14:54.060 --> 00:14:56.599 +Personally, I find that the second method + +00:14:56.600 --> 00:15:01.799 +is the most practical, the most convenient and practical + +00:15:01.800 --> 00:15:08.059 +in my opinion. So I'll give a quick example of that. + +00:15:08.060 --> 00:15:14.159 +So I could... Let's say I wanted to have a function + +00:15:14.160 --> 00:15:20.699 +for calculating capacitive reactance. + +00:15:20.700 --> 00:15:28.899 +I'll define that in algebraic mode first. + +00:15:28.900 --> 00:15:33.639 +The function for that is 1 over 2 pi + +00:15:33.640 --> 00:15:41.599 +the frequency and the capacitance. + +00:15:41.600 --> 00:15:44.959 +Drop that on the stack. You see, it does automatically + +00:15:44.960 --> 00:15:52.079 +get simplified a little bit, but it's the same function. + +00:15:52.080 --> 00:15:58.839 +And then I press letters Z and F. Do that again. + +00:15:58.840 --> 00:16:06.239 +Z and F to start transforming that into a stored function. + +00:16:06.240 --> 00:16:11.039 +It asks me to select a user key, a single key press. + +00:16:11.040 --> 00:16:15.479 +I'll use the letter c. + +00:16:15.480 --> 00:16:19.079 +Then it's going to ask for a longer command name. + +00:16:19.080 --> 00:16:24.639 +I've actually defined this once before, so it prefilled in + +00:16:24.640 --> 00:16:38.339 +that command name. + +00:16:38.340 --> 00:16:42.999 +Then I need to enter which variables in the formula + +00:16:43.000 --> 00:16:46.559 +are actual arguments, rather than just symbols + +00:16:46.560 --> 00:16:52.559 +to be evaluated later. I prefer to put this in with frequency + +00:16:52.560 --> 00:16:54.279 +and the capacitance after that, + +00:16:54.280 --> 00:16:57.799 +but actually in this particular case, + +00:16:57.800 --> 00:17:07.339 +it doesn't matter at all to the mathematics. + +00:17:07.340 --> 00:17:11.399 +So, now all I have to do, that this is defined, + +00:17:11.400 --> 00:17:15.199 +is I can drop the frequency on the stack, + +00:17:15.200 --> 00:17:24.399 +which we'll say, for this example, will be 4.5 MHz, + +00:17:24.400 --> 00:17:32.279 +and then drop on the capacitance, which in this example + +00:17:32.280 --> 00:17:40.319 +will be 22 pF. + +00:17:40.320 --> 00:17:42.439 +Then I'll call the function that I just defined. + +00:17:42.440 --> 00:17:45.239 +I don't really like having to try to remember + +00:17:45.240 --> 00:17:48.679 +the short letters that I've come up with, + +00:17:48.680 --> 00:17:57.839 +so I'll just use the longer name. + +00:17:57.840 --> 00:17:59.799 +I need to evaluate one more time + +00:17:59.800 --> 00:18:05.619 +because the symbol pi is in there and not yet evaluated. + +00:18:05.620 --> 00:18:07.539 +And so if I've done that right, + +00:18:07.540 --> 00:18:12.159 +we have a capacitive reactance of about 1600 ohms. + +NOTE Plotting + +00:18:12.160 --> 00:18:16.839 +As the last feature that I'll mention here, + +00:18:16.840 --> 00:18:24.059 +Emacs Calc does have an interface with gnuplot, + +00:18:24.060 --> 00:18:30.799 +if you want to have Calc work as your graphing calculator. + +00:18:30.800 --> 00:18:33.159 +I do need to be honest and mention + +00:18:33.160 --> 00:18:35.579 +that I don't generally use it myself + +00:18:35.580 --> 00:18:39.719 +because there's another program in GNOME + +00:18:39.720 --> 00:18:43.499 +that I've found to be generally more convenient + +00:18:43.500 --> 00:18:47.399 +for the things that I want to graph quickly. + +00:18:47.400 --> 00:18:53.399 +But I think I can give you a simple example. + +00:18:53.400 --> 00:19:00.339 +So first, we need to drop a range on the stack. + +00:19:00.340 --> 00:19:06.619 +Let's say 0 to 10. + +00:19:06.620 --> 00:19:11.639 +And then we need to drop the function on the stack. + +00:19:11.640 --> 00:19:17.839 +And then I believe it's the letters g and f that graph this. + +00:19:17.840 --> 00:19:22.319 +Let's see. Yep, there we go. + +00:19:22.320 --> 00:19:25.059 +So there's our function and it looks nice. + +00:19:25.060 --> 00:19:26.659 +That was pretty easy. + +00:19:26.660 --> 00:19:29.019 +That's the fast way to do it. + +00:19:29.020 --> 00:19:32.839 +I will, as a disclaimer, mention that + +00:19:32.840 --> 00:19:34.159 +using this quick approach, + +00:19:34.160 --> 00:19:38.759 +that sometimes more complicated graphs + +00:19:38.760 --> 00:19:39.999 +will not turn out nicely, + +00:19:40.000 --> 00:19:44.339 +because by default, the resolution will be pretty low. + +00:19:44.340 --> 00:19:48.119 +That is to say it's... gnuplot is going to be + +00:19:48.120 --> 00:19:49.899 +skipping a lot of points + +00:19:49.900 --> 00:19:52.039 +and so you'll have to learn a bit more + +00:19:52.040 --> 00:19:55.319 +about how to use the interface, + +00:19:55.320 --> 00:19:59.519 +what parameters to pass if you want all your graphs + +00:19:59.520 --> 00:20:03.699 +to come out looking nice. + +00:20:03.700 --> 00:20:08.799 +So that covers all the features that I wanted to cover. + +NOTE Wish list + +00:20:08.800 --> 00:20:13.279 +I wanted to briefly mention a wish list of items + +00:20:13.280 --> 00:20:16.679 +that I'd like to see in Calc. + +00:20:16.680 --> 00:20:23.639 +One of them would be improper integrals. + +00:20:23.640 --> 00:20:25.159 +So that's like our definite integrals + +00:20:25.160 --> 00:20:32.859 +except for where a limit of integration is infinity. + +00:20:32.860 --> 00:20:38.559 +That's something that can be useful in a few applications. + +00:20:38.560 --> 00:20:41.079 +Something else that would be neat to have would be + +00:20:41.080 --> 00:20:45.679 +annotations for row entries. So for example + +00:20:45.680 --> 00:20:48.819 +if I was putting together a sum of numbers + +00:20:48.820 --> 00:20:53.279 +for, say, my monthly budget, + +00:20:53.280 --> 00:20:57.479 +let's say I was paying $2,000 for my rent + +00:20:57.480 --> 00:21:03.831 +and let's say $800 a month for my groceries, + +00:21:03.832 --> 00:21:07.931 +(a lot of kids to feed there) + +00:21:07.932 --> 00:21:14.565 +and then say another $60 for dining out, and so on, + +00:21:14.566 --> 00:21:18.259 +it would be nice if there was some way + +00:21:18.260 --> 00:21:21.319 +to put a little annotation next to each number + +00:21:21.320 --> 00:21:23.399 +so that you could remember + +00:21:23.400 --> 00:21:27.039 +what the meaning of that number was more easily. + +00:21:27.040 --> 00:21:31.199 +I actually looked into programming this into Calc myself, + +00:21:31.200 --> 00:21:35.919 +but discovered that it would require reprogramming + +00:21:35.920 --> 00:21:41.839 +quite a bit of Calc to make that work well + +00:21:41.840 --> 00:21:43.479 +across all calc functionality, + +00:21:43.480 --> 00:21:46.939 +and so, eventually, I gave up. + +00:21:46.940 --> 00:21:51.139 +But I'd still really like to have that feature. + +00:21:51.140 --> 00:21:52.039 +The final thing, though + +00:21:52.040 --> 00:21:54.579 +I think this would not necessarily belong in Calc, + +00:21:54.580 --> 00:21:57.919 +I think it would be cool if Emacs had some way + +00:21:57.920 --> 00:22:00.599 +to run numerical solutions + +00:22:00.600 --> 00:22:02.599 +for systems of differential equations, + +00:22:02.600 --> 00:22:06.019 +also known as a differential analyzer. + +00:22:06.020 --> 00:22:09.279 +So this would allow you to be able to set up simulation models + +00:22:09.280 --> 00:22:11.679 +involving systems of differential equations, + +00:22:11.680 --> 00:22:14.879 +for example, a spring mass system, or pressure temperature, + +00:22:14.880 --> 00:22:18.039 +or what have you, and then run the simulation + +00:22:18.040 --> 00:22:22.119 +using numerical approximation. + +00:22:22.120 --> 00:22:24.079 +Maybe it would be silly + +00:22:24.080 --> 00:22:25.999 +to actually put that in Calc itself, + +00:22:26.000 --> 00:22:30.339 +but a nice interface maybe to some other software, + +00:22:30.340 --> 00:22:33.299 +simple software that did that, + +00:22:33.300 --> 00:22:35.779 +an easy to use interface for that + +00:22:35.780 --> 00:22:38.599 +would be really great. + +NOTE Wrapping up + +00:22:38.600 --> 00:22:41.800 +So that's my entire talk. + +00:22:41.801 --> 00:22:44.534 +I'll just mention some information. + +00:22:44.535 --> 00:22:48.365 +If you want to learn more about me + +00:22:48.366 --> 00:22:50.119 +or things that I'm interested in, + +00:22:50.120 --> 00:22:57.779 +I do not any longer have a web presence. + +00:22:57.780 --> 00:22:59.659 +I don't have a website anymore, + +00:22:59.660 --> 00:23:03.359 +but I do have a Gemini capsule + +00:23:03.360 --> 00:23:07.139 +that I post to all the time. + +00:23:07.140 --> 00:23:13.879 +And if you can install, if you're willing to install the... + +00:23:13.880 --> 00:23:19.079 +Gemini browser known as Elpher + +00:23:19.080 --> 00:23:23.698 +into Emacs, which is available from ELPA, + +00:23:23.699 --> 00:23:27.359 +then you can browse directly to it + +00:23:27.360 --> 00:23:31.439 +and look around my Gemini capsule. + +00:23:31.440 --> 00:23:35.920 +Thank you very much. diff --git a/2025/captions/emacsconf-2025-commonlisp--common-lisp-images-communicating-likeahuman-through-shared-emacs-slime-and-eev--screwlisp--main.vtt b/2025/captions/emacsconf-2025-commonlisp--common-lisp-images-communicating-likeahuman-through-shared-emacs-slime-and-eev--screwlisp--main.vtt new file mode 100644 index 00000000..2ad2f285 --- /dev/null +++ b/2025/captions/emacsconf-2025-commonlisp--common-lisp-images-communicating-likeahuman-through-shared-emacs-slime-and-eev--screwlisp--main.vtt @@ -0,0 +1,1260 @@ +WEBVTT captioned by jay_bird + +NOTE Introduction + +00:00:00.000 --> 00:00:07.119 +Hey, everyone. This talk is on this tradition, + +00:00:07.120 --> 00:00:10.639 +intelligent agents in Emacs + +00:00:10.640 --> 00:00:13.799 +using my Leonardo software individuals, + +00:00:13.800 --> 00:00:16.919 +which I've mistyped as I just wrote here, I see. + +00:00:16.920 --> 00:00:20.159 +Thank you to Sacha and everyone + +00:00:20.160 --> 00:00:25.239 +at EmacsConf and Emacs, I guess. + +00:00:25.240 --> 00:00:26.599 +Sorry that I was running late. + +00:00:26.600 --> 00:00:29.759 +I'm screwlisp.small-web.org. + +00:00:29.760 --> 00:00:33.999 +I run those one or two weekly shows for a long time, + +00:00:34.000 --> 00:00:35.599 +the Lispy Gopher Climate. + +00:00:35.600 --> 00:00:42.199 +I'm active on the Mastodon at @screwlisp@gamerplus.org. + +00:00:42.200 --> 00:00:46.719 +I'm screwtape on lambda.moo.mud.org. + +00:00:46.720 --> 00:00:50.474 +And I ported, over the last kind of year, + +00:00:50.475 --> 00:00:58.499 +years, to some extent, I ported Eric Sandewall's system + +00:00:58.500 --> 00:01:01.519 +for developing intelligent software agents, + +00:01:01.520 --> 00:01:04.879 +which he finished working on in 2014. + +00:01:04.880 --> 00:01:10.119 +I got it working again around 2025. + +00:01:10.120 --> 00:01:14.199 +First, we're going to take a long arc. + +00:01:14.200 --> 00:01:16.759 +We're going to motivate... This is the idea. + +00:01:16.760 --> 00:01:18.119 +You can see I'm using Org Mode, + +00:01:18.120 --> 00:01:19.959 +which I hope provides a good example + +00:01:19.960 --> 00:01:25.359 +for all the Org-Mode-oriented talks this conference. + +00:01:25.360 --> 00:01:26.399 +But you can also see + +00:01:26.400 --> 00:01:33.107 +that I'm using Eduardo Ochs's eev minor mode with Org. + +00:01:33.108 --> 00:01:35.640 +But we can see a little bit of the difference + +00:01:35.641 --> 00:01:39.207 +between these two, and that will kind of evolve into + +00:01:39.208 --> 00:01:45.259 +my style with the agent communication in Emacs. + +00:01:45.260 --> 00:01:52.999 +So you can see I used eev anchors as my Emacs headings. + +00:01:53.000 --> 00:01:56.839 +In eev, you just evaluate Elisp expressions + +00:01:56.840 --> 00:01:58.679 +as links to places. + +00:01:58.680 --> 00:02:01.679 +An anchor will link you somewhere else in the document. + +00:02:01.680 --> 00:02:04.807 +So my table of contents links to my talk, I guess. + +00:02:04.808 --> 00:02:07.507 +Anchors come in two halves, + +00:02:07.508 --> 00:02:12.940 +so that's why I built that unique table of contents + +00:02:12.941 --> 00:02:21.479 +experience there. What else am I going to say? + +NOTE Totally normal computing + +00:02:21.480 --> 00:02:24.174 +So first, let's just do some totally normal computing + +00:02:24.175 --> 00:02:27.140 +because intelligence is going to be difficult to describe. + +00:02:27.141 --> 00:02:31.100 +Let's just try and compute normally in Emacs in Org Mode + +00:02:31.101 --> 00:02:34.359 +and then segue more so into eev, + +00:02:34.360 --> 00:02:38.359 +and then maybe I would like if an agent was intelligent, + +00:02:38.360 --> 00:02:40.839 +I would think that an intelligent agent + +00:02:40.840 --> 00:02:43.319 +would do something like what I'm doing. + +00:02:43.320 --> 00:02:47.239 +It should be recognizably similar to what I do myself. + +00:02:47.240 --> 00:02:52.399 +I don't think the word intelligence is relevant + +00:02:52.400 --> 00:02:55.679 +if it's not related to something I'm not familiar with. + +NOTE Using Emacs as a human + +00:02:55.680 --> 00:03:00.999 +Using Emacs as a human, reading headings from my article, + +00:03:01.000 --> 00:03:03.919 +using Common Lisp. Right, my friend jeremy_list + +00:03:03.920 --> 00:03:06.879 +wrote actually a big project, + +00:03:06.880 --> 00:03:09.799 +but part of it was base64 encoding, + +00:03:09.800 --> 00:03:17.439 +and I just yoinked his C code for base64 encoding, I think. + +00:03:17.440 --> 00:03:20.759 +This is just clearly some C-based 64 encoding. + +00:03:20.760 --> 00:03:24.279 +If you go to my blog, his project is actually a C++ project + +00:03:24.280 --> 00:03:29.579 +and you can see me doing this with C++ rather than C. + +00:03:29.580 --> 00:03:33.319 +But basically, you can go to my blog articles + +00:03:33.320 --> 00:03:40.299 +if you want more detail to read something instead. + +00:03:40.300 --> 00:03:42.433 +And then here's some embeddable Common Lisp, + +00:03:42.434 --> 00:03:48.439 +Jack Daniel's ECL ANSI Common Lisp compiler I guess. + +00:03:48.440 --> 00:03:49.639 +This is just what it looks like. + +00:03:49.640 --> 00:03:52.239 +You can see I'm using Org Mode trickily, + +00:03:52.240 --> 00:03:56.119 +using noweb to put the lines of the C source block + +00:03:56.120 --> 00:04:00.279 +in this one. We're tangling it to this file + +00:04:00.280 --> 00:04:01.919 +rather than evaluating it. + +00:04:01.920 --> 00:04:05.279 +So, you know, literate programming, tangle and weave. + +00:04:05.280 --> 00:04:06.999 +We're just using Org Mode + +00:04:07.000 --> 00:04:09.197 +like the other Org Mode people + +00:04:09.198 --> 00:04:12.079 +are all showing us this conference, I guess. + +00:04:12.080 --> 00:04:13.399 +Then we have to compile it. + +00:04:13.400 --> 00:04:16.039 +It's always hard to remember these invocations for me. + +00:04:16.040 --> 00:04:20.159 +Results file. The file is my .fas file, + +00:04:20.160 --> 00:04:24.559 +because the way ECL's C and C++ integration works + +00:04:24.560 --> 00:04:30.519 +is that it just has to be seen by compile-file in Lisp. + +00:04:30.520 --> 00:04:32.119 +I cached this earlier. + +00:04:32.120 --> 00:04:36.199 +Oh, I should actually start Lisp, actually, shouldn't I? + +00:04:36.200 --> 00:04:39.639 +How are we going to do this? + +00:04:39.640 --> 00:04:47.099 +(setq inferior-lisp-program "ecl"). We could M-x slime. + +00:04:47.100 --> 00:04:48.919 +Because... we better actually load this. + +00:04:48.920 --> 00:04:54.119 +I did a dry run before. + +00:04:54.120 --> 00:04:58.259 +I think we can just load this, because I already did it. + +00:04:58.260 --> 00:05:04.079 +But I cached it. Let's nuke the cache. + +00:05:04.080 --> 00:05:06.599 +Okay, I'm going to say that that probably worked. + +00:05:06.600 --> 00:05:09.319 +Now, as you saw, that base64 encoding + +00:05:09.320 --> 00:05:13.619 +was just, I guess, number to character code + +00:05:13.620 --> 00:05:19.140 +to other character code. So I wrote this higher-level Lisp one, + +00:05:19.141 --> 00:05:20.599 +but that's not really the point. + +00:05:20.600 --> 00:05:26.199 +Obviously, Emacs also has Base64 encoding. + +00:05:26.200 --> 00:05:27.979 +It's just a point that we might have + +00:05:27.980 --> 00:05:29.959 +C++ and C external programs + +00:05:29.960 --> 00:05:31.239 +that we'd like to be integrating + +00:05:31.240 --> 00:05:37.139 +into our Emacs agents capabilities. + +00:05:37.140 --> 00:05:46.474 +Here we can see a normal named Org Mode source block. + +00:05:46.475 --> 00:05:50.474 +that calls that function, then an Org Mode source block + +00:05:50.475 --> 00:05:56.299 +that calls Emacs's base64-decode-string as a way of + +00:05:56.300 --> 00:05:57.940 +validating it, I guess. + +00:05:57.941 --> 00:06:00.140 +We go to Org, so we can see... + +00:06:00.141 --> 00:06:04.407 +I have a named call to that function calling the Lisp function + +00:06:04.408 --> 00:06:07.040 +Org is just kind of like this. + +00:06:07.041 --> 00:06:11.559 +It's cached but I don't seem to have run it before. + +00:06:11.560 --> 00:06:13.574 +Then I do the Emacs decode. + +00:06:13.575 --> 00:06:15.974 +So if we just run this using C-c C-c, + +00:06:15.975 --> 00:06:17.240 +and we can kind of see + +00:06:17.241 --> 00:06:22.179 +what Org Mode is like a little bit here. + +00:06:22.180 --> 00:06:24.319 +All right, yes, so as we can see, + +00:06:24.320 --> 00:06:27.659 +oh hang on, let's run this as well actually. + +00:06:27.660 --> 00:06:32.193 +So the C embeddable Common Lisp + +00:06:32.194 --> 00:06:35.199 +base64 encoding gets us this. + +00:06:35.200 --> 00:06:38.079 +And then Emacs is decoding and gets us back, + +00:06:38.080 --> 00:06:40.319 +kind of validates it. I think I'm missing some things. + +00:06:40.320 --> 00:06:43.079 +I don't pad characters out to the correct byte lengths, + +00:06:43.080 --> 00:06:45.399 +that kind of thing, but it's fine. + +NOTE using this via eev as a human + +00:06:45.400 --> 00:06:48.719 +And then I kind of contrast that to, + +00:06:48.720 --> 00:06:53.179 +I really like what my friend mdhughes.tech, + +00:06:53.180 --> 00:06:57.319 +game dev of the ages, calls REPL-driven development, + +00:06:57.320 --> 00:07:06.139 +which he says is kind of the opposite of literate coding. + +00:07:06.140 --> 00:07:08.940 +I think eev, at least for me, + +00:07:08.941 --> 00:07:11.079 +is kind of like REPL-driven development. + +00:07:11.080 --> 00:07:16.159 +So in eev, if you just press F8, the thing happens. + +00:07:16.160 --> 00:07:17.479 +And if it's a red star line, + +00:07:17.480 --> 00:07:19.439 +the thing is an Emacs Lisp thing, + +00:07:19.440 --> 00:07:22.999 +and otherwise it goes to the eepitch target. + +00:07:23.000 --> 00:07:26.719 +So if I do this, great, now I'm pitching to that slime + +00:07:26.720 --> 00:07:32.759 +REPL ECL I made. And then I pressed F8. Press F8 again. + +00:07:32.760 --> 00:07:34.480 +The string got coerced to a list. + +00:07:34.481 --> 00:07:38.359 +F8. Now it's car codified. + +00:07:38.360 --> 00:07:41.319 +I quite like this, because this looks like something I can do + +00:07:41.320 --> 00:07:44.239 +and understand doing and reason about doing. + +00:07:44.240 --> 00:07:49.519 +Then I form a command to send from Lisp to Emacs. + +00:07:49.520 --> 00:07:52.599 +Then I do it and I recover the string from the beginning. + +00:07:52.600 --> 00:07:56.119 +I guess I had one of these here. Oh, by the way, look at + +00:07:56.120 --> 00:07:59.159 +What Org Mode did with an eev source block. + +00:07:59.160 --> 00:08:00.999 +And then when I close the source block + +00:08:01.000 --> 00:08:02.679 +using C-c ', + +00:08:02.680 --> 00:08:05.319 +it brings me back to the Org doc, + +00:08:05.320 --> 00:08:09.159 +which was a cool synergy between the eev minor mode + +00:08:09.160 --> 00:08:16.019 +and eev source blocks in Org Mode that I noticed. + +00:08:16.020 --> 00:08:22.599 +And so I kind of want my agents to be like this eev usage. + +00:08:22.600 --> 00:08:25.159 +Clearly, Org is super powerful, + +00:08:25.160 --> 00:08:28.159 +but I don't even like writing calls like this, + +00:08:28.160 --> 00:08:32.079 +where you write the function that will happen last first, + +00:08:32.080 --> 00:08:39.039 +so you're kind of writing right to left, first to last. + +00:08:39.040 --> 00:08:41.239 +Whereas in REPL-driven development, + +00:08:41.240 --> 00:08:43.199 +I guess I'm writing top to bottom, + +00:08:43.200 --> 00:08:46.979 +and eev, I guess, executable logs + +00:08:46.980 --> 00:08:48.599 +are logs that are like that. + +00:08:48.600 --> 00:08:52.378 +So I kind of like eev's view for reasoning + +00:08:52.379 --> 00:08:54.399 +more than Org's Tangle. + +00:08:54.400 --> 00:08:57.319 +Obviously, Tangle is trying to do tricky things, + +00:08:57.320 --> 00:09:01.359 +but maybe they have different specializations, + +00:09:01.360 --> 00:09:04.879 +and eev's one is more close + +00:09:04.880 --> 00:09:07.799 +to my own version of intelligence, maybe. + +NOTE Software individuals using eev in Emacs like a human + +00:09:07.800 --> 00:09:13.539 +Software individuals using eev in Emacs like a human. + +00:09:13.540 --> 00:09:17.279 +Yeah, you can always visit my blog post for more detail. + +00:09:17.280 --> 00:09:20.039 +Right, I made a CLOS object + +00:09:20.040 --> 00:09:22.519 +in Common Lisp to wrap doing this. + +00:09:22.520 --> 00:09:23.639 +It's not really the topic. + +00:09:23.640 --> 00:09:27.959 +It's in the appendix somewhere if you need it. + +00:09:27.960 --> 00:09:29.559 +So I've just executed that. + +00:09:29.560 --> 00:09:32.079 +You can look at the appendix in your own time. + +NOTE Sandewall's leonardo system + +00:09:32.080 --> 00:09:33.959 +Jumping over to actually starting + +00:09:33.960 --> 00:09:36.319 +our hypothetical intelligent agent. + +00:09:36.320 --> 00:09:38.239 +I guess we're doing eev here. + +00:09:38.240 --> 00:09:46.759 +So if we open this, press F8 a bunch of times. + +00:09:46.760 --> 00:09:49.199 +Oh, and if you were cloning it yourself, + +00:09:49.200 --> 00:09:56.719 +I guess that's what you would do. setq eepitch-buffer-name. + +00:09:56.720 --> 00:10:00.319 +Oh yeah, if you went to an eepitch shell and then came back. + +00:10:00.320 --> 00:10:01.679 +You would have had to do that, but I didn't. + +00:10:01.680 --> 00:10:04.239 +I didn't, so I didn't need to. + +00:10:04.240 --> 00:10:07.279 +Sandewall's style is to use relative paths + +00:10:07.280 --> 00:10:11.974 +to tell which agent is acting inside a software individual. + +00:10:11.975 --> 00:10:13.359 +Remembering a software individual + +00:10:13.360 --> 00:10:15.239 +is potentially a bunch of agents. + +00:10:15.240 --> 00:10:18.479 +And we load... So one individual, + +00:10:18.480 --> 00:10:21.919 +all the agents in each individual share a kernel. + +00:10:21.920 --> 00:10:25.599 +So only one agent in one software individual + +00:10:25.600 --> 00:10:28.279 +is active at any given time, but the agents are separate. + +00:10:28.280 --> 00:10:31.279 +They just all have to share the kernel resource, + +00:10:31.280 --> 00:10:38.319 +which is the Remus agent. Oh, I got rid of this. + +00:10:38.320 --> 00:10:43.279 +And start the CLE is the thing. + +00:10:43.280 --> 00:10:46.119 +Oh, I did need to have an EmacsConf knowledge base. + +00:10:46.120 --> 00:10:48.959 +Well, let's just keep eepitching for a little bit. + +00:10:48.960 --> 00:10:55.259 +So I think I made... I'm going to call it emacsconf-kb. + +00:10:55.260 --> 00:10:59.679 +Right, that looks likely. And I think that the agent... + +00:10:59.680 --> 00:11:03.479 +I can check this. I could have checked that. + +00:11:03.480 --> 00:11:12.699 +I could have done something like (get emacsconf-kb contents). + +00:11:12.700 --> 00:11:13.479 +Yeah, and you can see + +00:11:13.480 --> 00:11:15.879 +there's a location inside it which is agent1, + +00:11:15.880 --> 00:11:17.519 +which I assume is an entity file + +00:11:17.520 --> 00:11:20.599 +that I was working with before. + +00:11:20.600 --> 00:11:21.919 +And then what were we going to do? + +00:11:21.920 --> 00:11:28.279 +Oh yeah, back to the embeddable Common Lisp image. + +00:11:28.280 --> 00:11:36.099 +So if I just press our button back to there... + +NOTE Start a loop for one leonardo software individual + +00:11:36.100 --> 00:11:41.119 +And so my idea is that for an Emacs agent, + +00:11:41.120 --> 00:11:46.999 +basically, I'd like to have an Emacs Lisp list. + +00:11:47.000 --> 00:11:49.640 +And just when stuff gets into that list, + +00:11:49.641 --> 00:11:53.239 +the agent which is always running, but running slowly, + +00:11:53.240 --> 00:11:58.359 +will incrementally just do the stuff it finds in that list. + +00:11:58.360 --> 00:12:00.759 +Populating that list probably gets into stuff + +00:12:00.760 --> 00:12:03.199 +like your Beliefs, Desires, Intents framework + +00:12:03.200 --> 00:12:06.159 +and those kind of well-known and well-studied algorithms. + +00:12:06.160 --> 00:12:07.799 +That's not the point here. + +00:12:07.800 --> 00:12:14.259 +I just want to have a list in Emacs that my ECL... + +00:12:14.260 --> 00:12:16.079 +I'm just going to run a loop in ECL, + +00:12:16.080 --> 00:12:18.319 +and the ECL is going to keep sending + +00:12:18.320 --> 00:12:22.399 +anything it finds in that Emacs Lisp list + +00:12:22.400 --> 00:12:25.399 +to the software agent. The agent is also in Emacs, + +00:12:25.400 --> 00:12:28.759 +so it would be able to populate its own list itself + +00:12:28.760 --> 00:12:36.159 +if it had an idea of evaluating desires and chances to improve + +00:12:36.160 --> 00:12:37.559 +whatever it wants to improve + +00:12:37.560 --> 00:12:39.999 +and chances to avoid whatever it wants to avoid. + +00:12:40.000 --> 00:12:47.599 +We talked a little bit too much. Let's just start this. + +00:12:47.600 --> 00:12:51.539 +Sorry that I'm manually setting up my screen. + +00:12:51.540 --> 00:12:55.499 +Then let's put CLisp over here. + +00:12:55.500 --> 00:12:58.679 +Right, we could work with this, right? + +00:12:58.680 --> 00:13:00.099 +This loop isn't very important. + +00:13:00.100 --> 00:13:04.919 +It's just a Common Lisp loop. I copy my friend jmbr's style + +00:13:04.920 --> 00:13:08.199 +of using Lisp machine-style keyword arguments + +00:13:08.200 --> 00:13:12.119 +instead of symbols like cl-loop, + +00:13:12.120 --> 00:13:16.719 +the compatibility thing in Emacs Lisp does. + +00:13:16.720 --> 00:13:28.139 +I'd never initialized that. Well, let's do that. + +00:13:28.140 --> 00:13:30.679 +Okay, now we have the list. + +00:13:30.680 --> 00:13:35.019 +And just every 30, let's turn it down to every 20 seconds. + +00:13:35.020 --> 00:13:37.159 +Hypothetically, it's going to put + +00:13:37.160 --> 00:13:39.999 +whatever it finds in there, into there. + +00:13:40.000 --> 00:13:46.239 +And so, I think, yeah, and now... Great. + +00:13:46.240 --> 00:13:50.099 +So here I'm just going to fill it with stuff. + +00:13:50.100 --> 00:13:54.839 +And this is quite interesting, I think. + +00:13:54.840 --> 00:13:58.479 +It just shows I can put a whole bunch of stuff into that list. + +00:13:58.480 --> 00:14:01.199 +Ideally, the agent would populate it itself + +00:14:01.200 --> 00:14:03.359 +with a BDI algorithm or something. + +00:14:03.360 --> 00:14:04.919 +But if we just put some stuff in there, + +00:14:04.920 --> 00:14:07.799 +we'll see that it will all get sent + +00:14:07.800 --> 00:14:14.799 +basically using Eduardo's eepitch internal machinery, at least. + +00:14:14.800 --> 00:14:17.479 +And hence, it meets my requirement + +00:14:17.480 --> 00:14:20.779 +that it works exactly like I work. + +00:14:20.780 --> 00:14:25.859 +And then in eev, I just have to press M-e. + +00:14:25.860 --> 00:14:31.479 +Oh, it works via Emacs server, and I didn't start that, + +00:14:31.480 --> 00:14:39.719 +so if we server-start, hopefully... + +00:14:39.720 --> 00:14:42.799 +And then, ideally, things will just begin happening + +00:14:42.800 --> 00:14:53.119 +in this slime-repl C/Lisp agent. + +00:14:53.120 --> 00:15:05.419 +Oh, if this was still running. + +00:15:05.420 --> 00:15:07.199 +Okay, well we got at least one, + +00:15:07.200 --> 00:15:09.639 +but hypothetically lots of these will happen. + +00:15:09.640 --> 00:15:13.699 +So, show agent, I guess, + +00:15:13.700 --> 00:15:17.039 +happened over here. I put a whole bunch of "sleep-for"s in, + +00:15:17.040 --> 00:15:19.719 +because I thought that going slowly + +00:15:19.720 --> 00:15:21.319 +would make it seem more human. + +00:15:21.320 --> 00:15:24.639 +Like I saw in Eduardo's talk last year + +00:15:24.640 --> 00:15:29.099 +which is where I learned about eev. + +00:15:29.100 --> 00:15:32.319 +The system is a little fragile. + +00:15:32.320 --> 00:15:41.079 +Hypothetically, we have a whole bunch of agents. + +00:15:41.080 --> 00:15:43.039 +I guess every time it gets sent, + +00:15:43.040 --> 00:15:44.999 +it checks that we're in the right agent. + +00:15:45.000 --> 00:15:46.999 +And it's not actually just sending a string, + +00:15:47.000 --> 00:15:52.799 +it's sending a sequence of string actions over there. + +00:15:52.800 --> 00:15:57.479 +And so we see Emacs Lisp hypothetically put, + +00:15:57.480 --> 00:16:06.859 +I guess it put this "foo bar baz!" into an entity, message-1, + +00:16:06.860 --> 00:16:11.899 +which should be of type message, I guess, conceivably. + +00:16:11.900 --> 00:16:13.319 +I forget if I set that up earlier. + +00:16:13.320 --> 00:16:14.719 +It's in the appendix somewhere. + +00:16:14.720 --> 00:16:17.999 +And then it just called, it did a sequence of actions + +00:16:18.000 --> 00:16:21.319 +which was really just one action of showing that. + +00:16:21.320 --> 00:16:26.399 +And then I called b64-encode on message1, + +00:16:26.400 --> 00:16:30.599 +which I believe will have set message-1 encoded. + +00:16:30.600 --> 00:16:37.242 +Can I check that manually while it's happening? + +00:16:37.243 --> 00:16:51.499 +Disaster. Well that's what it should have been. + +00:16:51.500 --> 00:16:54.940 +Well, I did mention it was a little bit fragile. + +00:16:54.941 --> 00:17:03.279 +What if we put... Can we kind of rescue this? + +00:17:03.280 --> 00:17:07.239 +I don't want to try redoing this. It's slightly fragile. + +00:17:07.240 --> 00:17:12.639 +What it would do, we can see the actions are kind of getting there, + +00:17:12.640 --> 00:17:16.719 +but somehow my message didn't end up getting encoded + +00:17:16.720 --> 00:17:18.119 +by that sequence of actions. + +00:17:18.120 --> 00:17:23.279 +So this decode will have also made the decoded one be null. + +NOTE Let's do it manually + +00:17:23.280 --> 00:17:26.239 +Let's just do it manually. Should have worked. + +00:17:26.240 --> 00:17:30.559 +b64-encode, which calls out to Emacs + +00:17:30.560 --> 00:17:37.299 +to get everything actually done. + +00:17:37.300 --> 00:17:41.519 +Oh, I got interrupted by the agent. + +00:17:41.520 --> 00:17:43.320 +Well, if I do it manually, it worked. + +00:17:43.321 --> 00:17:53.519 +Hypothetically, the queue thing should have worked. Great. + +00:17:53.520 --> 00:17:56.840 +Well, you can see it's kind of working. + +00:17:56.841 --> 00:17:57.440 +Could be more robust. + +00:17:57.441 --> 00:18:03.640 +The reason is that I think what I did is a bit fragile, + +00:18:03.641 --> 00:18:07.107 +but the intent is that FIPA, + +00:18:07.108 --> 00:18:09.307 +Foundation for Intelligent Physical Agents's + +00:18:09.308 --> 00:18:15.639 +SL standard has tools for reliability + +00:18:15.640 --> 00:18:19.919 +through repetition and checking outcomes and that kind of thing. + +00:18:19.920 --> 00:18:22.959 +So I would use those. I'm not putting too much work + +00:18:22.960 --> 00:18:26.679 +into being ultra-reliable right now, but it kind of worked. + +00:18:26.680 --> 00:18:29.759 +We saw, I guess, at least Embeddable Common Lisp + +00:18:29.760 --> 00:18:35.599 +believed it used emacsclient externally, asynchronously, + +00:18:35.600 --> 00:18:38.359 +to send these to Emacs within Emacs. + +00:18:38.360 --> 00:18:41.599 +I put a whole bunch of sleeps into its thing + +00:18:41.600 --> 00:18:44.999 +to make it look slow and human-like, kind of happened + +00:18:45.000 --> 00:18:52.719 +because Emacs' model is that it's kind of single-threaded. + +00:18:52.720 --> 00:18:59.639 +Can I just... I bet if we run this again + +00:18:59.640 --> 00:19:02.119 +It'll at least look like it's succeeding + +00:19:02.120 --> 00:19:05.039 +because I fixed the base64 encoding + +00:19:05.040 --> 00:19:11.399 +and so forth in the background. I wonder if it will. + +NOTE Wrapping up + +00:19:11.400 --> 00:19:15.559 +In the meantime, let's wrap up this talk to some extent. + +00:19:15.560 --> 00:19:18.799 +Then I'm just kind of saying what I'm expecting to happen. + +00:19:18.800 --> 00:19:20.479 +I took out next action. + +00:19:20.480 --> 00:19:25.279 +Originally, I was keeping the list inside of the agent. + +00:19:25.280 --> 00:19:27.879 +Then I decided to keep the list inside Emacs + +00:19:27.880 --> 00:19:31.679 +because I have kind of first class Emacs is my IDE, + +00:19:31.680 --> 00:19:37.607 +so I have better access to what's going on in my IDE. + +NOTE Intelligence + +00:19:37.608 --> 00:19:39.559 +Then I wanted to talk about intelligence a little bit + +00:19:39.560 --> 00:19:41.199 +in whatever my remaining time is. + +00:19:41.200 --> 00:19:43.039 +I just have these great bullet points + +00:19:43.040 --> 00:19:45.559 +of nosrednA yduJ and Eric Sandewall. + +00:19:45.560 --> 00:19:50.039 +So nosrednA yduJ, when she was on the show quite a long time ago, + +00:19:50.040 --> 00:19:55.559 +she... I keep describing things as expert systems + +00:19:55.560 --> 00:19:57.039 +and she wanted to know what I meant + +00:19:57.040 --> 00:19:58.359 +when I said expert systems, + +00:19:58.360 --> 00:20:00.199 +and I gave her a Lisp software example + +00:20:00.200 --> 00:20:02.618 +and she said she personally wrote + +00:20:02.619 --> 00:20:06.279 +that software in the 80s that I was referring to + +00:20:06.280 --> 00:20:08.239 +and she wanted to know how it was an expert system. + +00:20:08.240 --> 00:20:10.039 +What I mean when I say expert system + +00:20:10.040 --> 00:20:19.839 +is a system that works kind of like I do and eev's eepitch does. + +00:20:19.840 --> 00:20:21.999 +It's where we can really reason + +00:20:22.000 --> 00:20:24.199 +in a very human-relatable way + +00:20:24.200 --> 00:20:26.479 +about what the inputs to the program is. + +00:20:26.480 --> 00:20:31.399 +And also a program should be exposed to other programs + +00:20:31.400 --> 00:20:36.559 +in terms of like a well-structured transfer of knowledge as inputs, + +00:20:36.560 --> 00:20:38.010 +and it should have a well-structured + +00:20:38.011 --> 00:20:41.939 +transfer of knowledge kind of outputs. + +00:20:41.940 --> 00:20:47.159 +I don't know why this b64-encode message wasn't working. + +00:20:47.160 --> 00:20:49.999 +Then we kind of faked it into working. + +00:20:50.000 --> 00:20:52.399 +It's going to be embarrassing for me + +00:20:52.400 --> 00:20:58.739 +if anybody watches this. But yeah, so yduJ's thing... + +00:20:58.740 --> 00:20:59.959 +And then I was going to also build + +00:20:59.960 --> 00:21:02.679 +that into Eric Sandewall's one. + +00:21:02.680 --> 00:21:05.639 +So this is my vision of expert systems + +00:21:05.640 --> 00:21:07.779 +as kind of maybe this is an important + +00:21:07.780 --> 00:21:11.679 +general style loosely associated with Lisp. + +00:21:11.680 --> 00:21:14.399 +Same as the Lisp editor Emacs. + +00:21:14.400 --> 00:21:17.665 +So Eric Sandewall's description of intelligence + +00:21:17.666 --> 00:21:21.159 +was that his grandchildren were intelligent. + +00:21:21.160 --> 00:21:26.439 +So if we had software agents that were intelligent, + +00:21:26.440 --> 00:21:32.439 +this would be true if and maybe only if they were similar + +00:21:32.440 --> 00:21:33.719 +to his grandchildren + +00:21:33.720 --> 00:21:36.319 +who were a good reference for intelligence. + +00:21:36.320 --> 00:21:39.199 +And grandchildren live for a really long time. + +00:21:39.200 --> 00:21:42.879 +They kind of learn gradually. + +00:21:42.880 --> 00:21:46.879 +They don't run on GPUs for a few minutes + +00:21:46.880 --> 00:21:51.879 +and then get thrown out forever, something like that. + +00:21:51.880 --> 00:21:54.959 +And so this is the kind of vision of, I guess, + +00:21:54.960 --> 00:21:57.919 +the Leonardo system software individual stuff. + +00:21:57.920 --> 00:22:03.946 +You can see we kind of faked it into... + +00:22:03.947 --> 00:22:06.320 +at least the show get message one decoded bits were working. + +00:22:06.321 --> 00:22:07.300 +I'm not sure what was happening + +00:22:07.301 --> 00:22:12.674 +with the Elisp ones that worked interactively, + +00:22:12.675 --> 00:22:18.607 +but then they didn't work in my loopy thing. + +00:22:18.608 --> 00:22:21.307 +Oh yeah, and then so I mentioned + +00:22:21.308 --> 00:22:24.640 +thank you to Sacha at the start of this talk. + +00:22:24.641 --> 00:22:26.974 +And so Eric Sandewall's emphasis + +00:22:26.975 --> 00:22:31.340 +that you'd really like intelligent software agents, + +00:22:31.341 --> 00:22:34.174 +Leonardo system agents, to be like your grandchildren. + +00:22:34.175 --> 00:22:40.659 +And I was talking to somebody, maybe to Ramin Honary + +00:22:40.660 --> 00:22:44.959 +who's doing the schemacs talk this year + +00:22:44.960 --> 00:22:46.874 +about Sacha's writing. + +00:22:46.875 --> 00:22:48.840 +A lot of Sacha's writing is about + +00:22:48.841 --> 00:22:51.774 +her experiences of life and technology, + +00:22:51.775 --> 00:22:54.374 +and especially raising A* + +00:22:54.375 --> 00:22:59.740 +and her observations of her progeny A*'s + +00:22:59.741 --> 00:23:05.319 +experiences of life and technology, + +00:23:05.320 --> 00:23:07.874 +I would say as well as being + +00:23:07.875 --> 00:23:18.039 +the Emacs News and Emacs conf doer that she is. + +00:23:18.040 --> 00:23:22.740 +Yeah, and so I think a lot of what Sacha is seen doing + +00:23:22.741 --> 00:23:25.840 +and concerned with are specifically what Eric Sandewall + +00:23:25.841 --> 00:23:31.207 +identifies as the study of intelligence as such, + +00:23:31.208 --> 00:23:36.479 +as should apply to computing as well. That was my thought + +00:23:36.480 --> 00:23:42.979 +on Sacha, Eric Sandewall, intelligence, and yduJ. + +00:23:42.980 --> 00:23:44.240 +I have this note from pizzapal... + +00:23:44.241 --> 00:23:46.274 +I didn't realize that Microsoft had announced + +00:23:46.275 --> 00:23:49.679 +that 2025 was going to be the year of the software agent. + +00:23:49.680 --> 00:23:51.199 +I only found this out in hindsight + +00:23:51.200 --> 00:23:54.199 +when I saw people crowing on the Mastodon + +00:23:54.200 --> 00:23:58.079 +about how Microsoft had basically declared + +00:23:58.080 --> 00:24:00.779 +that their Year of the Agent marketing campaign + +00:24:00.780 --> 00:24:04.459 +was a failure + +00:24:04.460 --> 00:24:09.279 +where basically people didn't like the same old web services + +00:24:09.280 --> 00:24:11.359 +but now while you're accessing, + +00:24:11.360 --> 00:24:15.239 +while you're formally kind of accessing a web service, + +00:24:15.240 --> 00:24:16.959 +the kind of web service that used to be called + +00:24:16.960 --> 00:24:19.279 +serverless web services, this kind of thing, + +00:24:19.280 --> 00:24:23.879 +but you're just being gibbered at by Microsoft Copilot + +00:24:23.880 --> 00:24:27.119 +while you're trying to use regular services. + +00:24:27.120 --> 00:24:29.279 +And people turned out not to like this. + +00:24:29.280 --> 00:24:32.399 +I think that, as we can see in this agent, + +00:24:32.400 --> 00:24:36.374 +the agent really needs to be running on its own clock + +00:24:36.375 --> 00:24:37.907 +and independently of you. + +00:24:37.908 --> 00:24:42.279 +Like if you imagine your body is getting + +00:24:42.280 --> 00:24:46.074 +novel, slightly speculative instructions from your brain + +00:24:46.075 --> 00:24:50.680 +constantly throughout your entire waking day, quite slowly, + +00:24:50.681 --> 00:24:54.974 +this is what an agent should be like. + +00:24:54.975 --> 00:24:59.540 +And it should be... Sandewall wrote about this. + +00:24:59.541 --> 00:25:01.540 +Basically, computer programs + +00:25:01.541 --> 00:25:04.840 +aren't going to want to use human natural language with each other. + +00:25:04.841 --> 00:25:06.674 +There's nothing desirable about that, + +00:25:06.675 --> 00:25:10.674 +so you wouldn't have two hypothetical Microsoft agents, + +00:25:10.675 --> 00:25:13.399 +which are just regular web services with + +00:25:13.400 --> 00:25:16.340 +a GPT model gibbering at you + +00:25:16.341 --> 00:25:19.839 +while you're trying to use the web service. + +00:25:19.840 --> 00:25:22.539 +I think we can see... + +00:25:22.540 --> 00:25:26.740 +Microsoft did the wrong thing with the word agent, + +00:25:26.741 --> 00:25:30.707 +allowing that agent is an overloaded term like static. + +00:25:30.708 --> 00:25:34.256 +I'm going to stop this. I'm not going to try and fix this. + +00:25:34.257 --> 00:25:36.313 +Sorry, everybody. Thank you. Talk to you on the Mastodon. + +00:25:36.314 --> 00:25:37.919 +Hopefully, see you on the show. + +00:25:37.920 --> 00:25:40.399 +See you at your conference talks. + +00:25:40.400 --> 00:25:45.599 +My blog has writing and examples of this with multi-agents, + +00:25:45.600 --> 00:25:50.819 +more C and C++ stuff, Lisp things. + +00:25:50.820 --> 00:25:53.439 +You're welcome to come on my show to be interviewed, + +00:25:53.440 --> 00:25:56.640 +however formally we do that. See everybody next time. diff --git a/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main--chapters.vtt b/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main--chapters.vtt new file mode 100644 index 00000000..23622244 --- /dev/null +++ b/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main--chapters.vtt @@ -0,0 +1,77 @@ +WEBVTT + + +00:00:00.000 --> 00:00:19.839 +Before we begin + +00:00:19.840 --> 00:01:02.799 +The 4-year overnight success + +00:01:02.800 --> 00:01:15.599 +The real title + +00:01:15.600 --> 00:01:34.279 +Why not gnus/mu4e/notmuch? + +00:01:34.280 --> 00:02:17.919 +The honest answer + +00:02:17.920 --> 00:02:49.439 +The org-gmail philosophy + +00:02:49.440 --> 00:03:21.199 +Architecture (the boring but important slide) + +00:03:21.200 --> 00:04:37.479 +Demo 1: From gmail to org + +00:04:37.480 --> 00:05:43.039 +Settings + +00:05:43.040 --> 00:07:56.879 +Downloading + +00:07:56.880 --> 00:09:33.679 +Replying + +00:09:33.680 --> 00:10:57.159 +Label management + +00:10:57.160 --> 00:12:04.119 +Refiling + +00:12:04.120 --> 00:13:37.139 +Archiving + +00:13:37.140 --> 00:15:53.679 +Action commands + +00:15:53.680 --> 00:16:28.279 +Org Agenda + +00:16:28.280 --> 00:17:07.439 +Trash + +00:17:07.440 --> 00:17:40.559 +Real workflow: GTD + +00:17:40.560 --> 00:18:35.959 +Real Workflow: P.A.R.A. + +00:18:35.960 --> 00:20:07.679 +What this is NOT + +00:20:07.680 --> 00:20:54.759 +Technical decisions + +00:20:54.760 --> 00:21:41.439 +Roadmap + +00:21:41.440 --> 00:22:32.939 +Contributing + +00:22:32.940 --> 00:22:41.119 +The big picture + +00:22:41.120 --> 00:23:04.400 +Let's connect diff --git a/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main.vtt b/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main.vtt new file mode 100644 index 00000000..06ab2200 --- /dev/null +++ b/2025/captions/emacsconf-2025-gmail--orggmail-a-deep-integration-of-gmail-into-your-org-mode--bala-ramadurai--main.vtt @@ -0,0 +1,1764 @@ +WEBVTT captioned by bala + +NOTE Before we begin + +00:00:00.000 --> 00:00:01.759 +Hello everyone. + +00:00:01.760 --> 00:00:03.439 +My name is Bala Ramadurai. + +00:00:03.440 --> 00:00:07.839 +Today I'm going to be talking about org-gmail. + +00:00:07.840 --> 00:00:10.039 +That's something that I put together. + +00:00:10.040 --> 00:00:12.719 +This is what I call gmail meets org mode. + +00:00:12.720 --> 00:00:15.119 +And they get along too. + +00:00:15.120 --> 00:00:16.519 +Let's talk about email + +00:00:16.520 --> 00:00:19.839 +and how to manage email via org mode. + +NOTE The 4-year overnight success + +00:00:19.840 --> 00:00:23.919 +This project is a four year overnight success. + +00:00:23.920 --> 00:00:26.039 +It was in 2021. + +00:00:26.040 --> 00:00:27.279 +I said, Hey, wait a second. + +00:00:27.280 --> 00:00:30.519 +It'll be so cool if we can integrate Gmail with org. + +00:00:30.520 --> 00:00:32.959 +So I started trying out new things. + +00:00:32.960 --> 00:00:34.399 +And immediately I realized + +00:00:34.400 --> 00:00:36.799 +this is a much larger project than I thought. + +00:00:36.800 --> 00:00:40.599 +So it lived in someday maybe.org + +00:00:40.600 --> 00:00:42.599 +for about three years. + +00:00:42.600 --> 00:00:48.759 +Enter 2024 AI arrives and in 2025 I had a working + +00:00:48.760 --> 00:00:50.919 +prototype in 24 hours flat. + +00:00:50.920 --> 00:00:55.759 +So three years and 364 days, nothing much happened + +00:00:55.760 --> 00:00:58.159 +and one day it actually got it working. + +00:00:58.160 --> 00:01:00.919 +Sometimes procrastination is just waiting for the + +00:01:00.920 --> 00:01:02.799 +right tools. + +NOTE The real title + +00:01:02.800 --> 00:01:05.519 +The real title should have been org mail for + +00:01:05.520 --> 00:01:08.479 +people who like org mode more than email. + +00:01:08.480 --> 00:01:12.799 +The Gmail monster that has always been attacking us. + +00:01:12.800 --> 00:01:15.599 +Okay, but we still have to deal with email. + +NOTE Why not gnus/mu4e/notmuch? + +00:01:15.600 --> 00:01:18.599 +One of the most common questions that I've got so far. + +00:01:18.600 --> 00:01:24.639 +Why not gnus or mu4e or notmuch, or other tools. + +00:01:24.640 --> 00:01:26.359 +They are amazing. + +00:01:26.360 --> 00:01:29.239 +Use them if it works for you, absolutely. + +00:01:29.240 --> 00:01:30.319 +Just go right ahead. + +00:01:30.320 --> 00:01:32.239 +If it works for you, don't change anything + +00:01:32.240 --> 00:01:34.279 +because this looks cool. + +NOTE The honest answer + +00:01:34.280 --> 00:01:36.599 +Well, the honest answer for me is that + +00:01:36.600 --> 00:01:39.399 +they want to be your email client. + +00:01:39.400 --> 00:01:40.879 +That's not what I am after. + +00:01:40.880 --> 00:01:42.959 +I don't want an email client. + +00:01:42.960 --> 00:01:45.039 +I have enough email clients already. + +00:01:45.040 --> 00:01:46.239 +I don't want one more. + +00:01:46.240 --> 00:01:49.359 +And they require 500 lines of config. + +00:01:49.360 --> 00:01:50.679 +I've tried it. + +00:01:50.680 --> 00:01:53.039 +It's a lot of maintenance for myself, + +00:01:53.040 --> 00:01:54.359 +I still have those somewhere. + +00:01:54.360 --> 00:01:57.319 +The mu4e config or the gnus config. + +00:01:57.320 --> 00:01:59.439 +They struggle with Gmail's labels, + +00:01:59.440 --> 00:02:01.839 +threading, messages and deletion. + +00:02:01.840 --> 00:02:05.439 +I find it tough, and it's either + +00:02:05.440 --> 00:02:07.079 +all in emacs or nothing. + +00:02:07.080 --> 00:02:10.079 +It cannot be a combinatorial approach, + +00:02:10.080 --> 00:02:11.159 +is what I realized. + +00:02:11.160 --> 00:02:13.519 +So I said, why can't we have both? + +00:02:13.520 --> 00:02:15.959 +I want the org mode's focus + +00:02:15.960 --> 00:02:17.919 +and Gmail's flexibility. + +NOTE The org-gmail philosophy + +00:02:17.920 --> 00:02:20.559 +Org-gmail philosophy is very simple. + +00:02:20.560 --> 00:02:22.199 +You triage in Gmail. + +00:02:22.200 --> 00:02:26.279 +Use the fast web UI for the easy stuff + +00:02:26.280 --> 00:02:27.879 +and process in org mode. + +00:02:27.880 --> 00:02:30.639 +Pull important threads where you do real work. + +00:02:30.640 --> 00:02:33.719 +And of course, a two way sync is possible. + +00:02:33.720 --> 00:02:36.639 +Changes flow both directions. + +00:02:36.640 --> 00:02:39.479 +Big inspiration has been org-gcal. + +00:02:39.480 --> 00:02:42.319 +I named it org-gmail because I saw org-gcal. + +00:02:42.320 --> 00:02:43.159 +It was so cool. + +00:02:43.160 --> 00:02:44.759 +I really wanted it. + +00:02:44.760 --> 00:02:47.719 +Think of it like an org capture for email, but + +00:02:47.720 --> 00:02:49.439 +just bidirectional. + +NOTE Architecture (the boring but important slide) + +00:02:49.440 --> 00:02:52.999 +Alright, the architecture (boring, but important + +00:02:53.000 --> 00:02:56.679 +slide) is that Gmail interacts with Python via an + +00:02:56.680 --> 00:02:59.439 +API and interacts with Emacs Lisp. + +00:02:59.440 --> 00:03:02.519 +User commands, org formatting, all that magic is + +00:03:02.520 --> 00:03:04.279 +done on the Emacs side with Lisp. + +00:03:04.280 --> 00:03:09.140 +Python side handles the Gmail API, OAuth, + +00:03:09.141 --> 00:03:12.407 +json wrangling and Gmail API handles + +00:03:12.408 --> 00:03:13.959 +the actual email data. + +00:03:13.960 --> 00:03:17.679 +You can do pip install, add to the load path, and + +00:03:17.680 --> 00:03:21.199 +10 minute OAuth setup, you are all set. + +NOTE Demo 1: From gmail to org + +00:03:21.200 --> 00:03:27.079 +We'll switch over to demo from gmail to org. + +00:03:27.080 --> 00:03:29.039 +How do you go about doing that? + +00:03:29.040 --> 00:03:32.879 +I will start off with a demo folder that I have. + +00:03:32.880 --> 00:03:35.119 +It has this tree structure. + +00:03:35.120 --> 00:03:36.559 +Ignore the tilde files. + +00:03:36.560 --> 00:03:37.919 +So this is what it contains. + +00:03:37.920 --> 00:03:41.639 +An org folder with all the working directory, the + +00:03:41.640 --> 00:03:44.639 +actual where the life of org mode is. + +00:03:44.640 --> 00:03:48.239 +And I have a credentials.json, this is for logging + +00:03:48.240 --> 00:03:49.119 +into Gmail. + +00:03:49.120 --> 00:03:50.799 +This is a file that you can download. + +00:03:50.800 --> 00:03:53.319 +The instructions are in my README in the + +00:03:53.320 --> 00:03:54.159 +repository. + +00:03:54.160 --> 00:03:56.119 +You can find out how to get yourself a + +00:03:56.120 --> 00:03:57.719 +credentials.json. + +00:03:57.720 --> 00:03:59.159 +It's not very difficult. + +00:03:59.160 --> 00:04:01.479 +Once you have these, you're all set. + +00:04:01.480 --> 00:04:05.359 +All you need to do is if you have straight or any + +00:04:05.360 --> 00:04:08.539 +of the other VC packages ready, that you can take + +00:04:08.540 --> 00:04:11.500 +a Git repository and have that in your folder, you + +00:04:11.501 --> 00:04:13.399 +can do that, or you can do it like this. + +00:04:13.400 --> 00:04:15.159 +Have a Git clone. + +00:04:15.160 --> 00:04:16.479 +Like I'm doing it right now. + +00:04:16.480 --> 00:04:19.239 +Just clone it, keep it in. + +00:04:19.240 --> 00:04:20.874 +And now you'll see + +00:04:20.875 --> 00:04:23.199 +that the Gmail is already there. + +00:04:23.200 --> 00:04:25.199 +There are two files that are really ultra + +00:04:25.200 --> 00:04:28.039 +important, which is, gmail_label_manager.py and + +00:04:28.040 --> 00:04:30.479 +org-gmail.el. + +00:04:30.480 --> 00:04:31.899 +These are the two files that do the email + +00:04:31.900 --> 00:04:33.199 +processing. + +00:04:33.200 --> 00:04:35.199 +I have a plain vanilla Emacs + +00:04:35.200 --> 00:04:37.479 +that I'm going to use for the demo. + +NOTE Settings + +00:04:37.480 --> 00:04:39.439 +These are a few settings + +00:04:39.440 --> 00:04:42.159 +that you will need in order to get going. + +00:04:42.160 --> 00:04:45.959 +So one is the Gmail itself, the elisp, and the + +00:04:45.960 --> 00:04:47.039 +Python script. + +00:04:47.040 --> 00:04:49.399 +You'll need to require the package. + +00:04:49.400 --> 00:04:52.399 +The org agenda files need to be set. + +00:04:52.400 --> 00:04:54.839 +If they're already there, then yes, it needs to + +00:04:54.840 --> 00:04:55.879 +include the org files. + +00:04:55.880 --> 00:04:59.279 +Main settings are, you need an org file in order + +00:04:59.280 --> 00:05:02.239 +to download all the emails from Gmail. + +00:05:02.240 --> 00:05:03.879 +You need the credentials path. + +00:05:03.880 --> 00:05:05.519 +You need the Python script, + +00:05:05.520 --> 00:05:07.439 +wherever it is pointed to that. + +00:05:07.440 --> 00:05:10.039 +The date drawer, you can customize it + +00:05:10.040 --> 00:05:11.319 +to whatever you want. + +00:05:11.320 --> 00:05:12.159 +I call it org-gmail. + +00:05:12.160 --> 00:05:14.639 +You can set it to ignore certain labels, + +00:05:14.640 --> 00:05:15.799 +not to download it. + +00:05:15.800 --> 00:05:17.359 +You're not interested in certain labels + +00:05:17.360 --> 00:05:18.479 +being downloaded. + +00:05:18.480 --> 00:05:19.599 +You can set that + +00:05:19.600 --> 00:05:21.959 +and process time out of 300 seconds. + +00:05:21.960 --> 00:05:25.359 +These are some things to keep life sane in this + +00:05:25.360 --> 00:05:27.819 +plain vanilla emacs. + +00:05:27.820 --> 00:05:30.799 +So I have this refile targets and stuff. + +00:05:30.800 --> 00:05:33.579 +That's the main org-gmail settings are all here. + +00:05:33.580 --> 00:05:37.879 +I'm going to eval this buffer so that we have all + +00:05:37.880 --> 00:05:40.039 +of it and we are all set. + +00:05:40.040 --> 00:05:43.039 +So we have org-gmail ready to work right now. + +NOTE Downloading + +00:05:43.040 --> 00:05:44.959 +The first thing I'm going to show you is + +00:05:44.960 --> 00:05:48.079 +org-gmail-download-by-label. + +00:05:48.080 --> 00:05:50.599 +This is the demo Gmail that I have. + +00:05:50.600 --> 00:05:53.619 +They all have some kind of test emails and I'm + +00:05:53.620 --> 00:05:56.199 +going to label them. + +00:05:56.200 --> 00:06:00.719 +I've created this hierarchy of labels here based + +00:06:00.720 --> 00:06:04.119 +on Tiago Forte's PARA - Project, Area, Resources, + +00:06:04.120 --> 00:06:06.079 +Archives structure. + +00:06:06.080 --> 00:06:09.439 +1Projects, DemoProject1, 2Areas, DemoArea, + +00:06:09.440 --> 00:06:12.119 +4Archives, 2025, OldProject. + +00:06:12.120 --> 00:06:14.959 +I've labeled them inside my Gmail. + +00:06:14.960 --> 00:06:20.759 +Okay, now let's go to Emacs and we will now + +00:06:20.760 --> 00:06:23.719 +download these things, but before downloading + +00:06:23.720 --> 00:06:26.919 +them, you will need to authenticate. + +00:06:26.920 --> 00:06:28.800 +So for that, you can start + +00:06:28.801 --> 00:06:30.900 +with any org-gmail command. + +00:06:30.901 --> 00:06:35.079 +So I'm going to take org-gmail-download-by-label. + +00:06:35.080 --> 00:06:38.839 +When I press that, it immediately opens a session + +00:06:38.840 --> 00:06:39.999 +in my browser. + +00:06:40.000 --> 00:06:40.659 +Okay? + +00:06:40.660 --> 00:06:45.799 +What you can't see is a list of my Gmail accounts + +00:06:45.800 --> 00:06:46.759 +that I'm going to select. + +00:06:46.760 --> 00:06:50.039 +I'm going to select my one Gmail account, and I'm + +00:06:50.040 --> 00:06:53.199 +going to show you the next screen. + +00:06:53.200 --> 00:06:56.999 +So in this screen, you'll have to continue and + +00:06:57.000 --> 00:06:59.119 +select, and the authentication is completed. + +00:06:59.120 --> 00:07:02.679 +So once it's, this is done, you can close this. + +00:07:02.680 --> 00:07:08.999 +Come back to Emacs and you will have Select Gmail. + +00:07:09.000 --> 00:07:10.439 +So it has tab support. + +00:07:10.440 --> 00:07:11.719 +If you click tab, it will tell you + +00:07:11.720 --> 00:07:13.919 +what all labels are available. + +00:07:13.920 --> 00:07:16.319 +I can pick anyone. + +00:07:16.320 --> 00:07:21.839 +1Projects/DemoProject1, and let's see what happens. + +00:07:21.840 --> 00:07:26.479 +It starts downloading and it downloads. + +00:07:26.480 --> 00:07:28.799 +And all three messages, four messages, + +00:07:28.800 --> 00:07:29.719 +five messages. + +00:07:29.720 --> 00:07:30.759 +Six, seven. + +00:07:30.760 --> 00:07:33.319 +There are 11 messages in total, + +00:07:33.320 --> 00:07:36.039 +and it's downloading all of them. + +00:07:36.040 --> 00:07:40.319 +~/demo/org/0Inbox. + +00:07:40.320 --> 00:07:42.079 +That's where I have it. + +00:07:42.080 --> 00:07:44.399 +And here are the emails. + +00:07:44.400 --> 00:07:46.439 +They're all in org mode, format. + +00:07:46.440 --> 00:07:48.399 +All the emails are in here. + +00:07:48.400 --> 00:07:49.719 +Since it's org mode, + +00:07:49.720 --> 00:07:52.679 +it can fold them all and you will see those emails. + +00:07:52.680 --> 00:07:54.279 +So these are five emails + +00:07:54.280 --> 00:07:56.879 +that are present for the DemoProject1. + +NOTE Replying + +00:07:56.880 --> 00:07:59.279 +The next one I wanted to show you was + +00:07:59.280 --> 00:08:01.039 +reply without leaving emacs. + +00:08:01.040 --> 00:08:04.919 +Let's go back to emacs. + +00:08:04.920 --> 00:08:06.479 +How am I going to reply? + +00:08:06.480 --> 00:08:09.319 +org-gmail-reply-at-point. + +00:08:09.320 --> 00:08:11.379 +That's how I'm going to reply. + +00:08:11.380 --> 00:08:13.119 +Reply all or reply. + +00:08:13.120 --> 00:08:13.759 +Okay. + +00:08:13.760 --> 00:08:16.199 +Let's first find out what is the email all about. + +00:08:16.200 --> 00:08:16.719 +Okay. + +00:08:16.720 --> 00:08:19.319 +There's just one sender with just one recipient. + +00:08:19.320 --> 00:08:21.399 +So the reply all or reply + +00:08:21.400 --> 00:08:23.020 +doesn't make a difference. + +00:08:23.021 --> 00:08:23.700 +Okay? + +00:08:23.701 --> 00:08:28.339 +So we will reply at point and if it's Reply All, + +00:08:28.340 --> 00:08:30.999 +it includes my own email as well. + +00:08:31.000 --> 00:08:34.399 +In this, my email is .mx. + +00:08:34.400 --> 00:08:35.159 +Okay. + +00:08:35.160 --> 00:08:39.279 +Cc, I can cc anybody I want and I won't do that. + +00:08:39.280 --> 00:08:42.039 +I have a Gmail reply window. + +00:08:42.040 --> 00:08:43.439 +Split window here. + +00:08:43.440 --> 00:08:47.159 +C-c C-c is what will send the reply. + +00:08:47.160 --> 00:08:50.479 +C-c C-k is what will cancel the reply. + +00:08:50.480 --> 00:08:51.919 +I want to reply. + +00:08:51.920 --> 00:08:57.779 +So let's say test reply from within emacs + +00:08:57.780 --> 00:09:00.559 +and bala@balaramadurai.net + +00:09:00.560 --> 00:09:01.719 +should receive this email. + +00:09:01.720 --> 00:09:02.999 +Okay. + +00:09:03.000 --> 00:09:07.599 +Initial successfully reply sent for this email id. + +00:09:07.600 --> 00:09:08.599 +Okay. + +00:09:08.600 --> 00:09:11.959 +A feature request I can already imagine is + +00:09:11.960 --> 00:09:15.439 +the reply also appearing at the bottom of this. + +00:09:15.440 --> 00:09:16.759 +It's not yet there. + +00:09:16.760 --> 00:09:18.279 +In the next version I will have that. + +00:09:18.280 --> 00:09:22.319 +Let's check if I've have sent that email. + +00:09:22.320 --> 00:09:25.319 +Let's check in the sent box. + +00:09:25.320 --> 00:09:29.119 +I just checked in the sent and yes, + +00:09:29.120 --> 00:09:31.959 +a test reply from within emacs, does show up. + +00:09:31.960 --> 00:09:33.039 +Okay, great. + +00:09:33.040 --> 00:09:33.679 +That works. + +NOTE Label management + +00:09:33.680 --> 00:09:36.959 +The third demo is going to be on label management. + +00:09:36.960 --> 00:09:38.439 +How do I manage labels? + +00:09:38.440 --> 00:09:39.359 +Let's see. + +00:09:39.360 --> 00:09:40.399 +Let's go back to emacs. + +00:09:40.400 --> 00:09:46.599 +Suppose, I am not keen on this DemoProject1 for this. + +00:09:46.600 --> 00:09:50.839 +It should belong to DemoArea, okay? + +00:09:50.840 --> 00:09:52.919 +For this thread itself doesn't belong to this. + +00:09:52.920 --> 00:09:57.879 +Let's do org-gmail-edit-label-at-point. + +00:09:57.880 --> 00:10:00.879 +Well, I don't want one project at all. + +00:10:00.880 --> 00:10:05.899 +It should be under 2Areas/DemoArea. + +00:10:05.900 --> 00:10:08.559 +I don't think it, it's not tab supported. + +00:10:08.560 --> 00:10:09.519 +You should know this. + +00:10:09.520 --> 00:10:09.999 +I will. + +00:10:10.000 --> 00:10:11.439 +That's another feature request. + +00:10:11.440 --> 00:10:14.599 +2Areas/DemoArea. + +00:10:14.600 --> 00:10:17.679 +And when I say this, it should update it. + +00:10:17.680 --> 00:10:19.039 +Yes, it has updated it. + +00:10:19.040 --> 00:10:21.039 +So you can see that it has updated + +00:10:21.040 --> 00:10:22.159 +the label here as well. + +00:10:22.160 --> 00:10:26.159 +We can go check if it has indeed changed it + +00:10:26.160 --> 00:10:29.399 +in our... what is the name of the email? + +00:10:29.400 --> 00:10:31.439 +It's a test mail for one project demo + +00:10:31.440 --> 00:10:33.459 +and the date is at 3 12. + +00:10:33.460 --> 00:10:36.839 +Let's check if DemoArea has it. + +00:10:36.840 --> 00:10:39.559 +It's not refreshed, but there are two, + +00:10:39.560 --> 00:10:42.279 +two emails now under DemoArea. + +00:10:42.280 --> 00:10:43.639 +It's obviously done the job well. + +00:10:43.640 --> 00:10:46.119 +It's happening as expected. + +00:10:46.120 --> 00:10:49.159 +I just turned this back from DemoArea to + +00:10:49.160 --> 00:10:50.159 +DemoProject1. + +00:10:50.160 --> 00:10:53.759 +If you decide that I want to move all of + +00:10:53.760 --> 00:10:56.559 +DemoProject1 to archive, I'm done with the + +00:10:56.560 --> 00:10:57.159 +project. + +NOTE Refiling + +00:10:57.160 --> 00:10:58.319 +Oh, by the way, you could... + +00:10:58.320 --> 00:10:59.599 +that's the whole point + +00:10:59.600 --> 00:11:00.919 +of this is to have + +00:11:00.920 --> 00:11:05.199 +all of this refiled into your project, + +00:11:05.200 --> 00:11:06.719 +you can do that. + +00:11:06.720 --> 00:11:08.919 +I don't know if I have that. + +00:11:08.920 --> 00:11:11.639 +Yes, I have demo project one + +00:11:11.640 --> 00:11:17.519 +and I could have emails and I created a node + +00:11:17.520 --> 00:11:20.439 +and I moved everything there to that folder + +00:11:20.440 --> 00:11:24.799 +so that when I want to look at the demo project. + +00:11:24.800 --> 00:11:25.959 +Let me look at this. + +00:11:25.960 --> 00:11:29.559 +In that context, in the project context and email, + +00:11:29.560 --> 00:11:33.959 +I have notes, let's say, and one of the emails is + +00:11:33.960 --> 00:11:34.839 +a note. + +00:11:34.840 --> 00:11:38.359 +And I want to be able to keep it that way. + +00:11:38.360 --> 00:11:41.719 +I could refile it and put it under notes as well, + +00:11:41.720 --> 00:11:44.439 +saying that this has some password, it has some + +00:11:44.440 --> 00:11:46.559 +reference that I need to have it there. + +00:11:46.560 --> 00:11:48.199 +I can have it under notes as well. + +00:11:48.200 --> 00:11:49.919 +So that's the advantage. + +00:11:49.920 --> 00:11:52.879 +Once it's inside the org mode system, you can do + +00:11:52.880 --> 00:11:55.239 +many things that are usually org-modesy. + +00:11:55.240 --> 00:11:57.599 +So you can do all of that within your org mode + +00:11:57.600 --> 00:12:00.199 +with emails, manipulate them, see it under a + +00:12:00.200 --> 00:12:01.359 +context, reply to that. + +00:12:01.360 --> 00:12:02.574 +All of that can happen + +00:12:02.575 --> 00:12:04.119 +right within your project context. + +NOTE Archiving + +00:12:04.120 --> 00:12:05.439 +You're done with this project. + +00:12:05.440 --> 00:12:06.999 +You want to move to archive. + +00:12:07.000 --> 00:12:09.479 +So what you need to do is you don't have to be + +00:12:09.480 --> 00:12:11.839 +here, you can do it this from anywhere. + +00:12:11.840 --> 00:12:15.199 +Bulk movement of labels, you can do it. + +00:12:15.200 --> 00:12:19.679 +If you had consult, embark, ivy, or helm, this will + +00:12:19.680 --> 00:12:22.439 +show up as a dropdown and it looks neater. + +00:12:22.440 --> 00:12:24.399 +This, I'm using a vanilla emacs, so this is what + +00:12:24.400 --> 00:12:26.279 +you would see, but it has tab support. + +00:12:26.280 --> 00:12:27.519 +You can use that. + +00:12:27.520 --> 00:12:32.319 +So I want to transfer 1Projects/DemoArea1, I want + +00:12:32.320 --> 00:12:32.999 +to move it to archive. + +00:12:33.000 --> 00:12:36.719 +So let's say 4Archives is the folder. + +00:12:36.720 --> 00:12:41.759 +So 4Archives/2025 already set this up. + +00:12:41.760 --> 00:12:46.739 +I have not created this label in Gmail. + +00:12:46.740 --> 00:12:48.680 +So do I need to go back to Gmail and create that? + +00:12:48.681 --> 00:12:49.839 +No, not at all. + +00:12:49.840 --> 00:12:52.159 +You can do this from the comfort of your org mode. + +00:12:52.160 --> 00:12:54.679 +Go in here and enter this. + +00:12:54.680 --> 00:12:55.639 +Let's see what happens. + +00:12:55.640 --> 00:12:57.759 +So it found that it is not there. + +00:12:57.760 --> 00:13:01.439 +So it created a new label and it's now moving all + +00:13:01.440 --> 00:13:04.799 +of the emails, all of those threads into archives + +00:13:04.800 --> 00:13:07.079 +without deleting 1Projects/DemoProject1. + +00:13:07.080 --> 00:13:09.639 +So it still has 1Projects/DemoProject1 and it also + +00:13:09.640 --> 00:13:10.639 +updated the labels here. + +00:13:10.640 --> 00:13:12.239 +So the project has been moved. + +00:13:12.240 --> 00:13:14.999 +If you want to move it to archive this entire... + +00:13:15.000 --> 00:13:16.479 +You can do that too. + +00:13:16.480 --> 00:13:19.079 +Let's see if it has archives. + +00:13:19.080 --> 00:13:21.959 +Yes, it has archives and I have it... + +00:13:21.960 --> 00:13:24.519 +have a heading called 2025. + +00:13:24.520 --> 00:13:26.274 +My demo project can be moved there + +00:13:26.275 --> 00:13:27.279 +and I'm done here. + +00:13:27.280 --> 00:13:28.399 +My project was done. + +00:13:28.400 --> 00:13:30.079 +All the emails are moved to archive. + +00:13:30.080 --> 00:13:32.479 +So is this project from my project folder. + +00:13:32.480 --> 00:13:34.919 +You can integrate it into your workflow, your org + +00:13:34.920 --> 00:13:37.139 +mode workflow. + +NOTE Action commands + +00:13:37.140 --> 00:13:39.039 +Next, we have action commands. + +00:13:39.040 --> 00:13:41.399 +What all can you do with singular emails? + +00:13:41.400 --> 00:13:44.999 +You can do four things with single emails. + +00:13:45.000 --> 00:13:49.659 +Delegate, defer, act on it or trash at point. + +00:13:49.660 --> 00:13:50.399 +Defer. + +00:13:50.400 --> 00:13:52.519 +Doesn't seem to work yet. + +00:13:52.520 --> 00:13:53.599 +That is a snooze part. + +00:13:53.600 --> 00:13:54.759 +Doesn't seem to work yet. + +00:13:54.760 --> 00:13:55.719 +That's another bug. + +00:13:55.720 --> 00:13:58.239 +That's the second bug I have in my package. + +00:13:58.240 --> 00:13:59.599 +But the rest of them work. + +00:13:59.600 --> 00:14:02.519 +Delegate is to move it to somebody so they can do + +00:14:02.520 --> 00:14:03.119 +the job. + +00:14:03.120 --> 00:14:06.399 +Act is you will do it with a context with a to-do + +00:14:06.400 --> 00:14:08.119 +which will show up in your agenda. + +00:14:08.120 --> 00:14:10.839 +Trash the email from your Gmail, and you are done. + +00:14:10.840 --> 00:14:12.079 +Okay, let's do that. + +00:14:12.080 --> 00:14:13.879 +Let's go back to emacs. + +00:14:13.880 --> 00:14:18.079 +I just moved everything back to DemoProject1. + +00:14:18.080 --> 00:14:20.919 +I moved everything back so that I can demonstrate + +00:14:20.920 --> 00:14:22.679 +the four actions that I'm showing. + +00:14:22.680 --> 00:14:23.519 +At least three actions. + +00:14:23.520 --> 00:14:25.039 +One of them doesn't work yet. + +00:14:25.040 --> 00:14:30.179 +So if I want to delegate it to somebody in this + +00:14:30.180 --> 00:14:31.607 +case myself, but I can delegate it + +00:14:31.608 --> 00:14:32.839 +to anybody I want. + +00:14:32.840 --> 00:14:36.159 +So how do I do that so I can delegate this, or + +00:14:36.160 --> 00:14:38.959 +delegate is org-gmail-delegate-at-point. + +00:14:38.960 --> 00:14:42.279 +Move the cursor to the email, delegate it to this + +00:14:42.280 --> 00:14:44.919 +guy balaramadurai.net. + +00:14:44.920 --> 00:14:46.607 +You can add a note saying, + +00:14:46.608 --> 00:14:50.799 +Hey, act on task quickly. + +00:14:50.800 --> 00:14:53.399 +Boss is watching. + +00:14:53.400 --> 00:14:54.919 +Okay? + +00:14:54.920 --> 00:14:59.519 +So you can say yes, and this thread gets forwarded + +00:14:59.520 --> 00:15:01.440 +to your colleague + +00:15:01.441 --> 00:15:03.559 +and so that they can take this up. + +00:15:03.560 --> 00:15:05.819 +It has indeed arrived here. + +00:15:05.820 --> 00:15:07.039 +You can see. + +00:15:07.040 --> 00:15:08.839 +Boss is watching. + +00:15:08.840 --> 00:15:11.399 +The test reply was also arrived here. + +00:15:11.400 --> 00:15:13.079 +You can also see that email. + +00:15:13.080 --> 00:15:15.359 +So that was delegate. + +00:15:15.360 --> 00:15:18.519 +So how do we set up actions? + +00:15:18.520 --> 00:15:21.199 +So let's not mess this email. + +00:15:21.200 --> 00:15:22.599 +This is the second email. + +00:15:22.600 --> 00:15:23.919 +How do you act + +00:15:23.920 --> 00:15:24.919 +on it? + +00:15:24.920 --> 00:15:27.159 +That's an action you set for yourself is add + +00:15:27.160 --> 00:15:31.000 +action at point, and you do that. + +00:15:31.001 --> 00:15:32.940 +What action can you set for yourself? + +00:15:32.941 --> 00:15:38.619 +Write a long report using an LLM. + +00:15:38.620 --> 00:15:42.279 +Okay, so it's changed the status to a to-do task. + +00:15:42.280 --> 00:15:44.959 +And the to-do is right here. + +00:15:44.960 --> 00:15:47.833 +Now, here you can schedule it + +00:15:47.834 --> 00:15:50.700 +to say tomorrow 9:00 AM. + +00:15:50.701 --> 00:15:51.800 +So there you go. + +00:15:51.801 --> 00:15:53.679 +At 9:00 AM I'll be looking at this. + +NOTE Org Agenda + +00:15:53.680 --> 00:15:55.119 +Now here's the cool part. + +00:15:55.120 --> 00:15:58.933 +You can actually find the whole thing + +00:15:58.934 --> 00:16:00.559 +in org agenda. + +00:16:00.560 --> 00:16:03.267 +So my entire email threads + +00:16:03.268 --> 00:16:05.959 +are all in the org agenda. + +00:16:05.960 --> 00:16:09.679 +They're all present here and my task associated is + +00:16:09.680 --> 00:16:11.319 +also here in the agenda. + +00:16:11.320 --> 00:16:12.767 +This is marked TODO, + +00:16:12.768 --> 00:16:14.919 +means I haven't acted on that email. + +00:16:14.920 --> 00:16:16.159 +It's still pending. + +00:16:16.160 --> 00:16:19.119 +And what do I have to do is right here within + +00:16:19.120 --> 00:16:21.559 +here, which is the action that I have to carry on + +00:16:21.560 --> 00:16:22.719 +in the DemoProject1. + +00:16:22.720 --> 00:16:24.533 +You can see that demo project one + +00:16:24.534 --> 00:16:25.739 +is showing up in the bottom. + +00:16:25.740 --> 00:16:28.279 +Okay. It's cool way you can also see it in the agenda. + +NOTE Trash + +00:16:28.280 --> 00:16:31.599 +I am not interested in this email at all. + +00:16:31.600 --> 00:16:33.199 +Let's say I, I want to delete it. + +00:16:33.200 --> 00:16:33.719 +Let's see. + +00:16:33.720 --> 00:16:35.319 +Yes, I finished the task. + +00:16:35.320 --> 00:16:37.599 +Now the task is finished. + +00:16:37.600 --> 00:16:40.039 +I really don't want to see this email, this + +00:16:40.040 --> 00:16:40.919 +message alone. + +00:16:40.920 --> 00:16:42.159 +Alright. + +00:16:42.160 --> 00:16:42.599 +Easy peasy. + +00:16:42.600 --> 00:16:46.679 +Let's do Gmail Trash at point. + +00:16:46.680 --> 00:16:48.020 +And you say, message, + +00:16:48.021 --> 00:16:49.901 +I don't want to delete the entire thread. + +00:16:49.880 --> 00:16:50.399 +Yes. + +00:16:50.400 --> 00:16:51.959 +Delete it. + +00:16:51.960 --> 00:16:53.239 +Delete the message alone. + +00:16:53.240 --> 00:16:54.239 +And it's gone. + +00:16:54.240 --> 00:16:56.039 +But it's not gone. + +00:16:56.040 --> 00:16:57.159 +It's gone to the trash. + +00:16:57.160 --> 00:16:59.733 +The entire subtree was deleted + +00:16:59.734 --> 00:17:01.880 +and we have a clean flow here. + +00:17:01.881 --> 00:17:04.479 +I have archived, it is still in the archive, but + +00:17:04.480 --> 00:17:07.439 +it's still active according to my Gmail folder. + +NOTE Real workflow: GTD + +00:17:07.440 --> 00:17:09.574 +Next I'm going to show you is + +00:17:09.575 --> 00:17:12.039 +real workflow capture. + +00:17:12.040 --> 00:17:15.799 +You can use label in email with ToProcess + +00:17:15.800 --> 00:17:19.359 +downloaded to the inbox.org, and do one of these + +00:17:19.360 --> 00:17:20.599 +you've already seen. + +00:17:20.600 --> 00:17:22.519 +Later still doesn't work. + +00:17:22.520 --> 00:17:25.639 +I will get it to work, but hopefully by the time + +00:17:25.640 --> 00:17:28.759 +the conference is up, you will have the feature up + +00:17:28.760 --> 00:17:29.399 +and ready. + +00:17:29.400 --> 00:17:32.679 +Rest of the stuff works, delegate works, trash + +00:17:32.680 --> 00:17:34.039 +works, and add action works. + +00:17:34.040 --> 00:17:37.479 +We are in weekly review can have an email context, + +00:17:37.480 --> 00:17:40.559 +not just links to an external URL. + +NOTE Real Workflow: P.A.R.A. + +00:17:40.560 --> 00:17:46.519 +In the P.A.R.A Our Project, Areas, Resources and + +00:17:46.520 --> 00:17:50.039 +Archives structure, you can have Gmail labels + +00:17:50.040 --> 00:17:51.199 +mirror your PARA structure. + +00:17:51.200 --> 00:17:53.879 +You can have that within your org mode structure, + +00:17:53.880 --> 00:17:56.239 +and you can mimic that very well. + +00:17:56.240 --> 00:17:59.419 +All the emails embedded in your own structure and + +00:17:59.420 --> 00:18:02.067 +you can download it by label + +00:18:02.068 --> 00:18:03.520 +in the place you want. + +00:18:03.521 --> 00:18:05.439 +I'm still working on that feature where you can + +00:18:05.440 --> 00:18:08.559 +have it inside the project structure itself rather + +00:18:08.560 --> 00:18:09.839 +than an index.org. + +00:18:09.840 --> 00:18:11.719 +It'll take some time, but I will do it. + +00:18:11.720 --> 00:18:14.839 +But right now you can refile it once it's in the + +00:18:14.840 --> 00:18:17.459 +index.org or any other file you choose, and then + +00:18:17.460 --> 00:18:21.039 +you can bulk move labels to archive and move that + +00:18:21.040 --> 00:18:23.319 +entire project repository to + +00:18:23.320 --> 00:18:26.319 +your archive also. That works very well. + +00:18:26.320 --> 00:18:29.919 +And your org files and Gmail stay in sync + +00:18:29.920 --> 00:18:33.039 +effortlessly, and the whole email part of it + +00:18:33.040 --> 00:18:35.239 +becomes part of your knowledge management system. + +00:18:35.240 --> 00:18:35.959 +Okay. + +NOTE What this is NOT + +00:18:35.960 --> 00:18:37.639 +We have reached the end of the demo. + +00:18:37.640 --> 00:18:39.667 +Hopefully you understood + +00:18:39.668 --> 00:18:41.799 +what org-gmail was all about. + +00:18:41.800 --> 00:18:44.639 +If you have any questions, let me know, but some + +00:18:44.640 --> 00:18:47.519 +bits of warning, I wanted to give you what this + +00:18:47.520 --> 00:18:50.239 +package is not, it's not a full fledged email + +00:18:50.240 --> 00:18:50.959 +client. + +00:18:50.960 --> 00:18:55.319 +It's meant for label management and importing some + +00:18:55.320 --> 00:18:58.739 +emails that you wanted or you're interested in to + +00:18:58.740 --> 00:19:02.079 +the context of your projects or areas or your GTD + +00:19:02.080 --> 00:19:02.679 +context. + +00:19:02.680 --> 00:19:07.159 +It is not a replacement for gnus, mu4e or notmuch. + +00:19:07.160 --> 00:19:10.479 +It is not a way to read all your emails in emacs. + +00:19:10.480 --> 00:19:13.439 +It cannot handle a large server load for sure. + +00:19:13.440 --> 00:19:15.679 +It is not offline capable. + +00:19:15.680 --> 00:19:17.359 +It needs API access. + +00:19:17.360 --> 00:19:19.479 +So bear that in mind. + +00:19:19.480 --> 00:19:22.159 +What it is, is a bridge between + +00:19:22.160 --> 00:19:23.879 +gmail and org mode. + +00:19:23.880 --> 00:19:27.359 +It's a way to manage important email threads. + +00:19:27.360 --> 00:19:29.559 +So you can keep updating threads. + +00:19:29.560 --> 00:19:32.039 +Whenever there's a reply, you keep downloading it + +00:19:32.040 --> 00:19:32.919 +to that thread. + +00:19:32.920 --> 00:19:35.279 +If you want to keep track of what's going on, what + +00:19:35.280 --> 00:19:37.079 +is the logical way you can all see it. + +00:19:37.080 --> 00:19:40.599 +You can even use ellama or one of those to make + +00:19:40.600 --> 00:19:41.999 +sense of the conversation. + +00:19:42.000 --> 00:19:45.279 +If it's a long longish thread, it's a power tool + +00:19:45.280 --> 00:19:49.079 +for the getting things done or Tiago Forte's PARA + +00:19:49.080 --> 00:19:49.839 +method. + +00:19:49.840 --> 00:19:52.119 +I've used PARA for a long time now. + +00:19:52.120 --> 00:19:54.839 +I'm a power user of PARA, so to speak, so I find + +00:19:54.840 --> 00:19:56.319 +this extremely useful myself. + +00:19:56.320 --> 00:19:59.067 +It's about 800 lines of Python + +00:19:59.068 --> 00:20:02.479 +and about 300 to 500 lines of elisp. + +00:20:02.480 --> 00:20:06.999 +It's usable in 10 minutes, but can remain powerful + +00:20:07.000 --> 00:20:07.679 +for years. + +NOTE Technical decisions + +00:20:07.680 --> 00:20:10.599 +So some technical decisions that I took. + +00:20:10.600 --> 00:20:13.419 +Why Python plus Gmail, API. + +00:20:13.420 --> 00:20:17.959 +Gmail API is better at handling than an imap with + +00:20:17.960 --> 00:20:19.359 +all the other metadata. + +00:20:19.360 --> 00:20:20.199 +I found it easier. + +00:20:20.200 --> 00:20:22.679 +Python has excellent Google API libraries. + +00:20:22.680 --> 00:20:25.479 +Email calls Python via the call-process. + +00:20:25.480 --> 00:20:27.559 +json is the interchange format. + +00:20:27.560 --> 00:20:29.879 +Why not pure elisp? + +00:20:29.880 --> 00:20:33.799 +For one, OAuth 2.0 flow is a bit complex. + +00:20:33.800 --> 00:20:35.719 +I found it a bit complex to meander on. + +00:20:35.720 --> 00:20:38.139 +That's probably why it took me three years, 364 + +00:20:38.140 --> 00:20:40.319 +days to get over it. + +00:20:40.320 --> 00:20:43.839 +Gmail API Client libraries are mature, easier to + +00:20:43.840 --> 00:20:48.459 +test/debug separately and lets emacs do what it + +00:20:48.460 --> 00:20:50.159 +does best, which is text editing. + +00:20:50.160 --> 00:20:54.759 +Pragmatism over purity, the emacs way since 1976. + +NOTE Roadmap + +00:20:54.760 --> 00:20:57.159 +Okay, some roadmap here. + +00:20:57.160 --> 00:21:00.239 +But near term I want better error messages. + +00:21:00.240 --> 00:21:02.759 +It still gives me python error messages. + +00:21:02.760 --> 00:21:05.039 +There are some asynchronous operations there's no + +00:21:05.040 --> 00:21:06.319 +blocking going on. + +00:21:06.320 --> 00:21:08.879 +It needs a search integration soon. + +00:21:08.880 --> 00:21:11.119 +Attachment I have not yet touched. + +00:21:11.120 --> 00:21:13.879 +That's a big big if, I don't know how to integrate + +00:21:13.880 --> 00:21:15.879 +with org-attach, I'm still wondering how to do + +00:21:15.880 --> 00:21:16.439 +that. + +00:21:16.440 --> 00:21:19.519 +Perhaps I should be able to interact with Outlook + +00:21:19.520 --> 00:21:21.919 +also, fast mail, proton mail. + +00:21:21.920 --> 00:21:24.759 +I don't know AI summaries of thread. + +00:21:24.760 --> 00:21:27.759 +I'm thinking ellama could do it, but I'm not very + +00:21:27.760 --> 00:21:28.319 +sure. + +00:21:28.320 --> 00:21:32.079 +Calendar integration is a nice idea, but org-gcal + +00:21:32.080 --> 00:21:33.319 +and org gmail... + +00:21:33.320 --> 00:21:34.279 +How will they interact? + +00:21:34.280 --> 00:21:35.839 +How will that work? + +00:21:35.840 --> 00:21:37.759 +I'm still not sure. I use them separately. + +00:21:37.760 --> 00:21:40.599 +And any other feature requests that you may have, + +00:21:40.600 --> 00:21:41.439 +just let me know. + +NOTE Contributing + +00:21:41.440 --> 00:21:44.919 +What I may need help with if you have the time and + +00:21:44.920 --> 00:21:47.439 +you're interested in this project, is to test it + +00:21:47.440 --> 00:21:49.079 +on macOS and Windows. + +00:21:49.080 --> 00:21:50.159 +I use Linux. + +00:21:50.160 --> 00:21:51.759 +I use it on Debian. + +00:21:51.760 --> 00:21:52.719 +It works fine. + +00:21:52.720 --> 00:21:54.359 +OAuth edge cases. + +00:21:54.360 --> 00:21:55.719 +I'm not sure how it works. + +00:21:55.720 --> 00:21:57.519 +It shows me some error or the other + +00:21:57.520 --> 00:21:58.119 +here and there. + +00:21:58.120 --> 00:22:00.919 +Definitely documentation needs improvements. + +00:22:00.920 --> 00:22:04.239 +Other email provider expertise will be welcome. + +00:22:04.240 --> 00:22:07.359 +What's ready is, GitHub repo with issues. + +00:22:07.360 --> 00:22:10.239 +You can start with that could be great if you can + +00:22:10.240 --> 00:22:11.879 +tell me some issues with that. + +00:22:11.880 --> 00:22:13.879 +Some kind of development guide. + +00:22:13.880 --> 00:22:15.479 +I am not a programmer. + +00:22:15.480 --> 00:22:18.079 +I vibe-coded most of it. + +00:22:18.080 --> 00:22:20.919 +So a development guide, a true blood developer, + +00:22:20.920 --> 00:22:22.479 +if they can come and tell me, + +00:22:22.480 --> 00:22:23.999 +here is what you should be doing, + +00:22:24.000 --> 00:22:26.239 +I'm more than happy to listen to that. + +00:22:26.240 --> 00:22:27.959 +And probably a test suite. + +00:22:27.960 --> 00:22:28.959 +I do that manually. + +00:22:28.960 --> 00:22:31.239 +All of this, some kind of help with that + +00:22:31.240 --> 00:22:32.939 +will also work. + +NOTE The big picture + +00:22:32.940 --> 00:22:36.559 +The big picture is org-mode and Gmail + +00:22:36.560 --> 00:22:37.679 +can be friends. + +00:22:37.680 --> 00:22:41.119 +They can bond over a cup of coffee. + +NOTE Let's connect + +00:22:41.120 --> 00:22:41.959 +Let's connect. + +00:22:41.960 --> 00:22:47.659 +Here are my details and I am all game to listen to + +00:22:47.660 --> 00:22:48.679 +your question and answers. + +00:22:48.680 --> 00:22:51.279 +I'm happy to give you any answer or responses that + +00:22:51.280 --> 00:22:51.999 +I find. + +00:22:52.000 --> 00:22:54.439 +Please do connect with me on LinkedIn. I have my + +00:22:54.440 --> 00:22:58.359 +website here, and please do fork or install + +00:22:58.360 --> 00:23:00.519 +org-gmail and let me know what you think. + +00:23:00.520 --> 00:23:02.759 +Let's talk about taming email. + +00:23:02.760 --> 00:23:04.400 +Thank you very much. diff --git a/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main--chapters.vtt b/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main--chapters.vtt new file mode 100644 index 00000000..fc516878 --- /dev/null +++ b/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main--chapters.vtt @@ -0,0 +1,50 @@ +WEBVTT + + +00:00:02.620 --> 00:01:25.239 +Introduction + +00:01:25.240 --> 00:02:49.959 +Demo + +00:02:49.960 --> 00:03:58.559 +Don't panic + +00:03:58.560 --> 00:05:46.239 +Configuring servers + +00:05:46.240 --> 00:06:26.599 +.authinfo + +00:06:26.600 --> 00:08:25.719 +Configuration + +00:08:25.720 --> 00:09:40.079 +Starting Gnus + +00:09:40.080 --> 00:10:19.899 +Always showing groups + +00:10:19.900 --> 00:11:30.119 +Reading messages + +00:11:30.120 --> 00:12:55.159 +Debugging IMAP + +00:12:55.160 --> 00:14:25.559 +Topics + +00:14:25.560 --> 00:15:24.319 +Customizing message display + +00:15:24.320 --> 00:17:26.659 +Sending emails + +00:17:26.660 --> 00:19:27.959 +Plans + +00:19:27.960 --> 00:20:12.759 +Wrapping up + +00:20:12.760 --> 00:21:37.760 +nnimap diff --git a/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main.vtt b/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main.vtt new file mode 100644 index 00000000..c4e86336 --- /dev/null +++ b/2025/captions/emacsconf-2025-gnus--reading-and-writing-emails-in-gnu-emacs-with-gnus--amin-bandali--main.vtt @@ -0,0 +1,1332 @@ +WEBVTT captioned by sachac + +NOTE Introduction + +00:00:02.620 --> 00:00:04.799 +Hello, my name is Amin Bandali, + +00:00:04.800 --> 00:00:06.359 +and today I'd like to talk about + +00:00:06.360 --> 00:00:08.799 +reading and writing emails in GNU Emacs + +00:00:08.800 --> 00:00:14.319 +using Gnus specifically. + +00:00:14.320 --> 00:00:16.879 +Gnus has had this sort of reputation + +00:00:16.880 --> 00:00:20.599 +of being difficult to approach and configure. + +00:00:20.600 --> 00:00:23.359 +That's understandable + +00:00:23.360 --> 00:00:26.319 +because it has many, many options + +00:00:26.320 --> 00:00:27.679 +and major and minor modes + +00:00:27.680 --> 00:00:30.679 +that interact in different ways with each other. + +00:00:30.680 --> 00:00:35.319 +And it also doesn't help that Gnus started originally + +00:00:35.320 --> 00:00:36.359 +as a newsreader + +00:00:36.360 --> 00:00:38.759 +rather than a mail client. + +00:00:38.760 --> 00:00:40.879 +So a lot of the terminology that it uses + +00:00:40.880 --> 00:00:42.519 +is also rooted in that, + +00:00:42.520 --> 00:00:45.559 +in reading and writing news. + +00:00:45.560 --> 00:00:48.119 +But nevertheless, with this video and talk, + +00:00:48.120 --> 00:00:52.159 +I hope to provide a sort + +00:00:52.160 --> 00:00:55.759 +of very quick introduction + +00:00:55.760 --> 00:00:57.539 +of starting to use Gnus + +00:00:57.540 --> 00:01:00.919 +to read and write email and send it. + +00:01:00.920 --> 00:01:02.679 +We will use Gnus' IMAP support, + +00:01:02.680 --> 00:01:06.119 +mainly because a lot of people + +00:01:06.120 --> 00:01:08.679 +these days have email accounts + +00:01:08.680 --> 00:01:10.759 +with mail service providers + +00:01:10.760 --> 00:01:12.039 +that support IMAP, + +00:01:12.040 --> 00:01:14.319 +which is an open standard. + +00:01:14.320 --> 00:01:17.479 +So it's widely available and supported + +00:01:17.480 --> 00:01:19.719 +across many different providers + +00:01:19.720 --> 00:01:25.239 +as well as mail clients or mail user agents as well. + +NOTE Demo + +00:01:25.240 --> 00:01:30.559 +Okay, so let's just jump straight right in. + +00:01:30.560 --> 00:01:34.279 +I will enter this demo directory that I created + +00:01:34.280 --> 00:01:36.919 +for the purposes of this demonstration + +00:01:36.920 --> 00:01:40.999 +and change my home directory to this one + +00:01:41.000 --> 00:01:49.839 +so that we can safely experiment with Gnus here. + +00:01:49.840 --> 00:01:53.979 +For this presentation, I've written up + +00:01:53.980 --> 00:01:56.839 +a quick initialization file or init file + +00:01:56.840 --> 00:01:59.719 +that I will share afterwards as well + +00:01:59.720 --> 00:02:01.639 +to get us going with Gnus. + +00:02:01.640 --> 00:02:04.519 +There's not much to it at the moment. + +00:02:04.520 --> 00:02:07.399 +Just set up the package archives and + +00:02:07.400 --> 00:02:09.479 +install the keycast package + +00:02:09.480 --> 00:02:14.079 +for showing the key presses in the mode line. + +00:02:14.080 --> 00:02:15.359 +Yeah, that's about it. + +00:02:15.360 --> 00:02:16.239 +And I'll also define + +00:02:16.240 --> 00:02:20.279 +a little like inline function +emacs.d + +00:02:20.280 --> 00:02:24.079 +that allows me to conveniently write + +00:02:24.080 --> 00:02:26.639 +and have it expanded + +00:02:26.640 --> 00:02:29.300 +or refer to files and directories, rather, + +00:02:29.301 --> 00:02:30.900 +paths that we could expand, + +00:02:30.901 --> 00:02:32.833 +inside my Emacs configuration directory. + +00:02:32.834 --> 00:02:37.500 +I also have this eval-last-sexp + +00:02:37.501 --> 00:02:41.119 +bound to a global key, + +00:02:41.120 --> 00:02:43.279 +so that I will be able to easily + +00:02:43.280 --> 00:02:47.519 +use it for this talk. + +00:02:47.520 --> 00:02:49.959 +Okay, let's jump right in. + +NOTE Don't panic + +00:02:49.960 --> 00:02:52.239 +First things first, don't panic. + +00:02:52.240 --> 00:02:55.267 +And that's actually also the name + +00:02:55.268 --> 00:02:58.359 +of the very first node + +00:02:58.360 --> 00:03:01.559 +in the Gnus manual when you open it. + +00:03:01.560 --> 00:03:02.839 +And it's actually nice. + +00:03:02.840 --> 00:03:04.479 +I definitely, definitely recommend + +00:03:04.480 --> 00:03:07.079 +that you look through + +00:03:07.080 --> 00:03:10.199 +at least the very first couple of chapters of this, + +00:03:10.200 --> 00:03:14.199 +skim through it, and later on refer to it + +00:03:14.200 --> 00:03:16.133 +whenever you find something confusing + +00:03:16.134 --> 00:03:19.499 +or don't understand it. + +00:03:19.500 --> 00:03:21.359 +But yeah, we'll start + +00:03:21.360 --> 00:03:22.399 +with these two paragraphs here. + +00:03:22.400 --> 00:03:23.639 +So again, a Gnus installation + +00:03:23.640 --> 00:03:28.119 +is basically just a list of one or more servers + +00:03:28.120 --> 00:03:30.119 +and the subscribed groups from those servers + +00:03:30.120 --> 00:03:32.319 +and articles in those groups. + +00:03:32.320 --> 00:03:34.279 +You can already kind of see + +00:03:34.280 --> 00:03:39.479 +where that influence of a newsreader comes in. + +00:03:39.480 --> 00:03:41.839 +But yeah, basically what it's saying is that, + +00:03:41.840 --> 00:03:43.839 +you know, we have one or more servers. + +00:03:43.840 --> 00:03:47.079 +We can think of them as email servers. + +00:03:47.080 --> 00:03:49.359 +Groups can be like, we can think + +00:03:49.360 --> 00:03:52.959 +of them as folders or directories. + +00:03:52.960 --> 00:03:55.239 +And yeah, articles, + +00:03:55.240 --> 00:03:58.559 +those would be like our email messages. + +NOTE Configuring servers + +00:03:58.560 --> 00:03:59.679 +With Gnus, we can add + +00:03:59.680 --> 00:04:06.119 +and configure servers mainly using two variables. + +00:04:06.120 --> 00:04:07.919 +One of them is the gnus-select-method + +00:04:07.920 --> 00:04:11.479 +and the other is gnus-secondary-select-methods. + +00:04:11.480 --> 00:04:15.759 +The first one predates the second one + +00:04:15.760 --> 00:04:17.559 +and I generally don't recommend using it, because + +00:04:17.560 --> 00:04:22.559 +first of all, it can only point + +00:04:22.560 --> 00:04:26.359 +to one server, and that server, + +00:04:26.360 --> 00:04:27.879 +because it's the primary, + +00:04:27.880 --> 00:04:32.559 +then Gnus won't add a prefix to its groups, + +00:04:32.560 --> 00:04:34.839 +so later on, as you get into + +00:04:34.840 --> 00:04:36.679 +more advanced features of Gnus + +00:04:36.680 --> 00:04:38.519 +and, for example, want to write rules + +00:04:38.520 --> 00:04:42.959 +to modify your message composition + +00:04:42.960 --> 00:04:47.039 +in a way for certain groups, or file mail, + +00:04:47.040 --> 00:04:48.799 +automatically classify mail, + +00:04:48.800 --> 00:04:51.879 +this distinction can become + +00:04:51.880 --> 00:04:53.959 +confusing and annoying. + +00:04:53.960 --> 00:04:57.199 +My recommendation is to always and only use + +00:04:57.200 --> 00:05:01.799 +the gnus-secondary-select-methods. + +00:05:01.800 --> 00:05:07.319 +Yeah, so let's do that here. + +00:05:07.320 --> 00:05:10.299 +I'm gonna uncomment this portion. + +00:05:10.300 --> 00:05:16.419 +So here, I set the primary select method to nil, + +00:05:16.420 --> 00:05:24.159 +and the second one, I define an nnimap server + +00:05:24.160 --> 00:05:30.039 +of the nnimap backend. + +00:05:30.040 --> 00:05:32.439 +I give it the name ec25gnus. + +00:05:32.440 --> 00:05:35.879 +What I want it to do is to + +00:05:35.880 --> 00:05:37.799 +connect to my mail server, + +00:05:37.800 --> 00:05:41.079 +which is at this address, + +00:05:41.080 --> 00:05:46.239 +and fetch emails from it over TLS with this username. + +NOTE .authinfo + +00:05:46.240 --> 00:05:50.719 +And then the passwords or the credentials, + +00:05:50.720 --> 00:05:56.839 +you can put them in the .authinfo file. + +00:05:56.840 --> 00:05:58.799 +Normally, you would want to, for example, + +00:05:58.800 --> 00:06:03.719 +encrypt this file with your GPG key. + +00:06:03.720 --> 00:06:06.719 +But for this demonstration, I haven't. + +00:06:06.720 --> 00:06:10.479 +So yeah, the format is the keyword "machine" + +00:06:10.480 --> 00:06:15.239 +followed by the name of your Gnus server or account, + +00:06:15.240 --> 00:06:17.199 +followed by the word "login", + +00:06:17.200 --> 00:06:19.199 +then your login username, + +00:06:19.200 --> 00:06:23.959 +and then the password, which here it's not shown. + +00:06:23.960 --> 00:06:26.599 +Yeah. + +NOTE Configuration + +00:06:26.600 --> 00:06:28.679 +But before we actually set this, + +00:06:28.680 --> 00:06:31.479 +I'll just show you that if we like start Gnus + +00:06:31.480 --> 00:06:33.719 +with M-x gnus, + +00:06:33.720 --> 00:06:36.439 +initially, it will just show + +00:06:36.440 --> 00:06:37.759 +an error like this. + +00:06:37.760 --> 00:06:40.399 +Even if we continue, it's empty. + +00:06:40.400 --> 00:06:43.399 +There's not much because Gnus doesn't know + +00:06:43.400 --> 00:06:47.039 +where to fetch these emails from. + +00:06:47.040 --> 00:06:52.159 +And that's what we will configure. + +00:06:52.160 --> 00:06:55.859 +Excuse me. + +00:06:55.860 --> 00:06:57.559 +Yeah, so just for convenience, + +00:06:57.560 --> 00:06:59.079 +we can bind Gnus to, + +00:06:59.080 --> 00:07:00.679 +for example, C-c g, as I've done here. + +00:07:00.680 --> 00:07:04.119 +You will want to set your name + +00:07:04.120 --> 00:07:05.799 +and email address, like so. + +00:07:05.800 --> 00:07:09.239 +Here we tell Emacs + +00:07:09.240 --> 00:07:11.439 +that we are going to be using Gnus for reading email, + +00:07:11.440 --> 00:07:12.839 +because Emacs comes + +00:07:12.840 --> 00:07:14.759 +with other email clients as well, + +00:07:14.760 --> 00:07:18.559 +such as Rmail, and in fact, defaults to Rmail, + +00:07:18.560 --> 00:07:24.839 +so this way, we tell it to use Gnus. + +00:07:24.840 --> 00:07:31.559 +By default, Gnus puts its newsrc file and other files, + +00:07:31.560 --> 00:07:34.319 +I believe it still scatters them + +00:07:34.320 --> 00:07:35.439 +in a few different directories + +00:07:35.440 --> 00:07:36.279 +in your home directory, + +00:07:36.280 --> 00:07:37.399 +so it's a little bit messy. + +00:07:37.400 --> 00:07:40.039 +So what I prefer to do is to just put it + +00:07:40.040 --> 00:07:42.439 +all under the Gnus directory + +00:07:42.440 --> 00:07:47.439 +inside of my Emacs configuration, as I do here. + +00:07:47.440 --> 00:07:50.639 +Yeah, and then here we just tell Gnus + +00:07:50.640 --> 00:07:53.319 +to, like, don't try to bother + +00:07:53.320 --> 00:07:55.759 +with a generic newsrc file + +00:07:55.760 --> 00:07:57.119 +that would be shared + +00:07:57.120 --> 00:07:58.399 +with other news readers. + +00:07:58.400 --> 00:07:59.679 +Just want to use it for email. + +00:07:59.680 --> 00:08:01.959 +And yeah, so we just tell Gnus + +00:08:01.960 --> 00:08:03.039 +to keep all of its data + +00:08:03.040 --> 00:08:08.079 +inside a dedicated .newsrc.eld + +00:08:08.080 --> 00:08:12.159 +(for Emacs Lisp data) file instead. + +00:08:12.160 --> 00:08:15.199 +And we can also have Gnus not prompt us + +00:08:15.200 --> 00:08:19.679 +when we want to exit with q. + +00:08:19.680 --> 00:08:23.399 +Anyway, so let's go ahead and evaluate this. + +00:08:23.400 --> 00:08:25.719 +So this has been set, + +NOTE Starting Gnus + +00:08:25.720 --> 00:08:32.267 +so if we type M-x gnus again, or hit C-c g, + +00:08:32.268 --> 00:08:35.699 +now we're faced with an empty buffer, + +00:08:35.700 --> 00:08:37.399 +and it says no news is good news, + +00:08:37.400 --> 00:08:38.399 +and that's actually + +00:08:38.400 --> 00:08:40.719 +one of the characteristics of Gnus + +00:08:40.720 --> 00:08:44.779 +is that by default it tries + +00:08:44.780 --> 00:08:47.619 +to like sort of declutter + +00:08:47.620 --> 00:08:49.199 +and show us a little less possible + +00:08:49.200 --> 00:08:50.819 +in the group buffer, + +00:08:50.820 --> 00:08:53.259 +meaning that if you don't have + +00:08:53.260 --> 00:08:55.639 +any groups with unread or marked + +00:08:55.640 --> 00:09:00.119 +or, like, starred messages, it will not show them. + +00:09:00.120 --> 00:09:03.959 +To actually see all of our groups or folders, + +00:09:03.960 --> 00:09:08.359 +we hit shift L or capital L, + +00:09:08.360 --> 00:09:12.419 +and we see that we have an inbox here, + +00:09:12.420 --> 00:09:14.879 +as expected. So we enter the inbox, + +00:09:14.880 --> 00:09:17.459 +and we see that there is an article there + +00:09:17.460 --> 00:09:20.779 +and it's already been marked as read. + +00:09:20.780 --> 00:09:22.679 +But if we mark it as unread + +00:09:22.680 --> 00:09:25.959 +and exit and enter Gnus again, + +00:09:25.960 --> 00:09:27.279 +this is what we would see. + +00:09:27.280 --> 00:09:28.839 +We would see that our group + +00:09:28.840 --> 00:09:34.099 +and then we enter it, we see our mail here. + +00:09:34.100 --> 00:09:36.159 +Yeah, and this is our very first email + +00:09:36.160 --> 00:09:40.079 +that we read in GNU Emacs here, inside Gnus. + +NOTE Always showing groups + +00:09:40.080 --> 00:09:43.839 +It might be useful to have Gnus always show + +00:09:43.840 --> 00:09:46.839 +certain groups or folders + +00:09:46.840 --> 00:09:48.319 +even if they don't have + +00:09:48.320 --> 00:09:52.339 +anything unread or marked inside of them. + +00:09:52.340 --> 00:09:56.039 +The way we can do that is + +00:09:56.040 --> 00:09:57.599 +by setting this variable + +00:09:57.600 --> 00:10:01.339 +gnus-permanently-visible-groups + +00:10:01.340 --> 00:10:03.039 +to a regular expression + +00:10:03.040 --> 00:10:09.119 +that describes the name of these groups. + +00:10:09.120 --> 00:10:11.539 +So if we launch Gnus again, + +00:10:11.540 --> 00:10:14.759 +this time, we see that that group is visible, + +00:10:14.760 --> 00:10:19.899 +even though there's no unread messages in it. + +NOTE Reading messages + +00:10:19.900 --> 00:10:24.399 +When we enter a group or folder, + +00:10:24.400 --> 00:10:26.719 +we will see a list of all of our messages. + +00:10:26.720 --> 00:10:27.799 +Here, we only have one. + +00:10:27.800 --> 00:10:31.939 +We can press M-u or Alt-u + +00:10:31.940 --> 00:10:34.679 +to mark something as unread. + +00:10:34.680 --> 00:10:38.539 +You can press d to mark it as read. + +00:10:38.540 --> 00:10:40.079 +If you press just u, + +00:10:40.080 --> 00:10:41.959 +it'll tick the article, + +00:10:41.960 --> 00:10:44.039 +which is kind of the equivalent + +00:10:44.040 --> 00:10:46.999 +of marking the message or email + +00:10:47.000 --> 00:10:50.539 +as starred in other email clients + +00:10:50.540 --> 00:10:55.719 +such as Thunderbird. + +00:10:55.720 --> 00:11:00.639 +We see that when there are groups + +00:11:00.640 --> 00:11:03.959 +that have starred or ticked messages + +00:11:03.960 --> 00:11:04.679 +inside of them, + +00:11:04.680 --> 00:11:05.599 +Gnus will mark them + +00:11:05.600 --> 00:11:16.019 +with this little star here, or asterisk. + +00:11:16.020 --> 00:11:17.639 +This talk is just barely + +00:11:17.640 --> 00:11:19.039 +scratching the surface. + +00:11:19.040 --> 00:11:21.080 +Let's see how far... + +00:11:21.081 --> 00:11:22.759 +How am I doing with the time? + +00:11:22.760 --> 00:11:30.119 +Okay, 11 minutes already. + +NOTE Debugging IMAP + +00:11:30.120 --> 00:11:32.079 +Just a couple of helpful things here, + +00:11:32.080 --> 00:11:36.919 +like this nnimap-record-commands variable. + +00:11:36.920 --> 00:11:38.519 +It's useful when you want to debug + +00:11:38.520 --> 00:11:40.119 +your IMAP setup with Gnus. + +00:11:40.120 --> 00:11:42.859 +If you set it to anything non-nil, + +00:11:42.860 --> 00:11:46.699 +it will log the commands that it runs + +00:11:46.700 --> 00:11:49.539 +to a special `*imap log*` buffer. + +00:11:49.540 --> 00:11:50.719 +And here I just set it + +00:11:50.720 --> 00:11:52.679 +to this init-file-debug variable, + +00:11:52.680 --> 00:11:55.159 +which is set to non-nil + +00:11:55.160 --> 00:11:56.439 +whenever you launch Emacs + +00:11:56.440 --> 00:11:59.279 +with the --debug-init switch, + +00:11:59.280 --> 00:12:02.239 +so that's pretty helpful. + +00:12:02.240 --> 00:12:05.119 +You want to also set your sent folder, + +00:12:05.120 --> 00:12:07.479 +basically, where Gnus will save + +00:12:07.480 --> 00:12:09.439 +a copy of the message that you just sent. + +00:12:09.440 --> 00:12:12.799 +Normally, I think the convention these days is, + +00:12:12.800 --> 00:12:16.599 +a lot of you know servers and clients + +00:12:16.600 --> 00:12:18.799 +use a dedicated sent folder, + +00:12:18.800 --> 00:12:24.339 +but with Gnus, I just prefer to use INBOX itself. + +00:12:24.340 --> 00:12:27.119 +Mainly because then I will have + +00:12:27.120 --> 00:12:28.759 +threading working for free, + +00:12:28.760 --> 00:12:31.939 +so I can read the entire thread + +00:12:31.940 --> 00:12:34.299 +of an email chain there in one place. + +00:12:34.300 --> 00:12:35.319 +Of course, we don't have to keep + +00:12:35.320 --> 00:12:38.899 +the messages in there forever. + +00:12:38.900 --> 00:12:42.079 +And in fact, Gnus has facilities, + +00:12:42.080 --> 00:12:43.479 +both manual and automated, + +00:12:43.480 --> 00:12:45.999 +for expiring emails into + +00:12:46.000 --> 00:12:52.679 +different locations or different folders. + +00:12:52.680 --> 00:12:55.159 +Yeah. So let's move on here. + +NOTE Topics + +00:12:55.160 --> 00:13:02.039 +Topics are another nice feature of Gnus. + +00:13:02.040 --> 00:13:03.279 +So this is useful + +00:13:03.280 --> 00:13:05.359 +for creating some topics + +00:13:05.360 --> 00:13:08.459 +and then classifying or grouping + +00:13:08.460 --> 00:13:10.599 +your directories there. + +00:13:10.600 --> 00:13:11.799 +So we will see the use + +00:13:11.800 --> 00:13:13.639 +of this in a moment, + +00:13:13.640 --> 00:13:17.019 +where, let's say, I want to add + +00:13:17.020 --> 00:13:19.999 +a second account to Gnus. + +00:13:20.000 --> 00:13:23.559 +This one I'm going to call ec25work. + +00:13:23.560 --> 00:13:24.679 +Let's pretend that + +00:13:24.680 --> 00:13:29.859 +this is my work email. + +00:13:29.860 --> 00:13:32.479 +So if we open Gnus now, + +00:13:32.480 --> 00:13:36.999 +we see that our work INBOX + +00:13:37.000 --> 00:13:37.959 +also shows up here. + +00:13:37.960 --> 00:13:41.299 +And because we enabled topic mode, + +00:13:41.300 --> 00:13:42.359 +we see that we have + +00:13:42.360 --> 00:13:43.439 +these sort of buttons + +00:13:43.440 --> 00:13:44.839 +like Gnus and misc here. + +00:13:44.840 --> 00:13:46.679 +And we can, I believe, + +00:13:46.680 --> 00:13:49.799 +create a topic with capital T n. + +00:13:49.800 --> 00:13:52.879 +We can call it personal, this one. + +00:13:52.880 --> 00:13:56.939 +Let's create another one, work. + +00:13:56.940 --> 00:13:59.579 +And then what we can do is go + +00:13:59.580 --> 00:14:02.799 +over the directory that we want, + +00:14:02.800 --> 00:14:04.759 +for example, this one, + +00:14:04.760 --> 00:14:08.219 +hit capital T m to move it + +00:14:08.220 --> 00:14:11.899 +to the personal topic, + +00:14:11.900 --> 00:14:13.079 +and this work one, + +00:14:13.080 --> 00:14:15.199 +move it to the work topic. + +00:14:15.200 --> 00:14:17.439 +So we can nicely classify + +00:14:17.440 --> 00:14:23.119 +and group our groups folders here, + +00:14:23.120 --> 00:14:24.719 +which is especially useful + +00:14:24.720 --> 00:14:25.559 +when you have hundreds of them. + +NOTE Customizing message display + +00:14:25.560 --> 00:14:29.759 +Anyhow, we can customize + +00:14:29.760 --> 00:14:35.039 +different aspects of message display. + +00:14:35.040 --> 00:14:35.839 +Like for example, + +00:14:35.840 --> 00:14:38.199 +we can this way customize + +00:14:38.200 --> 00:14:39.199 +and change the order of + +00:14:39.200 --> 00:14:41.599 +which headers we want to see and where. + +00:14:41.600 --> 00:14:45.199 +So if I launch Gnus + +00:14:45.200 --> 00:14:48.459 +and go back to this email here, + +00:14:48.460 --> 00:14:52.139 +these are the headers that we see at the top. + +00:14:52.140 --> 00:14:52.639 +Excuse me. + +00:14:52.640 --> 00:14:55.159 +And with Gnus we can always + +00:14:55.160 --> 00:14:57.799 +We can have it show all the headers + +00:14:57.800 --> 00:15:01.999 +by pressing t to toggle the headers. + +00:15:02.000 --> 00:15:04.579 +Here we can see all the nitty-gritty + +00:15:04.580 --> 00:15:06.359 +and all of the headers in the message + +00:15:06.360 --> 00:15:12.219 +and we can toggle it back with t again. + +00:15:12.220 --> 00:15:16.479 +We can modify and customize the sorting + +00:15:16.480 --> 00:15:20.019 +with dedicated sorting functions. + +00:15:20.020 --> 00:15:20.999 +It comes with a number of them + +00:15:21.000 --> 00:15:21.599 +out of the box + +00:15:21.600 --> 00:15:24.319 +but we can define them as well. + +NOTE Sending emails + +00:15:24.320 --> 00:15:29.759 +Now to send emails. Let's see. + +00:15:29.760 --> 00:15:30.999 +We will be using message, + +00:15:31.000 --> 00:15:34.939 +and that's what Gnus itself uses. + +00:15:34.940 --> 00:15:38.579 +So I will set things up here. + +00:15:38.580 --> 00:15:42.639 +Let's see. + +00:15:42.640 --> 00:15:44.519 +Okay, so first of all, + +00:15:44.520 --> 00:15:46.439 +we want to have Gnus mark + +00:15:46.440 --> 00:15:48.519 +the messages that we write to others + +00:15:48.520 --> 00:15:49.759 +as read automatically, + +00:15:49.760 --> 00:15:51.359 +so this option does that. + +00:15:51.360 --> 00:15:58.039 +And then we define posting styles this way + +00:15:58.040 --> 00:16:01.619 +using the prefix, the name + +00:16:01.620 --> 00:16:04.359 +of the IMAP server. + +00:16:04.360 --> 00:16:06.519 +And this is how we can tell it to use + +00:16:06.520 --> 00:16:09.199 +what email address for the From [header] + +00:16:09.200 --> 00:16:14.599 +and which SMTP server to send it with. + +00:16:14.600 --> 00:16:17.879 +Yeah, and then gcc is where Gnus will save + +00:16:17.880 --> 00:16:20.199 +the copy of the messages that we write. + +00:16:20.200 --> 00:16:24.139 +So if we go ahead and launch Gnus again. + +00:16:24.140 --> 00:16:26.279 +We can go into our personal email here, + +00:16:26.280 --> 00:16:28.919 +hit m to compose a new message. + +00:16:28.920 --> 00:16:33.559 +We can prepare an email to, + +00:16:33.560 --> 00:16:35.119 +let's say, our work address. + +00:16:35.120 --> 00:16:42.419 +Hello from EmacsConf 2025 Gnus talk. + +00:16:42.420 --> 00:16:47.639 +Hello, this is just a test. :) + +00:16:47.640 --> 00:16:55.739 +Yeah, and we hit send. + +00:16:55.740 --> 00:16:56.919 +The sending will be done + +00:16:56.920 --> 00:17:03.479 +using Emacs's built-in SMTP libraries. + +00:17:03.480 --> 00:17:05.119 +Sometimes it can take a moment. + +00:17:05.120 --> 00:17:07.599 +Okay, that's it. It's done. + +00:17:07.600 --> 00:17:09.259 +So if we go back out + +00:17:09.260 --> 00:17:11.559 +and if we hit g to get new news, + +00:17:11.560 --> 00:17:15.679 +we should be able to see our new email there + +00:17:15.680 --> 00:17:17.639 +in the other account that we just sent it to. + +00:17:17.640 --> 00:17:22.360 +So we can come here, open it, + +00:17:22.361 --> 00:17:26.659 +and there we go. + +NOTE Plans + +00:17:26.660 --> 00:17:29.239 +There is a lot to configure in Gnus, + +00:17:29.240 --> 00:17:31.439 +and we're just barely scratching the surface, + +00:17:31.440 --> 00:17:34.079 +and unfortunately I don't have the time + +00:17:34.080 --> 00:17:34.999 +to explain all of these + +00:17:35.000 --> 00:17:36.519 +but I do plan on doing + +00:17:36.520 --> 00:17:38.839 +a much longer running series, + +00:17:38.840 --> 00:17:41.499 +whether it's text or videos, + +00:17:41.500 --> 00:17:42.879 +showing how to configure + +00:17:42.880 --> 00:17:45.319 +and use a lot of these different aspects of Gnus. + +00:17:45.320 --> 00:17:49.519 +But yeah, here, near the end, just a couple of... + +00:17:49.520 --> 00:17:54.919 +quick things. I find it's nice to have message + +00:17:54.920 --> 00:17:56.519 +prompt us for [confirmation] + +00:17:56.520 --> 00:17:59.199 +that we do want to send a message. + +00:17:59.200 --> 00:18:01.359 +Actually, when it does that, I take + +00:18:01.360 --> 00:18:02.599 +another look over my email + +00:18:02.600 --> 00:18:07.059 +to make sure I don't have any typos. + +00:18:07.060 --> 00:18:09.519 +It's generally a good idea to wrap your messages + +00:18:09.520 --> 00:18:14.119 +around 70 or 72 characters. + +00:18:14.120 --> 00:18:16.619 +We do that here. + +00:18:16.620 --> 00:18:19.159 +We can tell Gnus to forward messages + +00:18:19.160 --> 00:18:22.599 +as a proper MIME part, + +00:18:22.600 --> 00:18:27.059 +instead of some half-broken way. + +00:18:27.060 --> 00:18:30.119 +This customization, the sendmail function, + +00:18:30.120 --> 00:18:34.239 +is how we tell Gnus with message + +00:18:34.240 --> 00:18:38.239 +to use the SMTP library to sending the email, + +00:18:38.240 --> 00:18:42.479 +and these two variables are useful for + +00:18:42.480 --> 00:18:45.959 +omitting our own email address + +00:18:45.960 --> 00:18:47.439 +when we want to send someone, + +00:18:47.440 --> 00:18:50.179 +like when we hit r, to reply to someone. + +00:18:50.180 --> 00:18:51.959 +if we configure these variables, + +00:18:51.960 --> 00:18:52.959 +then Gnus won't add + +00:18:52.960 --> 00:18:56.059 +our own address to the To or Cc, + +00:18:56.060 --> 00:18:58.479 +which is pretty useful. + +00:18:58.480 --> 00:18:59.919 +I also find it helpful + +00:18:59.920 --> 00:19:03.359 +to unbind C-c C-s. + +00:19:03.360 --> 00:19:04.974 +That's another key + +00:19:04.975 --> 00:19:06.319 +for sending the message [in addition to C-c C-c]. + +00:19:06.320 --> 00:19:09.719 +And because C-c C-d, + +00:19:09.720 --> 00:19:13.359 +which is very close to it on the QWERTY layout, + +00:19:13.360 --> 00:19:15.719 +is useful for saving a draft + +00:19:15.720 --> 00:19:16.839 +and then coming back to it, + +00:19:16.840 --> 00:19:20.079 +I don't want to accidentally hit C-c C-s, + +00:19:20.080 --> 00:19:22.039 +and send the message prematurely. + +00:19:22.040 --> 00:19:25.979 +So I unbind it. + +00:19:25.980 --> 00:19:27.959 +Yeah, anyway, that's about it. + +NOTE Wrapping up + +00:19:27.960 --> 00:19:31.039 +That's a kind of very quick tour + +00:19:31.040 --> 00:19:37.119 +and introduction of setting up Gnus. + +00:19:37.120 --> 00:19:40.719 +Here, we just configured a remote IMAP server, + +00:19:40.720 --> 00:19:43.519 +but we can also, of course, + +00:19:43.520 --> 00:19:46.359 +set up a local IMAP server such as Dovecot + +00:19:46.360 --> 00:19:48.399 +and point Gnus to there, + +00:19:48.400 --> 00:19:52.799 +and use programs like OfflineIMAP, I believe, + +00:19:52.800 --> 00:19:57.479 +or the mbsync program from isync package + +00:19:57.480 --> 00:20:02.939 +or isync project to synchronize our messages + +00:20:02.940 --> 00:20:04.479 +to local mail directories + +00:20:04.480 --> 00:20:06.279 +and then point Gnus to it. + +00:20:06.280 --> 00:20:08.359 +The reason we might want to use that + +00:20:08.360 --> 00:20:11.719 +is to always have a copy of our messages at hand + +00:20:11.720 --> 00:20:12.759 +so we can use offline. + +NOTE nnimap + +00:20:12.760 --> 00:20:17.439 +And why use nnimap specifically? + +00:20:17.440 --> 00:20:27.399 +As of now, the Maildir backend included with Gnus + +00:20:27.400 --> 00:20:29.679 +is very inefficient, + +00:20:29.680 --> 00:20:31.399 +especially when dealing with + +00:20:31.400 --> 00:20:33.839 +tens or hundreds of thousands of messages + +00:20:33.840 --> 00:20:36.659 +like some of us are. + +00:20:36.660 --> 00:20:38.759 +It just takes an eternity to try + +00:20:38.760 --> 00:20:43.259 +and index them and get going. + +00:20:43.260 --> 00:20:44.639 +In that case, what I recommend doing + +00:20:44.640 --> 00:20:47.799 +is instead of interfacing directly with Maildir, + +00:20:47.800 --> 00:20:52.359 +for Gnus, just install and run + +00:20:52.360 --> 00:20:54.359 +Dovecot, a local IMAP server, + +00:20:54.360 --> 00:20:59.819 +and point Gnus to that. + +00:20:59.820 --> 00:21:02.959 +I plan on writing tutorials or doing videos + +00:21:02.960 --> 00:21:06.639 +about these other aspects + +00:21:06.640 --> 00:21:10.519 +of configuring Gnus after the conference. + +00:21:10.520 --> 00:21:11.439 +That's about it for me, + +00:21:11.440 --> 00:21:14.119 +so I hope you find this helpful. + +00:21:14.120 --> 00:21:16.679 +If you have any questions, + +00:21:16.680 --> 00:21:18.239 +please feel free to email me + +00:21:18.240 --> 00:21:23.759 +at bandali@gnu.org or @kelar.org. + +00:21:23.760 --> 00:21:25.879 +You can take a look at my personal website + +00:21:25.880 --> 00:21:26.839 +where I plan on posting + +00:21:26.840 --> 00:21:31.059 +other Emacs and Gnus materials. + +00:21:31.060 --> 00:21:33.039 +And yeah, thank you for watching + +00:21:33.040 --> 00:21:35.159 +and I hope you enjoy the rest of the conference. + +00:21:35.160 --> 00:21:37.760 +Take care. diff --git a/2025/captions/emacsconf-2025-greader--gnu-emacs-greader-gnam-reader-mode-is-the-best-emacs-mode-in-existence--yuval-langer--main.vtt b/2025/captions/emacsconf-2025-greader--gnu-emacs-greader-gnam-reader-mode-is-the-best-emacs-mode-in-existence--yuval-langer--main.vtt new file mode 100644 index 00000000..780ff013 --- /dev/null +++ b/2025/captions/emacsconf-2025-greader--gnu-emacs-greader-gnam-reader-mode-is-the-best-emacs-mode-in-existence--yuval-langer--main.vtt @@ -0,0 +1,223 @@ +WEBVTT captioned by sachac + +NOTE Introduction + +00:00:01.460 --> 00:00:03.785 +Hi, I'm Yuval Langer. + +00:00:03.786 --> 00:00:09.479 +Some may know me as cow_2001 on IRC. + +00:00:09.480 --> 00:00:12.119 +I'd like to tell you about greader mode, + +00:00:12.120 --> 00:00:14.519 +a versatile text-to-speech package + +00:00:14.520 --> 00:00:18.399 +written by Michelangelo Rodriguez. + +00:00:18.400 --> 00:00:20.399 +Sometimes you want to read a bunch + +00:00:20.400 --> 00:00:23.039 +and cannot be bothered, right? + +00:00:23.040 --> 00:00:25.079 +You'd rather plop on your chair + +00:00:25.080 --> 00:00:27.519 +and let the words come to you. + +00:00:27.520 --> 00:00:31.157 +You can do it using greader Mode. + +NOTE What is greader mode? + +00:00:31.158 --> 00:00:33.119 +What is greader mode? + +00:00:33.120 --> 00:00:36.319 +Greader mode is a text-to-speech minor mode + +00:00:36.320 --> 00:00:40.399 +with which you can read any buffer using the point. + +00:00:40.400 --> 00:00:41.602 +You move your point + +00:00:41.603 --> 00:00:43.559 +right before the text you want to read + +00:00:43.560 --> 00:00:47.639 +and run greader-read command. + +00:00:47.640 --> 00:00:50.839 +You can then use the left and right arrow keys + +00:00:50.840 --> 00:00:56.599 +to jump to the previous sentence or the next sentence. + +NOTE Installing Greader + +00:00:56.600 --> 00:00:59.143 +Installing GReader: + +00:00:59.144 --> 00:01:05.439 +Greader is available on the GNU Emacs app store + +00:01:05.440 --> 00:01:07.285 +and its copyright assigned to + +00:01:07.286 --> 00:01:10.959 +the Free Software Foundation. + +00:01:10.960 --> 00:01:12.857 +To install Greader, + +00:01:12.858 --> 00:01:19.279 +you can run M-x list-packages RET. + +00:01:19.280 --> 00:01:23.099 +look it up with C-s greader, + +00:01:23.100 --> 00:01:26.679 +press i to mark it for installation, + +00:01:26.680 --> 00:01:31.759 +and then press x to execute the installation. + +NOTE Basic usage + +00:01:31.760 --> 00:01:33.211 +Basic usage: + +00:01:33.212 --> 00:01:37.559 +We can now open a text file and start reading. + +00:01:37.560 --> 00:01:42.599 +Let's open The Willows by Algernon Blackwood. + +00:01:42.600 --> 00:01:44.479 +I've never read the story, + +00:01:44.480 --> 00:01:48.279 +but HP Lovecraft said it was the best horror story + +00:01:48.280 --> 00:01:52.959 +he had ever read, so it is in my reading list. + +00:01:52.960 --> 00:02:01.519 +Now load greader using M-x greader-mode. + +00:02:01.520 --> 00:02:08.139 +To start reading, press C-r SPC. + +00:02:08.140 --> 00:02:10.559 +The Project Gutenberg ebook of The willows. + +00:02:10.560 --> 00:02:14.079 +This will run the greader-read command. + +00:02:14.080 --> 00:02:16.799 +To stop, press the SPC key. + +00:02:16.800 --> 00:02:20.819 +This will run the greader-stop command. + +NOTE Navigation + +00:02:20.820 --> 00:02:22.359 +Navigation: + +00:02:22.360 --> 00:02:24.679 +You can navigate like you normally do, + +00:02:24.680 --> 00:02:27.559 +but using the left or right arrow keys + +00:02:27.560 --> 00:02:30.199 +will move the point between sentences + +00:02:30.200 --> 00:02:33.087 +instead of characters. + +00:02:33.088 --> 00:02:36.639 +So... This ebook is... + +00:02:36.640 --> 00:02:38.095 +You may copy it, give it away, + +00:02:38.096 --> 00:02:41.479 +or reuse it if you are not. + +00:02:41.480 --> 00:02:43.580 +Let's move to the start of the story. + +00:02:57.040 --> 00:02:58.088 +"After leaving Vienna, + +00:02:58.089 --> 00:02:59.839 +and long before you come to Budapest, + +00:02:59.840 --> 00:03:00.919 +the Danube enters a region + +00:03:00.920 --> 00:03:02.919 +of singular loneliness and desolation, + +00:03:02.920 --> 00:03:04.879 +where its waters spread away on all sides, + +00:03:04.880 --> 00:03:06.199 +regardless of a main channel, + +00:03:06.200 --> 00:03:08.799 +and the country becomes a swamp for miles upon miles, + +00:03:08.800 --> 00:03:11.759 +covered by a vast sea of low willow bushes." + +NOTE Reading rate + +00:03:12.380 --> 00:03:15.839 +Reading rate: this reading rate is rather slow. + +00:03:15.840 --> 00:03:19.519 +Let's pick up the pace using the plus key. + +00:03:19.520 --> 00:03:23.519 +This will run the greader-inc-rate command. + +00:03:23.520 --> 00:03:26.780 +You must do that while greader is reading. + +00:03:37.885 --> 00:03:39.779 +Now it is too fast. + +00:03:39.780 --> 00:03:44.679 +We can slow down using the - key. + +00:03:44.680 --> 00:03:52.485 +This will run the greader-dec-rate command. + +00:03:54.560 --> 00:03:59.384 +"In high flood this great acreage + +00:03:59.385 --> 00:04:01.239 +of sand, shingle-beds, and willow-grown islands + +00:04:01.240 --> 00:04:02.439 +is almost topped by the water, + +00:04:02.440 --> 00:04:03.609 +but in normal seasons the bushes + +00:04:03.610 --> 00:04:04.919 +bend and rustle in the free winds, + +00:04:04.920 --> 00:04:06.399 +showing their silver leaves to the sunshine + +00:04:06.400 --> 00:04:08.320 +in an ever-moving plain of bewildering beauty." diff --git a/2025/captions/emacsconf-2025-juicemacs--juicemacs-exploring-speculative-jit-compilation-for-elisp-in-java--kana--main.vtt b/2025/captions/emacsconf-2025-juicemacs--juicemacs-exploring-speculative-jit-compilation-for-elisp-in-java--kana--main.vtt new file mode 100644 index 00000000..46820e94 --- /dev/null +++ b/2025/captions/emacsconf-2025-juicemacs--juicemacs-exploring-speculative-jit-compilation-for-elisp-in-java--kana--main.vtt @@ -0,0 +1,1238 @@ +WEBVTT captioned by kana + + +00:00:01.200 --> 00:00:02.803 +Hello! This is Kana! + +00:00:02.903 --> 00:00:04.367 +And today I'll be talking about + +00:00:04.368 --> 00:00:06.067 +<b>J</b>ust-<b>I</b>n-<b>T</b>ime compilation, or JIT, + +00:00:06.068 --> 00:00:07.363 +for Emacs Lisp, + +00:00:07.463 --> 00:00:11.163 +based on my work-in-progress Emacs clone, Juicemacs. + +00:00:11.263 --> 00:00:13.533 +Juicemacs aims to explore a few things + +00:00:13.534 --> 00:00:15.843 +that I've been wondering about for a while. + +00:00:15.943 --> 00:00:18.567 +For exmaple, what if we had better or even + +00:00:18.568 --> 00:00:21.223 +transparent concurrency in ELisp? + +00:00:21.323 --> 00:00:23.243 +Or, can we have a concurrent GUI? + +00:00:23.343 --> 00:00:26.783 +One that does not block, or is blocked by Lisp code? + +00:00:26.883 --> 00:00:31.067 +And finally what can JIT compilation do for ELisp? + +00:00:31.068 --> 00:00:34.083 +Will it provide better performance? + +00:00:34.183 --> 00:00:37.400 +However, a main problem with explorations + +00:00:37.401 --> 00:00:38.623 +in Emacs clones is that, + +00:00:38.723 --> 00:00:40.863 +Emacs is a whole universe. + +00:00:40.963 --> 00:00:43.600 +And that means, to make these explorations + +00:00:43.601 --> 00:00:45.383 +meaningful for Emacs users, + +00:00:45.483 --> 00:00:47.967 +we need to cover a lot of Emacs features, + +00:00:47.968 --> 00:00:50.543 +before we can ever begin. + +00:00:50.643 --> 00:00:53.923 +For example, one of the features of Emacs is that, + +00:00:54.023 --> 00:00:56.003 +it supports a lot of encodings. + +00:00:56.103 --> 00:00:59.267 +Let's look at this string: it can be encoded + +00:00:59.268 --> 00:01:03.643 +in both Unicode and Shift-JIS, a Japanese encoding system. + +00:01:03.743 --> 00:01:07.067 +But currently, Unicode does not have + +00:01:07.068 --> 00:01:09.803 +an official mapping for this "ki" (﨑) character. + +00:01:09.903 --> 00:01:12.767 +So when we map from Shift-JIS to Unicode, + +00:01:12.768 --> 00:01:14.423 +in most programming languages, + +00:01:14.523 --> 00:01:16.533 +you end up with something like this: + +00:01:16.534 --> 00:01:19.143 +it's a replacement character. + +00:01:19.243 --> 00:01:22.067 +But in Emacs, it actually extends + +00:01:22.068 --> 00:01:23.883 +the Unicode range by threefold, + +00:01:23.983 --> 00:01:26.833 +and uses the extra range to losslessly + +00:01:26.834 --> 00:01:29.483 +support characters like this. + +00:01:29.583 --> 00:01:31.923 +So if you want to support this feature, + +00:01:32.023 --> 00:01:34.033 +that basically rules out all string + +00:01:34.034 --> 00:01:37.243 +libraries with Unicode assumptions. + +00:01:37.843 --> 00:01:40.067 +For another, you need to support + +00:01:40.068 --> 00:01:41.883 +the regular expressions in Emacs, + +00:01:41.983 --> 00:01:45.023 +which are, really irregular. + +00:01:45.123 --> 00:01:46.900 +For example, it supports asserting + +00:01:46.901 --> 00:01:49.403 +about the user cursor position. + +00:01:49.503 --> 00:01:52.033 +And it also uses some character tables, + +00:01:52.034 --> 00:01:53.883 +that can be modified from Lisp code, + +00:01:53.983 --> 00:01:56.163 +to determine to case mappings. + +00:01:56.263 --> 00:01:59.567 +And all that makes it really hard, or even + +00:01:59.568 --> 00:02:05.123 +impossible to use any existing regexp libraries. + +00:02:05.223 --> 00:02:07.883 +Also, you need a functional garbage collector. + +00:02:07.983 --> 00:02:09.867 +You need threading primitives, because + +00:02:09.868 --> 00:02:12.323 +Emacs has already had some threading support. + +00:02:12.423 --> 00:02:14.533 +And you might want the performance of your clone + +00:02:14.534 --> 00:02:18.963 +to match Emacs, even with its native compilation enabled. + +00:02:19.063 --> 00:02:21.500 +Not to mention you also need a GUI for an editor. + +00:02:21.501 --> 00:02:23.543 +And so on. + +00:02:23.643 --> 00:02:25.633 +For Juicemacs, building on Java and + +00:02:25.634 --> 00:02:27.563 +a compiler framework called Truffle, + +00:02:27.663 --> 00:02:30.503 +helps in getting better performance; + +00:02:30.603 --> 00:02:32.933 +and by choosing a language with a good GC, + +00:02:32.934 --> 00:02:38.063 +we can actually focus more on the challenges above. + +00:02:38.163 --> 00:02:41.433 +Currently, Juicemacs has implemented three out of, + +00:02:41.434 --> 00:02:43.983 +at least four of the interpreters in Emacs. + +00:02:44.083 --> 00:02:46.363 +One for lisp code, one for bytecode, + +00:02:46.463 --> 00:02:48.567 +and one for regular expressions, + +00:02:48.568 --> 00:02:50.903 +all of them JIT-capable. + +00:02:51.003 --> 00:02:53.667 +Other than these, Emacs also has around + +00:02:53.668 --> 00:02:56.083 +two thousand built-in functions in C code. + +00:02:56.183 --> 00:02:57.333 +And Juicemacs has around + +00:02:57.334 --> 00:02:59.763 +four hundred of them implemented. + +00:02:59.863 --> 00:03:03.603 +It's not that many, but it is surprisingly enough + +00:03:03.703 --> 00:03:05.200 +to bootstrap Emacs and run + +00:03:05.201 --> 00:03:08.483 +the portable dumper, or pdump, in short. + +00:03:08.583 --> 00:03:11.243 +Let's have a try. + +00:03:11.343 --> 00:03:11.703 + + +00:03:11.803 --> 00:03:14.923 +So this is the binary produced by Java native image. + +00:03:15.023 --> 00:03:17.167 +And it's loading all the files + +00:03:17.168 --> 00:03:18.763 +needed for bootstrapping. + +00:03:18.863 --> 00:03:22.233 +Then it dumps the memory to a file to + +00:03:22.234 --> 00:03:24.923 +be loaded later, giving us fast startup. + +00:03:25.023 --> 00:03:28.723 +As we can see here, it throws some frame errors + +00:03:28.823 --> 00:03:31.400 +because Juicemacs doesn't have an editor UI + +00:03:31.401 --> 00:03:33.283 +or functional frames yet. + +00:03:33.383 --> 00:03:35.367 +But otherwise, it can already run + +00:03:35.368 --> 00:03:36.643 +quite some lisp code. + +00:03:36.743 --> 00:03:40.400 +For example, this code uses the benchmark library + +00:03:40.401 --> 00:03:44.403 +to measure the performance of this Fibonacci function. + +00:03:44.503 --> 00:03:47.067 +And we can see here, the JIT engine is + +00:03:47.068 --> 00:03:51.163 +already kicking in and makes the execution faster. + +00:03:51.263 --> 00:03:53.483 +In addition to that, with a bit of workaround, + +00:03:53.583 --> 00:03:56.467 +Juicemacs can also run some of the ERT, + +00:03:56.468 --> 00:04:01.043 +or, <b>E</b>macs <b>R</b>egression <b>T</b>est suite, that comes with Emacs. + +00:04:01.143 --> 00:04:05.823 +So... Yes, there are a bunch of test failures, + +00:04:05.923 --> 00:04:07.933 +which means we are not that compatible + +00:04:07.934 --> 00:04:09.523 +with Emacs and need more work. + +00:04:09.623 --> 00:04:12.803 +But the whole testing procedure runs fine, + +00:04:12.903 --> 00:04:14.767 +and it has proper stack traces, + +00:04:14.768 --> 00:04:17.803 +which is quite useful for debugging Juicemacs. + +00:04:17.903 --> 00:04:21.033 +So with that, a rather functional JIT runtime, + +00:04:21.034 --> 00:04:25.983 +let's now try look into today's topic, JIT compilation for ELisp. + +00:04:26.083 --> 00:04:28.533 +So, you probably know that Emacs has supported + +00:04:28.534 --> 00:04:32.083 +native-compilation, or nativecomp in short, for some time now. + +00:04:32.183 --> 00:04:35.033 +It mainly uses GCC to compile Lisp code + +00:04:35.034 --> 00:04:37.363 +into native code, ahead of time. + +00:04:37.463 --> 00:04:41.433 +And during runtime, Emacs loads those compiled files, + +00:04:41.434 --> 00:04:44.523 +and gets the performance of native code. + +00:04:44.623 --> 00:04:47.643 +However, for example, for installed packages, + +00:04:47.743 --> 00:04:49.059 +we might want to compile them when we + +00:04:49.060 --> 00:04:51.823 +actually use them instead of ahead of time. + +00:04:51.923 --> 00:04:53.733 +And Emacs supports this through + +00:04:53.734 --> 00:04:55.683 +this <i>native-comp-jit-compilation</i> flag. + +00:04:55.783 --> 00:04:59.767 +What it does is, during runtime, Emacs sends + +00:04:59.768 --> 00:05:03.203 +loaded files to external Emacs worker processes, + +00:05:03.303 --> 00:05:06.903 +which will then compile those files asynchronously. + +00:05:07.003 --> 00:05:09.043 +And when the compilation is done, + +00:05:09.143 --> 00:05:11.967 +the current Emacs session will load the compiled code back + +00:05:11.968 --> 00:05:16.323 +and improves its performance, on the fly. + +00:05:16.423 --> 00:05:18.643 +When you look at this procedure, however, it is, + +00:05:18.743 --> 00:05:21.563 +ahead-of-time compilation, done at runtime. + +00:05:21.663 --> 00:05:25.123 +And it is what current Emacs calls JIT compilation. + +00:05:25.223 --> 00:05:27.867 +But if you look at some other JIT engines, + +00:05:27.868 --> 00:05:31.803 +you'll see much more complex architectures. + +00:05:31.903 --> 00:05:34.233 +So, take luaJIT for an example, + +00:05:34.234 --> 00:05:36.163 +in addition to this red line here, + +00:05:36.263 --> 00:05:38.767 +which leads us from an interpreted state + +00:05:38.768 --> 00:05:40.643 +to a compiled native state, + +00:05:40.743 --> 00:05:42.163 +which is also what Emacs does, + +00:05:42.263 --> 00:05:44.333 +LuaJIT also supports going from + +00:05:44.334 --> 00:05:47.523 +a compiled state back to its interpreter. + +00:05:47.623 --> 00:05:51.483 +And this process is called "deoptimization". + +00:05:51.583 --> 00:05:55.300 +In contrast to its name, deoptimization here actually + +00:05:55.301 --> 00:05:58.563 +enables a huge category of JIT optimizations. + +00:05:58.663 --> 00:06:00.163 +They are called speculation. + +00:06:01.463 --> 00:06:04.600 +Basically, with speculation, the compiler + +00:06:04.601 --> 00:06:07.683 +can use runtime statistics to speculate, + +00:06:07.783 --> 00:06:11.443 +to make bolder assumptions in the compiled code. + +00:06:11.543 --> 00:06:13.983 +And when the assumptions are invalidated, + +00:06:14.083 --> 00:06:18.323 +the runtime deoptimizes the code, updates statistics, + +00:06:18.423 --> 00:06:21.133 +and then recompile the code based on new assumptions, + +00:06:21.134 --> 00:06:24.443 +and that will make the code more performant. + +00:06:24.543 --> 00:06:26.763 +Let's look at an example. + +00:06:28.463 --> 00:06:30.967 +So, here is a really simple function, + +00:06:30.968 --> 00:06:33.083 +that adds one to the input number. + +00:06:33.183 --> 00:06:36.167 +But in Emacs, it is not that simple, + +00:06:36.168 --> 00:06:38.203 +because Emacs has three categories of numbers, + +00:06:38.303 --> 00:06:42.700 +that is, fix numbers, or machine-word-sized integers, + +00:06:42.701 --> 00:06:45.603 +floating numbers, and big integers. + +00:06:45.703 --> 00:06:47.600 +And when we compile this, we need + +00:06:47.601 --> 00:06:49.363 +to handle all three cases. + +00:06:49.463 --> 00:06:52.600 +And if we analyze the code produced by Emacs, + +00:06:52.601 --> 00:06:54.683 +as is shown by this gray graph here, + +00:06:54.783 --> 00:06:58.083 +we can see that it has, two paths: + +00:06:58.183 --> 00:07:01.403 +One fast path, that does fast fix number addition; + +00:07:01.503 --> 00:07:03.967 +and one for slow paths, that calls out + +00:07:03.968 --> 00:07:06.523 +to an external plus-one function, + +00:07:06.623 --> 00:07:09.683 +to handle floating number and big integers. + +00:07:09.783 --> 00:07:13.167 +Now, if we pass integers into this function, + +00:07:13.168 --> 00:07:16.283 +it's pretty fast because it's on the fast path. + +00:07:16.383 --> 00:07:19.767 +However, if we pass in a floating number, + +00:07:19.768 --> 00:07:21.843 +then it has to go through the slow path, + +00:07:21.943 --> 00:07:25.563 +doing an extra function call, which is slow. + +00:07:25.663 --> 00:07:28.733 +What speculation might help here is that, + +00:07:28.734 --> 00:07:31.443 +it can have flexible fast paths. + +00:07:31.543 --> 00:07:34.563 +When we pass a floating number into this function, + +00:07:34.663 --> 00:07:37.400 +which currently has only fixnumbers on the fast path, + +00:07:37.401 --> 00:07:40.723 +it also has to go through the slow path. + +00:07:40.823 --> 00:07:44.567 +But the difference is that, a speculative runtime can + +00:07:44.568 --> 00:07:47.763 +deoptimize and recompile the code to adapt to this. + +00:07:47.863 --> 00:07:50.367 +And when it recompiles, it might add + +00:07:50.368 --> 00:07:52.643 +floating number onto the fast path, + +00:07:52.743 --> 00:07:55.003 +and now floating number operations are also fast. + +00:07:55.103 --> 00:07:58.567 +And this kind of speculation is why + +00:07:58.568 --> 00:08:03.603 +speculative runtime can be really fast. + +00:08:03.703 --> 00:08:05.723 +Let's take a look at some benchmarks. + +00:08:05.823 --> 00:08:09.423 +They're obtained with the <i>elisp-benchmarks</i> library on ELPA. + +00:08:09.523 --> 00:08:12.600 +The blue line here is for nativecomp, + +00:08:12.601 --> 00:08:16.043 +and these blue areas mean that nativecomp is slower. + +00:08:16.143 --> 00:08:19.133 +And, likewise, green areas mean that + +00:08:19.134 --> 00:08:20.523 +Juicemacs is slower. + +00:08:20.623 --> 00:08:22.867 +At a glance, the two (or four) + +00:08:22.868 --> 00:08:25.143 +actually seems somehow on par, to me. + +00:08:25.243 --> 00:08:30.383 +But, let's take a closer look at some of them. + +00:08:30.483 --> 00:08:32.667 +So, the first few benchmarks are the classic, + +00:08:32.668 --> 00:08:33.983 +Fibonacci benchmarks. + +00:08:34.083 --> 00:08:36.933 +We know that, the series is formed by + +00:08:36.934 --> 00:08:39.203 +adding the previous two numbers in the series. + +00:08:39.303 --> 00:08:41.700 +And looking at this expression here, + +00:08:41.701 --> 00:08:44.043 +Fibonacci benchmarks are quite intensive + +00:08:44.143 --> 00:08:46.800 +in number additions, subtractions, + +00:08:46.801 --> 00:08:49.103 +and function calls, if you use recursions. + +00:08:49.203 --> 00:08:51.000 +And it is exactly why + +00:08:51.001 --> 00:08:54.323 +Fibonacci series is a good benchmark. + +00:08:54.423 --> 00:08:57.243 +And looking at the results here... wow. + +00:08:57.343 --> 00:08:59.843 +Emacs nativecomp executes instantaneously. + +00:08:59.943 --> 00:09:04.523 +It's a total defeat for Juicemacs, seemingly. + +00:09:04.623 --> 00:09:08.043 +Now, if you're into benchmarks, you know something is wrong here: + +00:09:08.143 --> 00:09:11.683 +we are comparing the different things. + +00:09:11.783 --> 00:09:14.200 +So let's look under the hood + +00:09:14.201 --> 00:09:15.483 +and disassemble the function + +00:09:15.583 --> 00:09:17.567 +with this convenient Emacs command + +00:09:17.568 --> 00:09:19.063 +called <i>disassemble</i>... + +00:09:19.163 --> 00:09:23.043 +And these two lines of code is what we got. + +00:09:23.143 --> 00:09:24.700 +So, we already can see + +00:09:24.701 --> 00:09:26.123 +what's going on here: + +00:09:26.223 --> 00:09:29.963 +GCC sees Fibonacci is a pure function, + +00:09:30.063 --> 00:09:31.867 +because it returns the same value + +00:09:31.868 --> 00:09:33.243 +for the same arguments, + +00:09:33.343 --> 00:09:35.700 +so GCC chooses to do the computation + +00:09:35.701 --> 00:09:36.723 +at compile time + +00:09:36.823 --> 00:09:39.133 +and inserts the final number directly + +00:09:39.134 --> 00:09:40.323 +into the compiled code. + +00:09:41.823 --> 00:09:43.603 +It is actually great! + +00:09:43.703 --> 00:09:45.400 +Because it shows that nativecomp + +00:09:45.401 --> 00:09:47.283 +knows about pure functions, + +00:09:47.383 --> 00:09:48.700 +and can do all kinds of things + +00:09:48.701 --> 00:09:51.203 +like removing or constant-folding them. + +00:09:51.303 --> 00:09:54.403 +And Juicemacs just does not do that. + +00:09:54.503 --> 00:09:57.367 +However, we are also concerned about + +00:09:57.368 --> 00:09:59.003 +the things we mentioned earlier: + +00:09:59.103 --> 00:10:00.900 +the performance of number additions, + +00:10:00.901 --> 00:10:02.983 +or function calls. + +00:10:03.083 --> 00:10:05.633 +So, in order to let the benchmarks + +00:10:05.634 --> 00:10:06.863 +show some extra things, + +00:10:06.963 --> 00:10:08.367 +we need to modify it a bit... + +00:10:08.368 --> 00:10:11.323 +by simply making things non-constant. + +00:10:11.423 --> 00:10:15.203 +With that, Emacs gets much slower now. + +00:10:15.303 --> 00:10:17.133 +And again, let's look what's + +00:10:17.134 --> 00:10:21.083 +happening behind these numbers. + +00:10:21.183 --> 00:10:23.500 +Similarly, with the <i>disassemble</i> command, + +00:10:23.501 --> 00:10:25.643 +we can look into the assembly. + +00:10:25.743 --> 00:10:28.019 +And again, we can already see + +00:10:28.020 --> 00:10:29.303 +what's happening here. + +00:10:29.403 --> 00:10:32.083 +So, Juicemacs, due to its speculation nature, + +00:10:32.183 --> 00:10:35.443 +supports fast paths for all three kind of numbers. + +00:10:35.543 --> 00:10:39.233 +However, currently, Emacs nativecomp + +00:10:39.234 --> 00:10:41.243 +does not have any fast path + +00:10:41.343 --> 00:10:43.433 +for the operations here like additions, + +00:10:43.434 --> 00:10:45.803 +or subtractions, or comparisons, + +00:10:45.903 --> 00:10:48.067 +which is exactly what + +00:10:48.068 --> 00:10:50.963 +Fibonacci benchmarks are measuring. + +00:10:51.063 --> 00:10:53.800 +Emacs, at this time, has to call some generic, + +00:10:53.801 --> 00:10:57.963 +external functions for them, and this is slow. + +00:11:00.063 --> 00:11:03.203 +But is nativecomp really that slow? + +00:11:03.303 --> 00:11:04.967 +So, I also ran the same benchmark + +00:11:04.968 --> 00:11:07.083 +in Common Lisp, with SBCL. + +00:11:07.183 --> 00:11:09.000 +And nativecomp is already fast, + +00:11:09.001 --> 00:11:11.003 +compared to untyped SBCL. + +00:11:11.103 --> 00:11:15.500 +It's because SBCL also emits call instructions + +00:11:15.501 --> 00:11:18.483 +when it comes to no type info. + +00:11:18.583 --> 00:11:21.700 +However, once we declare the types, + +00:11:21.701 --> 00:11:25.283 +SBCL is able to compile a fast path for fix numbers, + +00:11:25.383 --> 00:11:27.467 +which makes its performance on par + +00:11:27.468 --> 00:11:30.683 +with speculative JIT engines (that is, Juicemacs), + +00:11:30.783 --> 00:11:34.763 +because, now both of us are now on fast paths. + +00:11:36.063 --> 00:11:38.400 +Additionally, if we are bold enough + +00:11:38.401 --> 00:11:41.203 +to pass this safety zero flag to SBCL, + +00:11:41.303 --> 00:11:43.700 +it will remove all the slow paths + +00:11:43.701 --> 00:11:44.963 +and type checks, + +00:11:45.063 --> 00:11:46.367 +and its performance is close + +00:11:46.368 --> 00:11:48.643 +to what you get with C. + +00:11:48.743 --> 00:11:51.299 +Well, probably we don't want safety zero + +00:11:51.300 --> 00:11:52.063 +most of the time. + +00:11:52.163 --> 00:11:55.133 +But even then, if nativecomp were to + +00:11:55.134 --> 00:11:57.763 +get fast paths for more constructs, + +00:11:57.863 --> 00:11:59.867 +there certainly is quite + +00:11:59.868 --> 00:12:03.563 +some room for performance improvement. + +00:12:04.063 --> 00:12:06.803 +Let's look at some more benchmarks. + +00:12:06.903 --> 00:12:08.933 +For example, for this inclist, + +00:12:08.934 --> 00:12:10.923 +or increment-list, benchmark, + +00:12:11.023 --> 00:12:14.333 +Juicemacs is really slow here. Partly, + +00:12:14.334 --> 00:12:17.603 +it comes from the cost of Java boxing integers. + +00:12:17.703 --> 00:12:20.300 +On the other hand, for Emacs nativecomp, + +00:12:20.301 --> 00:12:22.043 +for this particular benchmark, + +00:12:22.143 --> 00:12:23.667 +it actually has fast paths + +00:12:23.668 --> 00:12:25.523 +for all of the operations. + +00:12:25.623 --> 00:12:27.723 +And that's why it can be so fast, + +00:12:27.823 --> 00:12:30.667 +and that also proves the nativecomp + +00:12:30.668 --> 00:12:33.843 +has a lot potential for improvement. + +00:12:33.943 --> 00:12:35.833 +There is another benchmark here + +00:12:35.834 --> 00:12:37.963 +that use advices. + +00:12:38.063 --> 00:12:40.500 +So Emacs Lisp supports using + +00:12:40.501 --> 00:12:42.203 +advices to override functions + +00:12:42.303 --> 00:12:44.833 +by wrapping the original function, and an advice + +00:12:44.834 --> 00:12:47.443 +function, two of them, inside a glue function. + +00:12:47.543 --> 00:12:51.467 +And in this benchmark, we advice the Fibonacci function + +00:12:51.468 --> 00:12:54.523 +to cache the first ten entries to speed up computation, + +00:12:54.623 --> 00:13:00.003 +as can be seen in the speed-up in the Juicemacs results. + +00:13:00.103 --> 00:13:02.900 +However, it seems that nativecomp does not yet + +00:13:02.901 --> 00:13:08.523 +compile glue functions, and that makes advices slower. + +00:13:08.623 --> 00:13:12.043 +With these benchmarks, let's discuss this big question: + +00:13:12.143 --> 00:13:16.563 +Should GNU Emacs adopt speculative JIT compilation? + +00:13:16.663 --> 00:13:18.967 +Well, the hidden question is actually, + +00:13:18.968 --> 00:13:21.223 +is it worth it? + +00:13:21.323 --> 00:13:24.163 +And, my personal answer is, maybe not. + +00:13:24.263 --> 00:13:28.133 +The first reason is that, slow paths, like, floating numbers, + +00:13:28.134 --> 00:13:31.043 +are actually not that frequent in Emacs. + +00:13:31.143 --> 00:13:34.100 +And optimizing for fast paths like fix numbers + +00:13:34.101 --> 00:13:37.983 +can already get us very good performance already. + +00:13:38.083 --> 00:13:40.333 +And the second or main reason is that, + +00:13:40.334 --> 00:13:43.163 +speculative JIT is very hard. + +00:13:43.263 --> 00:13:46.843 +LuaJIT, for example, took a genius to build. + +00:13:46.943 --> 00:13:50.967 +Even with the help of GCC, we need to hand-write + +00:13:50.968 --> 00:13:54.283 +all those fast path or slow path or switching logic. + +00:13:54.383 --> 00:13:58.133 +We need to find a way to deoptimize, which requires + +00:13:58.134 --> 00:14:01.803 +mapping machine registers back to interpreter stack. + +00:14:01.903 --> 00:14:04.067 +And also, speculation needs runtime info, + +00:14:04.068 --> 00:14:07.323 +which also costs us extra memory. + +00:14:07.423 --> 00:14:10.763 +Moreover, as is shown by some benchmarks above, + +00:14:10.863 --> 00:14:13.333 +there's some low-hanging fruits in nativecomp that + +00:14:13.334 --> 00:14:17.343 +might get us better performance with relatively lower effort. + +00:14:17.443 --> 00:14:22.163 +Compared to this, a JIT engine is a huge, huge undertaking. + +00:14:22.263 --> 00:14:26.123 +But, for Juicemacs, the JIT engine comes a lot cheaper, + +00:14:26.223 --> 00:14:29.067 +because, we are cheating by building on + +00:14:29.068 --> 00:14:33.443 +an existing compiler framework called Truffle. + +00:14:33.543 --> 00:14:35.883 +Truffle is a meta-compiler framework, + +00:14:35.983 --> 00:14:37.633 +which means that it lets you write + +00:14:37.634 --> 00:14:40.103 +an interpreter, add required annotations, + +00:14:40.203 --> 00:14:42.500 +and it will automatically turn the + +00:14:42.501 --> 00:14:45.643 +interpreter into a JIT runtime. + +00:14:45.743 --> 00:14:49.083 +So for example, here is a typical bytecode interpreter. + +00:14:49.183 --> 00:14:51.233 +After you add the required annotations, + +00:14:51.234 --> 00:14:52.523 +Truffle will know that, + +00:14:52.623 --> 00:14:55.533 +the bytecode here is constant, and it should + +00:14:55.534 --> 00:14:59.123 +unroll this loop here, to inline all those bytecode. + +00:14:59.223 --> 00:15:00.467 +And then, when Truffle + +00:15:00.468 --> 00:15:02.243 +compiles the code, it knows that: + +00:15:02.343 --> 00:15:05.233 +the first loop here does: x plus one, + +00:15:05.234 --> 00:15:07.723 +and the second does: return. + +00:15:07.823 --> 00:15:09.533 +And then it will compile all that into, + +00:15:09.534 --> 00:15:11.363 +return x plus 1, + +00:15:11.463 --> 00:15:14.067 +which is exactly what we would expect + +00:15:14.068 --> 00:15:17.683 +when compiling this pseudo code. + +00:15:17.783 --> 00:15:21.083 +Building on that, we can also easily implement speculation, + +00:15:21.183 --> 00:15:24.867 +by using this <i>transferToInterpreterAndInvalidate</i> function + +00:15:24.868 --> 00:15:26.123 +provided by Truffle. + +00:15:26.223 --> 00:15:28.533 +And Truffle will automatically turn that + +00:15:28.534 --> 00:15:30.683 +into deoptimization. + +00:15:30.783 --> 00:15:32.700 +Now, for example, when this add function + +00:15:32.701 --> 00:15:35.723 +is supplied with, two floating numbers. + +00:15:35.823 --> 00:15:38.243 +It will go through the slow path here, + +00:15:38.343 --> 00:15:40.960 +which might lead to a compiled slow path, + +00:15:40.961 --> 00:15:43.203 +or deoptimization. + +00:15:43.303 --> 00:15:45.733 +And going this deoptimization way, + +00:15:45.734 --> 00:15:48.223 +it can then update the runtime stats. + +00:15:48.323 --> 00:15:50.400 +And now, when the code is compiled again, + +00:15:50.401 --> 00:15:51.603 +Truffle will know, + +00:15:51.703 --> 00:15:54.100 +that these compilation stats, suggests that, + +00:15:54.101 --> 00:15:55.563 +we have floating numbers. + +00:15:55.663 --> 00:15:58.733 +And this floating point addition branch will + +00:15:58.734 --> 00:16:02.603 +then be incorporated into the fast path. + +00:16:02.703 --> 00:16:06.003 +To put it into Java code... + +00:16:06.103 --> 00:16:08.723 +Most operations are just as simple as this. + +00:16:08.823 --> 00:16:11.033 +And it supports fast paths for integers, + +00:16:11.034 --> 00:16:13.963 +floating numbers, and big integers. + +00:16:14.063 --> 00:16:17.133 +And the simplicity of this not only saves us work, + +00:16:17.134 --> 00:16:22.243 +but also enables Juicemacs to explore more things more rapidly. + +00:16:22.343 --> 00:16:26.483 +And actually, I have done some silly explorations. + +00:16:26.583 --> 00:16:30.203 +For example, I tried to constant-fold more things. + +00:16:30.303 --> 00:16:32.767 +Many of us have an Emacs config that stays + +00:16:32.768 --> 00:16:36.683 +largely unchanged, at least during one Emacs session. + +00:16:36.783 --> 00:16:39.667 +And that means many of the global variables + +00:16:39.668 --> 00:16:42.323 +in ELisp are constant. + +00:16:42.423 --> 00:16:44.600 +And with speculation, we can + +00:16:44.601 --> 00:16:46.683 +speculate about the stable ones, + +00:16:46.783 --> 00:16:49.563 +and try to inline them as constants. + +00:16:49.663 --> 00:16:51.733 +And this might improve performance, + +00:16:51.734 --> 00:16:53.083 +or maybe not? + +00:16:53.183 --> 00:16:55.367 +Because, we will need a full editor + +00:16:55.368 --> 00:16:58.123 +to get real world data. + +00:16:58.223 --> 00:17:01.733 +I also tried changing cons lists to be backed + +00:17:01.734 --> 00:17:05.243 +by some arrays, because, maybe arrays are faster, I guess? + +00:17:05.343 --> 00:17:09.033 +But in the end, <i>setcdr</i> requires some kind of indirection, + +00:17:09.034 --> 00:17:12.883 +and that actually makes the performance worse. + +00:17:12.983 --> 00:17:14.733 +And for regular expressions, + +00:17:14.734 --> 00:17:17.923 +I also tried borrowing techniques from PCRE JIT, + +00:17:18.023 --> 00:17:20.667 +which is quite fast in itself, but it is + +00:17:20.668 --> 00:17:24.163 +unfortunately unsupported by Java Truffle runtime. + +00:17:24.263 --> 00:17:27.333 +So, looking at these, well, + +00:17:27.334 --> 00:17:30.243 +explorations can fail, certainly. + +00:17:30.343 --> 00:17:32.800 +But, with Truffle and Java, these, + +00:17:32.801 --> 00:17:34.883 +for now, are not that hard to implement, + +00:17:34.983 --> 00:17:37.667 +and also very often, they teach us something + +00:17:37.668 --> 00:17:42.363 +in return, whether or not they fail. + +00:17:42.463 --> 00:17:45.333 +Finally, let's talk about some explorations + +00:17:45.334 --> 00:17:47.883 +that we might get into in the future. + +00:17:47.983 --> 00:17:49.683 +For the JIT engine, for example, + +00:17:49.783 --> 00:17:52.633 +currently I'm looking into the implementation of + +00:17:52.634 --> 00:17:56.883 +nativecomp to maybe reuse some of its optimizations. + +00:17:56.983 --> 00:18:01.323 +For the GUI, I'm very very slowly working on one. + +00:18:01.423 --> 00:18:03.733 +If it ever completes, I have one thing + +00:18:03.734 --> 00:18:06.603 +I'm really looking forward to implementing. + +00:18:06.703 --> 00:18:08.900 +That is, inlining widgets, or even + +00:18:08.901 --> 00:18:11.763 +other buffers, directly into a buffer. + +00:18:11.863 --> 00:18:13.967 +Well, it's because, people sometimes complain + +00:18:13.968 --> 00:18:16.003 +about Emacs's GUI capabilities, + +00:18:16.103 --> 00:18:19.767 +But I personally think that supporting inlining, + +00:18:19.768 --> 00:18:23.043 +like a whole buffer inside another buffer as a rectangle, + +00:18:23.143 --> 00:18:26.883 +could get us very far in layout abilities. + +00:18:26.983 --> 00:18:28.567 +And this approach should also + +00:18:28.568 --> 00:18:30.843 +be compatible with terminals. + +00:18:30.943 --> 00:18:32.933 +And I really want to see how this idea + +00:18:32.934 --> 00:18:36.003 +plays out with Juicemacs. + +00:18:36.103 --> 00:18:38.963 +And of course, there's Lisp concurrency. + +00:18:39.063 --> 00:18:42.167 +And currently i'm thinking of a JavaScript-like, + +00:18:42.168 --> 00:18:46.283 +transparent, single-thread model, using Java's virtual threads. + +00:18:46.383 --> 00:18:49.967 +But anyway, if you are interested in JIT compilation, + +00:18:49.968 --> 00:18:51.663 +Truffle, or anything above, + +00:18:51.763 --> 00:18:53.867 +or maybe you have your own ideas, + +00:18:53.868 --> 00:18:56.283 +you are very welcome to reach out! + +00:18:56.383 --> 00:19:00.033 +Juicemacs does need to implement many more built-in functions, + +00:19:00.034 --> 00:19:03.063 +and any help would be very appreciated. + +00:19:03.163 --> 00:19:05.800 +And I promise, it can be a very fun playground + +00:19:05.801 --> 00:19:08.343 +to learn about Emacs and do crazy things. + +00:19:08.443 --> 00:19:10.902 +Thank you! diff --git a/2025/captions/emacsconf-2025-llm--emacs-editors-and-llm-driven-workflows--andrew-hyatt--main.vtt b/2025/captions/emacsconf-2025-llm--emacs-editors-and-llm-driven-workflows--andrew-hyatt--main.vtt new file mode 100644 index 00000000..ea969e1e --- /dev/null +++ b/2025/captions/emacsconf-2025-llm--emacs-editors-and-llm-driven-workflows--andrew-hyatt--main.vtt @@ -0,0 +1,1069 @@ +WEBVTT captioned by amitav + +NOTE Introduction + +00:00:01.040 --> 00:00:03.079 +Hi, I'm Andrew Hyatt. + +00:00:03.080 --> 00:00:09.399 +I'm going to talk to you today about Emacs and AI, + +00:00:09.400 --> 00:00:10.879 +and where things are right now + +00:00:10.880 --> 00:00:12.119 +in the world of Emacs and AI, + +00:00:12.120 --> 00:00:14.159 +via large language models, + +00:00:14.160 --> 00:00:16.999 +and where things might be going, + +00:00:17.000 --> 00:00:22.699 +and what it means for the future of Emacs. + +00:00:22.700 --> 00:00:27.279 +I think what we're seeing with Emacs is interesting. + +00:00:27.280 --> 00:00:29.399 +We've seen a lot of different things + +00:00:29.400 --> 00:00:31.559 +come around in the past year, + +00:00:31.560 --> 00:00:33.119 +in the past several years. + +00:00:33.120 --> 00:00:35.079 +There's lots of different solutions. + +00:00:35.080 --> 00:00:36.759 +But in the past year, things have been very interesting. + +00:00:36.760 --> 00:00:39.679 +I think there's new and interesting questions + +00:00:39.680 --> 00:00:43.279 +about what does it mean to use Emacs? + +00:00:43.280 --> 00:00:45.479 +What does it mean to use any editor? + +00:00:45.480 --> 00:00:47.279 +I'm going to be talking about Emacs, + +00:00:47.280 --> 00:00:50.359 +and I'm going to show you various Emacs packages + +00:00:50.360 --> 00:00:53.079 +as demonstrations of these ideas. + +00:00:53.080 --> 00:00:59.839 +But there's the general question of + +00:00:59.840 --> 00:01:03.719 +what does it mean to use any editor, not just Emacs? + +00:01:03.720 --> 00:01:06.239 +What does it mean to do work? + +00:01:06.240 --> 00:01:10.719 +And I think the industry in general is facing these challenges + +00:01:10.720 --> 00:01:13.279 +of we don't really know where things are going to end up, + +00:01:13.280 --> 00:01:16.919 +but we do know the direction they're going. + +00:01:16.920 --> 00:01:20.039 +Emacs is a reflection of that. + +00:01:20.040 --> 00:01:23.239 +I think the answer for Emacs might be + +00:01:23.240 --> 00:01:25.719 +a little bit different than everything else, + +00:01:25.720 --> 00:01:28.599 +but I do want to show you what's out there + +00:01:28.600 --> 00:01:33.319 +so we can explore what are the possibilities + +00:01:33.320 --> 00:01:41.119 +of Emacs, AI, and generally how we get things done. + +00:01:41.120 --> 00:01:44.719 +Thanks. Let's dive right into it. + +NOTE Copilot + +00:01:44.720 --> 00:01:48.079 +We're going to start by showing you + +00:01:48.080 --> 00:01:51.039 +some things that are pretty well integrated, + +00:01:51.040 --> 00:01:55.279 +that look a lot like what you see in Emacs + +00:01:55.280 --> 00:01:58.679 +and fit in with the kinds of editing + +00:01:58.680 --> 00:02:02.639 +that you normally do in Emacs. + +00:02:02.640 --> 00:02:06.579 +So this is just kind of like, it's well integrated. + +00:02:06.580 --> 00:02:08.779 +So we're going to talk about Copilot and Semext. + +00:02:08.780 --> 00:02:12.679 +Copilot is by Microsoft via GitHub, + +00:02:12.680 --> 00:02:14.759 +and Semext is just my personal demo, + +00:02:14.760 --> 00:02:18.039 +but they're both showing you, you know, + +00:02:18.040 --> 00:02:24.399 +this kind of thing. Let's start with Copilot. + +00:02:24.400 --> 00:02:31.919 +Let's try out Copilot on just a standard bit of Elisp. + +00:02:31.920 --> 00:02:38.439 +We're going to write a Fibonacci function. + +00:02:38.440 --> 00:02:43.079 +Let's try out Emacs on a standard bit of Elisp. + +00:02:43.080 --> 00:02:49.279 +We're going to write a Fibonacci function. + +00:02:49.280 --> 00:02:53.159 +And you can see like as soon as we even start typing it, + +00:02:53.160 --> 00:02:56.339 +we get everything as a completion. + +00:02:56.340 --> 00:02:59.879 +So you can just press Tab here, + +00:02:59.880 --> 00:03:02.159 +and you've just completed + +00:03:02.160 --> 00:03:06.799 +a significant bunch of Emacs Lisp code. + +00:03:06.800 --> 00:03:09.919 +It will do this no matter where you are. + +00:03:09.920 --> 00:03:14.799 +So, pretty useful. It will just keep suggesting things. + +00:03:14.800 --> 00:03:16.439 +Do you want to do this? + +00:03:16.440 --> 00:03:17.479 +I'm not sure. + +00:03:17.480 --> 00:03:22.839 +But it usually is offering pretty reasonable things. + +00:03:22.840 --> 00:03:29.299 +So you could do this with code, + +00:03:29.300 --> 00:03:32.119 +of course, any code. + +00:03:32.120 --> 00:03:33.919 +You don't really even have to have a mode for it, right? + +00:03:33.920 --> 00:03:36.679 +That's kind of the beauty of AI is that + +00:03:36.680 --> 00:03:38.519 +you don't need any Emacs functionality for this, + +00:03:38.520 --> 00:03:39.519 +except for Copilot. + +00:03:39.520 --> 00:03:41.679 +It doesn't need to know the structure of your code. + +00:03:41.680 --> 00:03:45.279 +It doesn't need anything except for the text itself + +00:03:45.280 --> 00:03:51.239 +and whatever AI integration that this is. + +00:03:51.240 --> 00:03:53.739 +We can look at, you can do the same thing with Org-mode. + +00:03:53.740 --> 00:03:57.999 +So we could say create, no, + +00:03:58.000 --> 00:04:02.919 +how about let's, let's do, you know, spring cleaning. + +00:04:02.920 --> 00:04:10.839 +It's actually the fall, but still we'll say spring cleaning. + +00:04:10.840 --> 00:04:12.767 +And it'll start suggesting things that, you know, + +00:04:12.768 --> 00:04:15.439 +maybe at first, it doesn't really know what to do to + +00:04:15.440 --> 00:04:16.433 +clean up all code. + +00:04:16.434 --> 00:04:18.400 +It thinks I need to clean up code, but no, + +00:04:18.401 --> 00:04:21.839 +this is going to be actual, you know, + +00:04:21.840 --> 00:04:31.567 +clean hood over range. Clean out pantry. + +00:04:31.568 --> 00:04:33.879 +These are all really reasonable suggestions. + +00:04:33.880 --> 00:04:38.319 +You just keep going here. + +NOTE Semext + +00:04:38.320 --> 00:04:40.559 +I'm going to demonstrate Semext, + +00:04:40.560 --> 00:04:43.879 +which is a package I have on GNU Elpa, + +00:04:43.880 --> 00:04:48.719 +that is designed to integrate AI in a very Emacs-like way. + +00:04:48.720 --> 00:04:50.999 +And so what you could do is you could do a + +00:04:51.000 --> 00:04:54.799 +semext-search-forward. + +00:04:54.800 --> 00:04:58.719 +The UI looks just like other Emacs commands, + +00:04:58.720 --> 00:05:02.379 +but you can search for anything. + +00:05:02.380 --> 00:05:06.279 +There's really no way to express what I'm about to, + +00:05:06.280 --> 00:05:08.679 +what I'm trying to demonstrate + +00:05:08.680 --> 00:05:12.359 +in Emacs's normal search commands. + +00:05:12.360 --> 00:05:15.399 +You could really ask for anything. + +00:05:15.400 --> 00:05:18.759 +And it takes a little while, which is not Emacs-like, + +00:05:18.760 --> 00:05:20.033 +but everything else is sort of like + +00:05:20.034 --> 00:05:21.719 +it's designed to be like Emacs, + +00:05:21.720 --> 00:05:23.519 +except way more powerful. + +00:05:23.520 --> 00:05:27.119 +You don't need any mode to be active for this. + +00:05:27.120 --> 00:05:32.039 +You just need the library + +00:05:32.040 --> 00:05:34.759 +and an AI provider of some sort, either locally + +00:05:34.760 --> 00:05:41.199 +or, you know, your favorite cloud provider. + +NOTE Integrated AI experiences: gptel, ellama, chatgpt-shell, etc. + +00:05:41.200 --> 00:05:43.679 +Now we're going to move on to a different way + +00:05:43.680 --> 00:05:46.399 +of interacting with AI and Emacs. + +00:05:46.400 --> 00:05:52.319 +This way is less like the normal editing experience. + +00:05:52.320 --> 00:05:56.999 +So you lose some familiarity. However, in exchange, + +00:05:57.000 --> 00:05:58.079 +it is a lot more powerful. + +00:05:58.080 --> 00:06:00.119 +And there's a whole suite of these tools. + +00:06:00.120 --> 00:06:02.479 +I'm going to demonstrate gptel, + +00:06:02.480 --> 00:06:05.779 +which is the most popular one. + +00:06:05.780 --> 00:06:06.399 +But there are many. + +00:06:06.400 --> 00:06:08.479 +And I think different people have + +00:06:08.480 --> 00:06:11.759 +their own preferences of what they like to use. + +00:06:11.760 --> 00:06:12.999 +We're going to try now something + +00:06:13.000 --> 00:06:15.079 +that is a step away from just editing. + +00:06:15.080 --> 00:06:19.839 +And we're going to, I'm actually using gptel. + +00:06:19.840 --> 00:06:22.799 +There are several packages that are going to be + +00:06:22.800 --> 00:06:25.959 +doing the same sort of thing as I'm going to show you. + +00:06:25.960 --> 00:06:29.999 +gptel has sort of become the most popular one. + +00:06:30.000 --> 00:06:32.199 +So that's why I'm showing that to you. + +00:06:32.200 --> 00:06:39.319 +But let's just highlight everything and say gptel rewrite. + +00:06:39.320 --> 00:06:42.399 +And gptel basically just has a few things. + +00:06:42.400 --> 00:06:45.119 +There's different ways of thinking about this. + +00:06:45.120 --> 00:06:49.999 +With just a few very configurable menus, + +00:06:50.000 --> 00:06:53.959 +you can do a large variety of things. + +00:06:53.960 --> 00:06:59.819 +So let's give rewrite instructions. + +00:06:59.820 --> 00:07:06.600 +"Turn this into an iterative program + +00:07:06.601 --> 00:07:12.199 +instead of a recursive program." + +00:07:12.200 --> 00:07:17.799 +In Elisp, you really should not be using recursion. + +00:07:17.800 --> 00:07:20.359 +So we could say "return to be ready". + +00:07:20.360 --> 00:07:21.119 +Do we accept it? + +00:07:21.120 --> 00:07:24.519 +Yes, we accept it. Or we could iterate and say, no, no, + +00:07:24.520 --> 00:07:26.799 +that's not what we meant. We meant something else. + +00:07:26.800 --> 00:07:29.159 +Or you did something a little something wrong. + +00:07:29.160 --> 00:07:29.879 +Please fix it. + +00:07:29.880 --> 00:07:31.879 +So this is all very powerful. + +00:07:31.880 --> 00:07:33.799 +Is this editing? + +00:07:33.800 --> 00:07:40.279 +Well, it's in the editor. + +00:07:40.280 --> 00:07:42.759 +You could do this while editing, while deleting, + +00:07:42.760 --> 00:07:44.959 +you could be doing some sort of traditional editing. + +00:07:44.960 --> 00:07:47.679 +And then this, which is editing + +00:07:47.680 --> 00:07:48.919 +in the sense that it's in your editor, + +00:07:48.920 --> 00:07:51.039 +you might have to highlight + +00:07:51.040 --> 00:07:52.799 +some parts of the file and do things, + +00:07:52.800 --> 00:07:54.719 +but generally you don't even need to, + +00:07:54.720 --> 00:07:59.879 +or you go to a spot and you say, put code at this spot. + +00:07:59.880 --> 00:08:01.959 +It's kind of like editing. + +00:08:01.960 --> 00:08:05.839 +I would say it's not exactly editing, + +00:08:05.840 --> 00:08:10.159 +but it's at least something that must happen in an editor + +00:08:10.160 --> 00:08:12.359 +and it's well integrated into Emacs. + +00:08:12.360 --> 00:08:14.759 +As you can tell, it used very sort of + +00:08:14.760 --> 00:08:18.239 +modern standard Emacs UI paradigms + +00:08:18.240 --> 00:08:20.759 +and it's all written in Elisp. + +00:08:20.760 --> 00:08:23.779 +Everything is happening in Elisp here. + +00:08:23.780 --> 00:08:25.959 +So this is just very much an Emacs experience. + +00:08:25.960 --> 00:08:27.679 +It's just not exactly editing + +00:08:27.680 --> 00:08:29.879 +because the thing doing the editing + +00:08:29.880 --> 00:08:32.519 +is the AI and not you. + +00:08:32.520 --> 00:08:36.039 +You're just kind of telling it what to do. + +NOTE Outside the editor + +00:08:36.040 --> 00:08:41.119 +Now we're going to go and look at a way of interaction + +00:08:41.120 --> 00:08:43.239 +that's even more powerful + +00:08:43.240 --> 00:08:46.279 +and even more disconnected from the normal editing experience. + +00:08:46.280 --> 00:08:47.919 +In fact, it's so disconnected + +00:08:47.920 --> 00:08:52.399 +that most people are using this without an editor. + +00:08:52.400 --> 00:08:57.879 +These are things like Claude Code + +00:08:57.880 --> 00:09:01.079 +or the sort of open source equivalent, Aider. + +00:09:01.080 --> 00:09:05.039 +There's a few other things that follow this pattern as well. + +00:09:05.040 --> 00:09:07.479 +But it's very interesting in the sense + +00:09:07.480 --> 00:09:09.839 +that while you can integrate these with the editors, + +00:09:09.840 --> 00:09:12.039 +and I'm going to show you an Emacs integration, + +00:09:12.040 --> 00:09:13.519 +you don't need to. + +00:09:13.520 --> 00:09:16.939 +And that's not the way most people are using them. + +00:09:16.940 --> 00:09:19.759 +And I find it very interesting that sort of + +00:09:19.760 --> 00:09:23.719 +we're going back kind of full circle where, you know, + +00:09:23.720 --> 00:09:31.959 +in the 1960s or 70s, we were using Ed from the terminal + +00:09:31.960 --> 00:09:35.639 +to edit files, but then we created editors, + +00:09:35.640 --> 00:09:37.959 +and that was a really good idea. + +00:09:37.960 --> 00:09:40.167 +It is a lot easier to edit files + +00:09:40.168 --> 00:09:42.499 +when you have an actual UI. + +00:09:42.500 --> 00:09:46.879 +But now it's 2025, and we're back in the terminal, + +00:09:46.880 --> 00:09:50.799 +and we're editing files through the terminal, + +00:09:50.800 --> 00:09:53.599 +and you know what, it's great, + +00:09:53.600 --> 00:09:56.899 +but I think it's even better with Emacs. + +00:09:56.900 --> 00:10:00.279 +On the other hand, it comes with some trade-offs, + +00:10:00.280 --> 00:10:04.733 +as you can see, as we will see. + +NOTE Outside Experiences: claude-code.el, aidermacs, eca + +00:10:04.734 --> 00:10:07.467 +Okay, we're going to look at + +00:10:07.468 --> 00:10:20.320 +[audio glitch] Claude Code IDE, aidermacs, ECA. + +00:10:20.321 --> 00:10:22.639 +Last time, I didn't show you all the variants. + +00:10:22.640 --> 00:10:26.839 +I do want to show you eca, which points to, + +00:10:26.840 --> 00:10:29.799 +it is a very similar tool in what it does, + +00:10:29.800 --> 00:10:32.739 +but does have a different + +00:10:32.740 --> 00:10:37.239 +and I think better type of Emacs integration. + +00:10:37.240 --> 00:10:42.599 +All right, we're going to demonstrate Claude Code IDE, + +00:10:42.600 --> 00:10:46.839 +which is one of three Claude Code packages. + +00:10:46.840 --> 00:10:47.719 +It's a bit confusing. + +00:10:47.720 --> 00:10:52.039 +One of them will be demoed by another presenter + +00:10:52.040 --> 00:10:54.639 +at the Emacs conference, so stay tuned for that. + +00:10:54.640 --> 00:10:56.439 +Here I'm just going to give you a little taste + +00:10:56.440 --> 00:10:58.759 +of what these packages look like. + +00:10:58.760 --> 00:11:03.339 +So if we say Claude Code IDE, + +00:11:03.340 --> 00:11:06.839 +it presents us with basically + +00:11:06.840 --> 00:11:09.039 +almost exactly what you would get + +00:11:09.040 --> 00:11:11.519 +when you're running this in the terminal. + +00:11:11.520 --> 00:11:13.933 +And essentially there's a terminal interface. + +00:11:13.934 --> 00:11:16.659 +You can see that there's a vterm. + +00:11:16.660 --> 00:11:20.699 +But here we're going to say, "In scratch.el"... + +00:11:20.700 --> 00:11:23.400 +let's say what we want to happen. + +00:11:23.401 --> 00:11:32.133 +[In scratch.el, there is a fibonacci function. + +00:11:32.134 --> 00:11:39.567 +Can you add all normal elisp headers + +00:11:39.568 --> 00:11:43.859 +and footers to this file?] + +00:11:43.860 --> 00:11:45.840 +So, we just say what's going to happen, + +00:11:45.841 --> 00:11:48.399 +and this is going to do things in the background. + +00:11:48.400 --> 00:11:50.979 +It's not going to do things through Emacs. + +00:11:50.980 --> 00:11:54.079 +That said, there is an integration with Emacs, + +00:11:54.080 --> 00:12:00.659 +so that it can do things like show you these nice ediffs. + +00:12:00.660 --> 00:12:03.199 +My screen is not really wide enough + +00:12:03.200 --> 00:12:04.699 +to show you a really great ediff here, + +00:12:04.700 --> 00:12:06.239 +but you can kind of see what it's doing, + +00:12:06.240 --> 00:12:09.079 +and you can see, yeah, that looks good, + +00:12:09.080 --> 00:12:14.120 +so you could say yes, yes, accept the changes, + +00:12:14.121 --> 00:12:25.299 +and if we... Just need to revert the buffer. + +00:12:25.300 --> 00:12:28.459 +We can quit the printout of this. + +00:12:28.460 --> 00:12:33.019 +We see that it just did everything I asked it to. + +00:12:33.020 --> 00:12:36.139 +Is everything exactly right? + +00:12:36.140 --> 00:12:39.159 +Probably not. It's reasonable for a start though. + +00:12:39.160 --> 00:12:40.959 +But you could ask it to do anything. + +00:12:40.960 --> 00:12:45.339 +You could say, write unit tests for this, and it will. + +00:12:45.340 --> 00:12:49.019 +You could say, write me a suite of functions + +00:12:49.020 --> 00:12:52.579 +like Fibonacci, and it'll probably do something reasonable. + +00:12:52.580 --> 00:12:54.900 +But you can see this is not editing. + +00:12:54.901 --> 00:12:58.659 +There's nothing editing-like about this. + +00:12:58.660 --> 00:13:07.159 +That said, there is something that is editing. + +00:13:07.160 --> 00:13:08.599 +You need to give it instructions. + +00:13:08.600 --> 00:13:10.959 +You need to tell it what to do. + +NOTE Org files + +00:13:10.960 --> 00:13:19.619 +And what you could do is... You could have a project.org, + +00:13:19.620 --> 00:13:23.899 +and what you could do is you could have functions. + +00:13:23.900 --> 00:13:26.659 +The way I've done things often is .... + +00:13:26.660 --> 00:13:28.439 +You could say something like, + +00:13:28.440 --> 00:13:36.199 +unit tests for Fibonacci. How do you spell Fibonacci? + +00:13:36.200 --> 00:13:40.479 +I don't remember. But then you could say that this is, + +00:13:40.480 --> 00:13:47.159 +you could clock it, basically. org-clock. + +00:13:47.160 --> 00:13:48.879 +What I've done is... + +00:13:48.880 --> 00:13:50.399 +You could add custom commands to Claude Code, + +00:13:50.400 --> 00:13:53.119 +and you could just say, look, here's my Org file, + +00:13:53.120 --> 00:13:57.879 +read it and do the thing that I'm clocked in as. + +00:13:57.880 --> 00:14:01.159 +And then you can write a bunch of instructions here, like, + +00:14:01.160 --> 00:14:07.039 +I like to use ert for tests. Tests should, like, whatever. + +00:14:07.040 --> 00:14:08.639 +You should just say... everything + +00:14:08.640 --> 00:14:10.999 +you need to kind of specify. + +00:14:11.000 --> 00:14:13.199 +As you get to more complicated tasks, + +00:14:13.200 --> 00:14:16.679 +it's harder and harder to give it all the context + +00:14:16.680 --> 00:14:17.799 +it needs for a task, + +00:14:17.800 --> 00:14:22.299 +and Org Mode is actually a pretty good way to do this. + +00:14:22.300 --> 00:14:24.079 +I find that this works pretty well, + +00:14:24.080 --> 00:14:26.699 +and you can even have it instruct Claude + +00:14:26.700 --> 00:14:29.333 +to just mark things done in your Org file + +00:14:29.334 --> 00:14:30.679 +when they're done. + +00:14:30.680 --> 00:14:32.867 +And it knows how to do this, of course. + +00:14:32.868 --> 00:14:37.959 +So, let's just clock out. + +00:14:37.960 --> 00:14:45.239 +That's one way to do things. + +NOTE ECA + +00:14:45.240 --> 00:14:49.499 +So one other thing I'd like to show you is eca, + +00:14:49.500 --> 00:14:52.879 +which, compared to Claude Code, ECA is open source. + +00:14:52.880 --> 00:14:54.239 +It's very nice in that respect. + +00:14:54.240 --> 00:14:57.839 +It doesn't have to use Anthropic's models. + +00:14:57.840 --> 00:15:00.279 +You can use local models, + +00:15:00.280 --> 00:15:07.619 +but it has the advantage of integrating very well with Emacs. + +00:15:07.620 --> 00:15:08.559 +I'm not going to demonstrate it, + +00:15:08.560 --> 00:15:11.159 +because it works essentially the same thing you could do + +00:15:11.160 --> 00:15:14.119 +approximately the same kinds of things + +00:15:14.120 --> 00:15:15.479 +you could do with Claude Code. + +00:15:15.480 --> 00:15:17.439 +You just write what you want to happen + +00:15:17.440 --> 00:15:18.639 +and it will make it happen. + +00:15:18.640 --> 00:15:21.879 +It again does not do this through Emacs, + +00:15:21.880 --> 00:15:23.039 +but what it does do is + +00:15:23.040 --> 00:15:25.119 +it gives you a much better Emacs interface + +00:15:25.120 --> 00:15:26.919 +that's not terminal-based, + +00:15:26.920 --> 00:15:29.639 +because you're not using it through the terminal, + +00:15:29.640 --> 00:15:31.239 +or not even through comint, + +00:15:31.240 --> 00:15:35.599 +you are using it through a backend + +00:15:35.600 --> 00:15:37.499 +that is exchanging structured information + +00:15:37.500 --> 00:15:40.999 +with this process that is doing all the work. + +00:15:41.000 --> 00:15:41.900 +But other than that, + +00:15:41.901 --> 00:15:44.519 +it's the same model as Claude Code + +00:15:44.520 --> 00:15:52.059 +and projects of that nature. + +NOTE Editing + +00:15:52.060 --> 00:15:56.159 +We've seen in the demos that I gave + +00:15:56.160 --> 00:15:58.639 +that there are AI experiences + +00:15:58.640 --> 00:16:01.279 +that are very natural in the world of editing. + +00:16:01.280 --> 00:16:05.339 +because they, like Copilot, just offers completion, + +00:16:05.340 --> 00:16:09.479 +it fits very well with what we all do in Emacs. + +00:16:09.480 --> 00:16:14.279 +And it's truly, yes, it's kind of a cheat in a sense + +00:16:14.280 --> 00:16:15.639 +for editing experiences, + +00:16:15.640 --> 00:16:20.159 +because it can do so much, but it's just editing. + +00:16:20.160 --> 00:16:25.259 +Whereas things like gptel and those kinds of tools, + +00:16:25.260 --> 00:16:29.799 +they are clearly in an editor and using editor, + +00:16:29.800 --> 00:16:35.319 +they're using Emacs, but they represent sort of like, well, + +00:16:35.320 --> 00:16:37.759 +you can edit for a while, then you could use these tools + +00:16:37.760 --> 00:16:39.479 +to do something that is not editing, + +00:16:39.480 --> 00:16:45.899 +this AI just changing the buffer for you. And that's fine. + +00:16:45.900 --> 00:16:48.399 +It's still... It may not be editing, + +00:16:48.400 --> 00:16:52.033 +but it's still clearly something that + +00:16:52.034 --> 00:16:55.567 +is useful to do in Emacs + +00:16:55.568 --> 00:16:57.039 +and belongs in Emacs. + +00:16:57.040 --> 00:17:01.859 +But the new tools like Claude Code and things like that + +00:17:01.860 --> 00:17:02.639 +are kind of different. + +00:17:02.640 --> 00:17:06.639 +Yes, they will get better integrated with Emacs, + +00:17:06.640 --> 00:17:11.639 +but it's not clear that they really need to. + +00:17:11.640 --> 00:17:15.479 +They can do a lot of things without editing. + +00:17:15.480 --> 00:17:19.239 +In a sense, editing is obsolete in some sense. + +00:17:19.240 --> 00:17:23.459 +For as many tasks, you don't need to edit anymore. + +00:17:23.460 --> 00:17:26.439 +And that's a nice thing. + +00:17:26.440 --> 00:17:30.579 +No one really knows when all this will end, + +00:17:30.580 --> 00:17:36.879 +how far things will go. It could be that in a decade or so, + +00:17:36.880 --> 00:17:41.039 +no one's really editing for work anymore. + +00:17:41.040 --> 00:17:43.159 +Maybe you're just writing instructions. + +00:17:43.160 --> 00:17:44.319 +You could do that with anything. + +00:17:44.320 --> 00:17:47.439 +You don't need Emacs or any special editor. + +00:17:47.440 --> 00:17:50.439 +We could all be using Notepad. That would be bad. + +00:17:50.440 --> 00:17:58.039 +But... I think it could go that far, + +00:17:58.040 --> 00:18:01.839 +but it could be that, well, for many specialized things, + +00:18:01.840 --> 00:18:04.359 +people are still using editing for certain tasks, + +00:18:04.360 --> 00:18:07.000 +but most tasks are getting fed to just... + +00:18:07.001 --> 00:18:08.839 +AI is just doing those things. + +00:18:08.840 --> 00:18:15.759 +In any case, I think it's clear that editing is diminishing, + +00:18:15.760 --> 00:18:17.959 +the need for editing itself is diminishing. + +00:18:17.960 --> 00:18:21.879 +And in such a world, It's interesting to think + +00:18:21.880 --> 00:18:24.799 +where Emacs is headed, especially in relation to + +00:18:24.800 --> 00:18:26.359 +all the other editors. + +00:18:26.360 --> 00:18:28.599 +I think people will use Emacs less. + +00:18:28.600 --> 00:18:31.639 +But I think other editors, like VS Code, + +00:18:31.640 --> 00:18:37.999 +may simply disappear or be a relatively fringe tool. + +00:18:38.000 --> 00:18:42.719 +And Emacs is going to follow its own path. + +00:18:42.720 --> 00:18:44.679 +It's very extensible. It could do anything. + +00:18:44.680 --> 00:18:47.919 +If there's one thing Emacs can do, it's adapt. + +00:18:47.920 --> 00:18:51.679 +Emacs has been around for a long time. + +00:18:51.680 --> 00:18:54.799 +It's pretty clear that Emacs will be around for a long time. + +00:18:54.800 --> 00:18:58.879 +It might be that in the future, + +00:18:58.880 --> 00:19:04.339 +editing is some sort of like an artisanal activity that we do. + +00:19:04.340 --> 00:19:05.599 +It's kind of weird to think about it. + +00:19:05.600 --> 00:19:07.679 +It's not like baking bread. + +00:19:07.680 --> 00:19:10.079 +But it is the sense that AI might be + +00:19:10.080 --> 00:19:12.399 +churning out code in the way, you know, + +00:19:12.400 --> 00:19:14.199 +the factories are turning out bread, + +00:19:14.200 --> 00:19:17.139 +but if you really want the good stuff, + +00:19:17.140 --> 00:19:20.999 +you'll have to do it yourself. + +00:19:21.000 --> 00:19:23.959 +I don't know if it'll be exactly like that, + +00:19:23.960 --> 00:19:29.519 +but it could be that Emacs survives and thrives + +00:19:29.520 --> 00:19:33.559 +in a very kind of specialized ecosystem of people + +00:19:33.560 --> 00:19:35.599 +who contribute and use it in the way + +00:19:35.600 --> 00:19:39.539 +it has survived and thrive right now. + +00:19:39.540 --> 00:19:46.139 +And I think that's a really nice way for all this to end up. + +00:19:46.140 --> 00:19:48.719 +There's the whole sense of how society will end up + +00:19:48.720 --> 00:19:50.759 +if all this happens. I don't know, + +00:19:50.760 --> 00:19:54.639 +but Emacs will be there for us when whatever happens. + +00:19:54.640 --> 00:20:00.079 +So thank you, and let's help make Emacs the best it can be + +00:20:00.080 --> 00:20:04.880 +to survive and thrive in the next decade. diff --git a/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main--chapters.vtt b/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main--chapters.vtt new file mode 100644 index 00000000..fb1e557c --- /dev/null +++ b/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main--chapters.vtt @@ -0,0 +1,26 @@ +WEBVTT + + +00:00:01.120 --> 00:00:49.041 +What are reactive notebooks? + +00:00:49.042 --> 00:02:38.498 +Reactivity demo + +00:02:38.499 --> 00:03:21.079 +Org-Babel + +00:03:21.080 --> 00:03:51.900 +Running the whole buffer + +00:03:51.901 --> 00:04:21.660 +Caching + +00:04:21.760 --> 00:06:04.533 +Computation dependencies + +00:06:04.534 --> 00:07:29.965 +Making this even better + +00:07:29.966 --> 00:08:08.240 +Wrapping up diff --git a/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main.vtt b/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main.vtt new file mode 100644 index 00000000..22373ce6 --- /dev/null +++ b/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main.vtt @@ -0,0 +1,614 @@ +WEBVTT captioned by abhinav + +NOTE What are reactive notebooks? + +00:00:01.120 --> 00:00:03.033 +Hello, everyone. My name is Abhinav, + +00:00:03.034 --> 00:00:03.900 +and I'm going to talk about + +00:00:03.901 --> 00:00:07.140 +how to make Org Babel reactive. So reactivity here + +00:00:07.240 --> 00:00:10.000 +means reactivity in the sense of reactive notebooks. + +00:00:10.001 --> 00:00:11.600 +So if you used Org Babel, + +00:00:11.601 --> 00:00:13.933 +you might also have used Jupyter notebooks, + +00:00:13.934 --> 00:00:16.100 +which are basically notebooks primarily for + +00:00:16.200 --> 00:00:16.933 +Python programming, + +00:00:16.934 --> 00:00:20.100 +where you have these text and code blocks interleaved, + +00:00:20.101 --> 00:00:23.157 +and then you can execute every code block independently, + +00:00:23.158 --> 00:00:25.858 +and then you control the order of execution manually, + +00:00:25.859 --> 00:00:27.199 +or you can just run the code blocks + +00:00:27.200 --> 00:00:29.699 +from top to bottom. But with reactive notebooks, + +00:00:29.700 --> 00:00:32.927 +what happens is that there's another way of running + +00:00:32.928 --> 00:00:35.329 +which is basically by having all these + +00:00:35.330 --> 00:00:36.900 +dependent code blocks automatically get + +00:00:37.000 --> 00:00:38.900 +executed whenever you make a change. + +00:00:38.901 --> 00:00:40.774 +So for example, if you change a variable, + +00:00:40.775 --> 00:00:42.060 +everything else that's dependent on + +00:00:42.160 --> 00:00:44.433 +that variable will be executed automatically. + +00:00:44.434 --> 00:00:49.041 +I'll show you an example of what that looks like. + +NOTE Reactivity demo + +00:00:49.042 --> 00:00:51.762 +Right, here's an example reactive Notebook. + +00:00:51.763 --> 00:00:53.460 +So this is called Observable. + +00:00:53.560 --> 00:00:54.863 +Observable is this tool made by + +00:00:54.864 --> 00:00:57.679 +the creator of d3.js which is + +00:00:57.680 --> 00:01:01.499 +a famous JavaScript charting library. So here, the + +00:01:01.500 --> 00:01:03.667 +interface is very similar to Jupyter Notebook. + +00:01:03.668 --> 00:01:06.407 +You basically are having these cells + +00:01:06.408 --> 00:01:08.508 +and each cell could be a text cell, like here, + +00:01:08.509 --> 00:01:09.588 +this is a Markdown cell + +00:01:09.589 --> 00:01:11.609 +and then there are these code blocks. + +00:01:11.610 --> 00:01:15.250 +Now each code cell is basically defining a variable. + +00:01:15.251 --> 00:01:17.740 +This is important in reactive notebooks because + +00:01:17.840 --> 00:01:21.140 +each cell is connected to other cell via this variable + +00:01:21.240 --> 00:01:23.552 +usage. So here data is defined, + +00:01:23.553 --> 00:01:25.012 +then there is filtered which is defined + +00:01:25.013 --> 00:01:27.620 +which is dependent on data, and then this plot is + +00:01:27.720 --> 00:01:29.133 +dependent on filtered. + +00:01:29.134 --> 00:01:31.153 +So now, in a classical notebook, what I will do is + +00:01:31.154 --> 00:01:34.394 +if I change something here, let's say from 1 to 2, + +00:01:34.395 --> 00:01:34.854 +I will have to run this, and then run this plot block again + +00:01:34.855 --> 00:01:40.335 +to make the change be visible. + +00:01:40.336 --> 00:01:42.055 +But in a reactive notebook, what happens is + +00:01:42.056 --> 00:01:44.396 +I can just change this from some value + +00:01:44.397 --> 00:01:46.256 +to some value, and then execute, + +00:01:46.257 --> 00:01:48.817 +and then every descendant is also executed, + +00:01:48.818 --> 00:01:50.940 +because that's how the reactivity works. + +00:01:51.040 --> 00:01:51.937 +You change this variable, + +00:01:51.938 --> 00:01:53.080 +so this should also be changed, + +00:01:53.081 --> 00:01:55.238 +because this is dependent on this variable. + +00:01:55.239 --> 00:01:56.858 +Now this is really helpful + +00:01:56.859 --> 00:01:58.900 +if you have a very complex and messy notebook + +00:01:59.000 --> 00:02:01.199 +which is what actually happens in reality. + +00:02:01.200 --> 00:02:03.480 +You end up doing an exploratory analysis, + +00:02:03.481 --> 00:02:05.959 +and you have these code blocks lying here and there. + +00:02:05.960 --> 00:02:07.101 +Then you change something + +00:02:07.102 --> 00:02:09.281 +and then you have to keep something in your mind + +00:02:09.282 --> 00:02:11.362 +that if I change this, I need to run + +00:02:11.363 --> 00:02:13.023 +these five code blocks again + +00:02:13.024 --> 00:02:15.604 +to finally get to the result that I want to see. + +00:02:15.605 --> 00:02:20.467 +Stale state causes a lot of issues in Jupyter Notebooks. + +00:02:20.468 --> 00:02:23.788 +So this is really good for reactivity, sorry reproducibility, + +00:02:23.789 --> 00:02:26.630 +but this is also really good for + +00:02:26.631 --> 00:02:28.599 +just having this exploration + +00:02:28.600 --> 00:02:30.117 +that you're trying to do. For example, + +00:02:30.118 --> 00:02:31.761 +you're changing something and it's really easy + +00:02:31.762 --> 00:02:34.887 +to just see that change happening in real time + +00:02:34.888 --> 00:02:38.498 +in your outcome variables, right? + +NOTE Org-Babel + +00:02:38.499 --> 00:02:41.920 +So I was wondering how to introduce this reactivity in Org Mode. + +00:02:41.921 --> 00:02:43.200 +And here's how it will look like. + +00:02:43.201 --> 00:02:46.302 +So this is a demo Org Mode file. + +00:02:46.303 --> 00:02:48.603 +There are many Org Babel blocks here. + +00:02:48.604 --> 00:02:49.563 +So you start from here. + +00:02:49.564 --> 00:02:52.085 +Let's say this is a code block. It has a name. + +00:02:52.086 --> 00:02:53.665 +And then there's another code block, + +00:02:53.666 --> 00:02:55.426 +which is dependent on the previous one, + +00:02:55.427 --> 00:02:57.807 +as you can see here, and so on. + +00:02:57.808 --> 00:02:59.368 +And then finally, there's a plot here, + +00:02:59.369 --> 00:03:00.889 +which is a gnuplot code. + +00:03:00.890 --> 00:03:02.550 +And you can see the image here. + +00:03:02.551 --> 00:03:04.131 +Now, what happens usually is that + +00:03:04.132 --> 00:03:05.196 +if I change this value from, + +00:03:05.197 --> 00:03:09.199 +let's say, 113 to 112, nothing happens on its own right? + +00:03:09.200 --> 00:03:12.199 +There's an extra step of execution that I will have to do + +00:03:12.200 --> 00:03:15.079 +so I will do that, and then the value is changed. + +00:03:15.080 --> 00:03:17.699 +Now the problem is that only this value is changed and + +00:03:17.700 --> 00:03:21.079 +if I go down and see the image, nothing will have changed. + +NOTE Running the whole buffer + +00:03:21.080 --> 00:03:23.079 +So what I can do is basically, + +00:03:23.080 --> 00:03:24.818 +a really simple thing is that, + +00:03:24.819 --> 00:03:26.500 +a simple trick is to basically + +00:03:26.600 --> 00:03:29.445 +enable a hook, like, add a hook + +00:03:29.446 --> 00:03:30.525 +whenever you're saving the buffer, + +00:03:30.526 --> 00:03:31.866 +you just run the full buffer again, + +00:03:31.867 --> 00:03:34.287 +like run all the code blocks automatically. + +00:03:34.288 --> 00:03:36.849 +Now if you do that, you can basically make a change somewhere + +00:03:36.850 --> 00:03:37.889 +and then you can, you know, + +00:03:37.890 --> 00:03:41.071 +see how everything else is changing + +00:03:41.072 --> 00:03:42.712 +which gives you some sort of reactivity, + +00:03:42.713 --> 00:03:43.972 +but there's still a lot of computation + +00:03:43.973 --> 00:03:45.973 +that's being wasted. + +00:03:45.974 --> 00:03:49.595 +You might not want to change or run this code block again + +00:03:49.596 --> 00:03:51.900 +when something down there is changing. + +NOTE Caching + +00:03:51.901 --> 00:03:54.567 +So to counter that, you can actually add caching. + +00:03:54.568 --> 00:03:57.133 +So if you add caching to any code block, + +00:03:57.134 --> 00:03:59.800 +that code block will only be executed again + +00:03:59.801 --> 00:04:02.300 +if that code has changed or + +00:04:02.400 --> 00:04:04.755 +the input variables have changed. + +00:04:04.756 --> 00:04:06.336 +But the other problem is that + +00:04:06.337 --> 00:04:08.659 +you don't want caching to be enabled for a lot of cases + +00:04:08.660 --> 00:04:10.840 +where the code block is actually dependent on + +00:04:10.841 --> 00:04:12.722 +external state, like for example, + +00:04:12.723 --> 00:04:15.024 +some sort of randomness or time. + +00:04:15.025 --> 00:04:17.433 +So caching also is, you know, kind of, + +00:04:17.434 --> 00:04:18.967 +it's, like, an important thing to use, + +00:04:18.968 --> 00:04:21.660 +but it's probably not giving you the complete answer. + +NOTE Computation dependencies + +00:04:21.760 --> 00:04:25.973 +So what we can instead do is basically figure out + +00:04:25.974 --> 00:04:28.554 +the whole computation dependencies here. + +00:04:28.555 --> 00:04:31.275 +So let's say if I look at this buffer, + +00:04:31.276 --> 00:04:35.076 +here's how all the blocks are connected. + +00:04:35.077 --> 00:04:37.656 +So as you can see the plot code block + +00:04:37.657 --> 00:04:40.117 +is dependent on c and then legendpg, + +00:04:40.118 --> 00:04:43.918 +and they themselves are dependent on these other nodes. + +00:04:43.919 --> 00:04:47.279 +So when I make a change in b, I only want b to run + +00:04:47.280 --> 00:04:50.844 +and then c and then plot. I don't want anything else to run. + +00:04:50.845 --> 00:04:54.267 +So what I did was I wrote a small minor mode for Org Mode + +00:04:54.268 --> 00:04:55.368 +which does exactly this. + +00:04:55.369 --> 00:04:57.769 +So whenever you are in a code block + +00:04:57.770 --> 00:04:59.871 +and you are making a change and then you save it, + +00:04:59.872 --> 00:05:01.913 +it will just follow the trail from that code block + +00:05:01.914 --> 00:05:05.355 +to every other descendant which is going to be impacted, + +00:05:05.356 --> 00:05:09.719 +and it just runs all of them, and nothing else gets executed. + +00:05:09.720 --> 00:05:13.119 +So to see it in action, I will just enable that mode. + +00:05:13.120 --> 00:05:17.021 +Yeah, right. So now here, if I change this 113 to 112 + +00:05:17.022 --> 00:05:21.243 +and I save, this code, this variable gets changed. + +00:05:21.244 --> 00:05:23.744 +It's the same value because I did not update it again. + +00:05:23.745 --> 00:05:25.719 +And you can also see b also got changed + +00:05:25.720 --> 00:05:29.667 +because it's just following all the execution order and so on. + +00:05:29.668 --> 00:05:31.727 +The plot also got updated. + +00:05:31.728 --> 00:05:34.068 +We will be able to see more clearly + +00:05:34.069 --> 00:05:36.402 +once I change something more substantial. + +00:05:36.402 --> 00:05:36.402 +So here's another variable. + +00:05:36.403 --> 00:05:41.332 +So I added a small toggle button here, + +00:05:41.333 --> 00:05:43.468 +which is again part of the minor mode. + +00:05:43.469 --> 00:05:45.209 +So since this is nil, if I toggle it, + +00:05:45.210 --> 00:05:49.300 +it will become true. And this variable dictates whether + +00:05:49.400 --> 00:05:51.174 +the plot will have the legend or not. + +00:05:51.175 --> 00:05:54.457 +So if I toggle it to be t, now it's t + +00:05:54.458 --> 00:05:57.900 +and you can see that the plot has legend that's visible. + +00:05:57.901 --> 00:06:03.139 +If I toggle it back again to nil, the legend is gone. + +00:06:03.140 --> 00:06:04.533 +Now this is nice, this... + +NOTE Making this even better + +00:06:04.534 --> 00:06:06.380 +This is already pretty helpful for me + +00:06:06.480 --> 00:06:10.179 +but what we can do is we can make it even better. + +00:06:10.180 --> 00:06:11.400 +So one of the nicer ideas + +00:06:11.401 --> 00:06:13.015 +from these reactive notebooks + +00:06:13.016 --> 00:06:16.078 +is this idea of having an infinite canvas + +00:06:16.079 --> 00:06:19.022 +where you don't look at the document model, + +00:06:19.023 --> 00:06:20.623 +you look at the whole document + +00:06:20.624 --> 00:06:25.008 +as a canvas of multiple connected documents. + +00:06:25.009 --> 00:06:26.589 +One good thing that happens there is that + +00:06:26.590 --> 00:06:29.550 +you can basically have a piece of code somewhere + +00:06:29.551 --> 00:06:30.410 +and then piece of code + +00:06:30.411 --> 00:06:32.499 +somewhere very different position in the document, + +00:06:32.500 --> 00:06:34.732 +but you can put them together in the canvas + +00:06:34.733 --> 00:06:36.933 +and then see them side by side. + +00:06:36.934 --> 00:06:38.294 +So here also, let's say + +00:06:38.295 --> 00:06:41.996 +if I want to just have this image shown up at the top, + +00:06:41.997 --> 00:06:45.857 +what I can do is like I can pop this out, + +00:06:45.858 --> 00:06:49.938 +which opens a child frame, and then I can just go here. + +00:06:49.939 --> 00:06:52.460 +This child frame is showing the same image. + +00:06:52.461 --> 00:06:55.502 +So there's no change. So if I toggle this variable here, + +00:06:55.503 --> 00:06:58.423 +you can see that the image is updated. + +00:06:58.424 --> 00:07:02.199 +If I toggle it back to nil, the image, the legend is gone. + +00:07:02.200 --> 00:07:03.367 +And you can obviously, you know, + +00:07:03.368 --> 00:07:08.690 +you can make a lot of things come up as child frames. + +00:07:08.691 --> 00:07:09.430 +This is the same image. + +00:07:09.431 --> 00:07:11.291 +So even if you go down to the document, + +00:07:11.292 --> 00:07:13.810 +you will see the same image. + +00:07:13.811 --> 00:07:18.174 +So yeah, this is what I have right now. + +00:07:18.175 --> 00:07:21.956 +I'm definitely looking forward to making it more useful, + +00:07:21.957 --> 00:07:25.599 +probably including more kinds of child frames, + +00:07:25.600 --> 00:07:29.965 +maybe like making the whole document an infinite canvas. + +NOTE Wrapping up + +00:07:29.966 --> 00:07:32.099 +Alright, so that's the talk. + +00:07:32.100 --> 00:07:33.346 +If you're interested in the codebase, + +00:07:33.347 --> 00:07:34.446 +here's the homepage + +00:07:34.447 --> 00:07:35.546 +for the project [https://dev.lepisma.xyz/git/ob-rx]. + +00:07:35.547 --> 00:07:37.566 +So the next steps for me are basically + +00:07:37.567 --> 00:07:40.647 +making my workflow easier in matplotlib, + +00:07:40.648 --> 00:07:42.587 +which is a Python-based library, + +00:07:42.588 --> 00:07:45.348 +and d3.js, which is for JavaScript. + +00:07:45.349 --> 00:07:47.888 +For the JS thing, I might have to add + +00:07:47.889 --> 00:07:49.540 +the interactive JS child frames, + +00:07:49.640 --> 00:07:51.829 +and I am also looking forward to building something + +00:07:51.830 --> 00:07:53.969 +which can replicate the work + +00:07:53.970 --> 00:07:56.750 +of the Observable's infinite canvas, + +00:07:56.751 --> 00:07:57.490 +because that's something + +00:07:57.491 --> 00:08:00.619 +which I found really useful in my work with + +00:08:00.620 --> 00:08:02.240 +just JS visualizations. + +00:08:02.340 --> 00:08:05.540 +So yeah, happy to take questions on Etherpad + +00:08:05.560 --> 00:08:08.240 +and thank you for your time. diff --git a/2025/captions/emacsconf-2025-python--interactive-python-programming-in-emacs--david-vujic--main.vtt b/2025/captions/emacsconf-2025-python--interactive-python-programming-in-emacs--david-vujic--main.vtt new file mode 100644 index 00000000..d63a36c8 --- /dev/null +++ b/2025/captions/emacsconf-2025-python--interactive-python-programming-in-emacs--david-vujic--main.vtt @@ -0,0 +1,731 @@ +WEBVTT captioned by sachac + +00:00:00.000 --> 00:00:04.439 +Okay, so welcome to this session about interactive Python + +00:00:04.440 --> 00:00:09.679 +programming. My name is David Vujic and I live and work in + +00:00:09.680 --> 00:00:15.319 +Stockholm, Sweden. a developer and today I focus + +00:00:15.320 --> 00:00:20.439 +mainly on Python software development. So I do this at work + +00:00:20.440 --> 00:00:25.999 +and I also do this on my spare time in my open source projects. + +00:00:26.000 --> 00:00:30.479 +Before that, I've been part of the Lisp community. I've + +00:00:30.480 --> 00:00:33.700 +been a Clojure developer, and also, like, way back, + +00:00:33.701 --> 00:00:40.279 +I was in the Microsoft world and developed C# and .NET stuff. + +00:00:40.280 --> 00:00:45.999 +What I've been doing lately is to try to improve the + +00:00:46.000 --> 00:00:52.399 +developer experience when you write Python code. So what I + +00:00:52.400 --> 00:00:56.159 +want to talk about is this, but also I want to begin with + +00:00:56.160 --> 00:01:00.839 +feedback loops because I think it's very related to this + +00:01:00.840 --> 00:01:05.359 +interactive programming style, like having this nice + +00:01:05.360 --> 00:01:07.067 +feedback when you write code. + +00:01:07.068 --> 00:01:10.533 +So I'm going to begin with that. + +NOTE Feedback loops + +00:01:10.534 --> 00:01:14.199 +So this image, you know, this circle is supposed to be a + +00:01:14.200 --> 00:01:19.879 +visualization of a feedback loop. Let's say we write our + +00:01:19.880 --> 00:01:25.239 +code and then we deploy it to production. Then when it's + +00:01:25.240 --> 00:01:29.639 +running there, we can check if things work, or if maybe someone + +00:01:29.640 --> 00:01:35.319 +else will let us know. Maybe our customers will let us know. + +00:01:35.320 --> 00:01:39.639 +That's a pretty slow feedback loop with potential risks of + +00:01:39.640 --> 00:01:41.867 +damaging your business or whatever. + +00:01:41.868 --> 00:01:44.167 +This is obvious, of course. + +00:01:44.168 --> 00:01:50.000 +So a faster feedback loop probably is to have + +00:01:50.001 --> 00:01:54.066 +some kind of automation when you do commits + +00:01:54.067 --> 00:01:59.733 +or maybe you have this pull request things and even reviews. + +00:01:59.734 --> 00:02:02.933 +So maybe not always as fast as deploy, + +00:02:02.934 --> 00:02:05.839 +don't deploy directly to production, but + +00:02:05.840 --> 00:02:10.539 +it's probably safer and often you get this automated + +00:02:10.540 --> 00:02:16.199 +feedback faster anyway. But it's still kind of slow. You + +00:02:16.200 --> 00:02:20.239 +have to wait. You have to push things to GitHub maybe and + +00:02:20.240 --> 00:02:24.279 +wait. So there's faster ways for sure to get feedback. + +00:02:24.280 --> 00:02:27.967 +So a much faster way is to write code, + +00:02:27.968 --> 00:02:31.367 +and write some unit tests, and run those unit tests. + +00:02:31.368 --> 00:02:33.467 +So then you do everything on your local machine + +00:02:33.468 --> 00:02:39.039 +and you will fairly quickly learn if your code does + +00:02:39.040 --> 00:02:47.159 +what you think it does or if it doesn't. I want to zoom in to + +00:02:47.160 --> 00:02:55.999 +this test write code and test flow a bit. Let's do that. + +NOTE Test-driven development + +00:02:56.000 --> 00:02:59.759 +As a developer, I have used a thing called test-driven + +00:02:59.760 --> 00:03:05.999 +development for quite some time. I find that this way of + +00:03:06.000 --> 00:03:11.259 +working is very fast when it comes to getting feedback on + +00:03:11.260 --> 00:03:14.519 +what your code does and how you should continue the + +00:03:14.520 --> 00:03:19.980 +development. So, test-driven development, + +00:03:19.981 --> 00:03:24.220 +basically that you start writing a test for + +00:03:24.221 --> 00:03:27.020 +something that you want to develop, and then you continue + +00:03:27.021 --> 00:03:31.019 +developing that, and then you go back to the test, and modify + +00:03:31.020 --> 00:03:35.079 +and modify the code, and you go back and forth between the + +00:03:35.080 --> 00:03:36.959 +tests and the code. + +00:03:36.960 --> 00:03:44.419 +It's sort of like a ping-pong game. I find this very + +00:03:44.420 --> 00:03:50.519 +effective when you want to get feedback and to know how to + +00:03:50.520 --> 00:03:57.233 +continue the development. The most important thing + +00:03:57.234 --> 00:04:01.700 +that I feel is that you know what the code does. + +00:04:01.701 --> 00:04:05.559 +You learn very quickly. + +NOTE REPL-driven development + +00:04:05.560 --> 00:04:12.199 +Let's zoom into this TDD flow a little bit. The last couple of + +00:04:12.200 --> 00:04:17.379 +years, I've been doing a slightly different thing which is + +00:04:17.380 --> 00:04:21.979 +called REPL-driven development. REPL-driven + +00:04:21.980 --> 00:04:25.719 +development is very similar to test-driven development, + +00:04:25.720 --> 00:04:31.159 +but I find it even quicker. You get feedback even quicker + +00:04:31.160 --> 00:04:34.979 +than with a regular TDD setup. So REPL-driven development + +00:04:34.980 --> 00:04:41.199 +is about writing and evaluating code in a REPL basically. + +00:04:41.200 --> 00:04:46.839 +And you can do experiments and you can refactor and + +00:04:46.840 --> 00:04:51.699 +re-evaluate and you get instant feedback on what the code + +00:04:51.700 --> 00:04:54.799 +does and what you need to change. So I think that's even + +00:04:54.800 --> 00:04:59.519 +faster than test-driven development. + +00:04:59.520 --> 00:05:02.899 +Okay, REPL driven development. Let's go back. What's the + +00:05:02.900 --> 00:05:10.759 +REPL? Most of developers know what a REPL is. The most common + +00:05:10.760 --> 00:05:16.399 +setup is you open this shell and you use the REPL for your + +00:05:16.400 --> 00:05:19.359 +programming language. In this case I'm using the Python + +00:05:19.360 --> 00:05:25.619 +REPL or the IPython REPL which is an enhanced REPL for Python + +00:05:25.620 --> 00:05:30.679 +development. So what happens here is that we start a REPL + +00:05:30.680 --> 00:05:34.919 +session in isolation. So this session knows about the + +00:05:34.920 --> 00:05:38.119 +Python environment. So it knows about the Python language + +00:05:38.120 --> 00:05:42.359 +basically. So as soon as we start writing things, adding + +00:05:42.360 --> 00:05:47.359 +variables or creating writing functions or even doing + +00:05:47.360 --> 00:05:51.679 +imports. Then the session will be more and more aware of the + +00:05:51.680 --> 00:05:55.819 +code so we will add things to the to the session and then that + +00:05:55.820 --> 00:06:00.519 +means that we can run functions we can print out these + +00:06:00.520 --> 00:06:05.859 +variables and things like that. But with REPL driven + +00:06:05.860 --> 00:06:09.839 +development it's not really that well at least not what I + +00:06:09.840 --> 00:06:14.039 +mean with REPL driven development. So what I'm thinking of + +00:06:14.040 --> 00:06:19.639 +is that you are in your code editor where you have your + +00:06:19.640 --> 00:06:22.799 +autocomplete, and you have your syntax highlighting and + +00:06:22.800 --> 00:06:30.459 +your favorite theme, color theme, and all of those things. But + +00:06:30.460 --> 00:06:34.979 +instead, you have this running REPL in the background or in a + +00:06:34.980 --> 00:06:41.139 +smaller window or buffer. So that means that you write code + +00:06:41.140 --> 00:06:45.319 +and you can send that code to the running REPL, to the REPL + +00:06:45.320 --> 00:06:50.399 +session. You write and do everything as you would do when + +00:06:50.400 --> 00:06:55.219 +writing your code basically. In this case, in this + +00:06:55.220 --> 00:07:00.599 +example, I have evaluated these two functions. I've sent + +00:07:00.600 --> 00:07:05.819 +them to the REPL session so it's aware of these functions. + +00:07:05.820 --> 00:07:10.399 +Then I switched to a separate different module and + +00:07:10.400 --> 00:07:14.039 +evaluated that one. So the REPL session now knows about + +00:07:14.040 --> 00:07:19.039 +these two functions and also these two variables. That + +00:07:19.040 --> 00:07:23.999 +means that I can evaluate the state of those variables and + +00:07:24.000 --> 00:07:28.999 +change code and re-evaluate and things like that. So in this + +00:07:29.000 --> 00:07:33.639 +example if you look in the smaller area there you see that I + +00:07:33.640 --> 00:07:39.639 +have evaluated this res variable on line 6 and the output was + +00:07:39.640 --> 00:07:42.399 +that it's a dictionary with two keys and two values + +00:07:42.400 --> 00:07:51.219 +basically. So this setup works in basically any of your + +00:07:51.220 --> 00:07:54.079 +favorite code editors. So you can do this in Visual Studio + +00:07:54.080 --> 00:08:01.239 +Code, you can do this in PyCharm or Vim. But what I have done is + +00:08:01.240 --> 00:08:07.119 +that... More like what I have missed is that when I write code + +00:08:07.120 --> 00:08:10.239 +and do this evaluation, this is really cool, but then I need + +00:08:10.240 --> 00:08:15.459 +to switch context if I want to see the result. I have to switch + +00:08:15.460 --> 00:08:21.979 +context to this other window. I + +00:08:21.980 --> 00:08:25.759 +have my focus on the code and then I have to look in a different + +00:08:25.760 --> 00:08:31.799 +place to know the results. And if it's a larger output, then + +00:08:31.800 --> 00:08:37.479 +maybe I need to scroll. So I wanted to find out if it was + +00:08:37.480 --> 00:08:43.479 +possible to make this even smoother and faster, this + +00:08:43.480 --> 00:08:45.479 +feedback loop even faster, so I don't have to switch + +00:08:45.480 --> 00:08:52.119 +context. What I've done here is that... I can select a row or a + +00:08:52.120 --> 00:08:58.079 +region and I can evaluate and then an overlay, a small pop-up + +00:08:58.080 --> 00:09:03.119 +shows up with the evaluated result right next to it. So I can + +00:09:03.120 --> 00:09:07.519 +change code and re-evaluate and quickly see the result of it + +00:09:07.520 --> 00:09:12.640 +without doing this context switching. So the way I've done + +00:09:12.641 --> 00:09:20.679 +it is that I wanted to reuse the existing tooling that I + +00:09:20.680 --> 00:09:27.739 +already had. I know that my in-editor REPL, the IPython + +00:09:27.740 --> 00:09:31.559 +REPL, already does this evaluation. So I figured maybe I can + +00:09:31.560 --> 00:09:35.359 +extract the data and do this visualization as a separate + +00:09:35.360 --> 00:09:40.839 +thing. That's how I've done it. What I've done is that + +00:09:40.840 --> 00:09:47.199 +I've created this overlay and placed it where my cursor + +00:09:47.200 --> 00:09:50.859 +currently is, right next to the code. Then I've + +00:09:50.860 --> 00:09:55.719 +extracted the evaluated result and put it in this overlay. + +00:09:55.720 --> 00:10:01.039 +I also want this overlay to have this nice looking syntax, + +00:10:01.040 --> 00:10:04.759 +so I've set it to this Python mode, so we get this syntax + +00:10:04.760 --> 00:10:10.559 +highlighting. Make it look very readable. And as a nice + +00:10:10.560 --> 00:10:16.879 +developer experience thing, + +00:10:16.880 --> 00:10:20.379 +when you move the cursor, of course you don't want the + +00:10:20.380 --> 00:10:25.679 +overlay to be there. You want it to disappear. So those kinds + +00:10:25.680 --> 00:10:28.999 +of things I've added. So putting the overlay at the right + +00:10:29.000 --> 00:10:33.279 +place and feed it with the evaluated data and then make it + +00:10:33.280 --> 00:10:39.839 +disappear when it's not interesting to look at anymore. + +00:10:39.840 --> 00:10:44.639 +What I've described so far is something that I use on a + +00:10:44.640 --> 00:10:50.639 +daily basis, and it covers most of my needs while doing Python + +00:10:50.640 --> 00:10:56.119 +development. But one thing I still miss, and I miss it from my + +00:10:56.120 --> 00:11:03.479 +days as a Clojure developer, because over there we could + +00:11:03.480 --> 00:11:07.919 +have a running app on our local machine and we can have our + +00:11:07.920 --> 00:11:12.719 +editor, and the app and the editor were connected. So when I + +00:11:12.720 --> 00:11:17.199 +did some changes in the code, the app would change without + +00:11:17.200 --> 00:11:20.559 +any restarts or anything like that. And the same if I would + +00:11:20.560 --> 00:11:24.679 +change the state of the app, I can inspect the state from the + +00:11:24.680 --> 00:11:28.919 +code. So they were connected. They are connected. So I was + +00:11:28.920 --> 00:11:32.839 +thinking, hey, this would be really cool if we could have + +00:11:32.840 --> 00:11:39.199 +something like this in Python. And that reminded me of + +00:11:39.200 --> 00:11:43.839 +Jupyter and Jupyter notebooks because I think notebooks, + +00:11:43.840 --> 00:11:49.659 +the way you do things there, is very similar to what I was + +00:11:49.660 --> 00:11:56.879 +trying to achieve. So I was reading up a little bit on how this + +00:11:56.880 --> 00:12:00.919 +notebook thing works. It turns out that a notebook is a + +00:12:00.920 --> 00:12:05.279 +client that talks to a server, that communicates with a + +00:12:05.280 --> 00:12:08.799 +server. It's on the server that all this Python + +00:12:08.800 --> 00:12:14.159 +evaluation and all this thing happens. Then what I've + +00:12:14.160 --> 00:12:19.659 +done is that instead of starting up IPython in my editor, I + +00:12:19.660 --> 00:12:23.519 +start the Jupyter console instead. And then I can give it + +00:12:23.520 --> 00:12:27.159 +that unique ID and it will be connected to that running + +00:12:27.160 --> 00:12:30.919 +kernel. + +NOTE FastAPI CRUD + +00:12:30.920 --> 00:12:37.199 +In this example, I've created this FastAPI CRUD app that + +00:12:37.200 --> 00:12:41.919 +has this create, read, update, and delete endpoints. It + +00:12:41.920 --> 00:12:46.399 +has this, it's locally running, it has this database where + +00:12:46.400 --> 00:12:51.639 +you can do all these things. I'm running this FastAPI app + +00:12:51.640 --> 00:12:58.059 +in the kernel and then I've connected to, I've connected to + +00:12:58.060 --> 00:13:03.239 +the kernel in my editor too. Both of them are connected to + +00:13:03.240 --> 00:13:09.719 +the kernel. What I do now is that I want to initially create + +00:13:09.720 --> 00:13:15.239 +some data. I'm going to add this, creating this message. + +00:13:15.240 --> 00:13:19.899 +What I get back is a message ID. I want to experiment in + +00:13:19.900 --> 00:13:24.359 +my browser. What do I get with that message ID? I'm + +00:13:24.360 --> 00:13:30.239 +evaluating the read function. I instantly get this + +00:13:30.240 --> 00:13:34.779 +evaluated result, which was this hello world text. So what + +00:13:34.780 --> 00:13:39.919 +happens if I do some changes in this app? I'm going to grab + +00:13:39.920 --> 00:13:49.659 +this message ID and write something else. + +00:13:49.660 --> 00:13:53.759 +Now I can evaluate the same thing again, and you can see that + +00:13:53.760 --> 00:14:02.399 +the content has changed to this new value. My editor isn't + +00:14:02.400 --> 00:14:07.719 +in any debug mode or something like that. It doesn't know + +00:14:07.720 --> 00:14:11.239 +what database it is. It doesn't have any environment + +00:14:11.240 --> 00:14:14.479 +variables set up or something like that. It is only + +00:14:14.480 --> 00:14:17.599 +connected to the kernel, and the kernel is aware of that. It's + +00:14:17.600 --> 00:14:20.479 +running the app. It has the connection strings and + +00:14:20.480 --> 00:14:28.799 +everything that is needed. So that's how this thing works. + +00:14:28.800 --> 00:14:34.199 +Now I want to do some inline hacking because I want to store + +00:14:34.200 --> 00:14:37.799 +this input that is sent from this app because I want to work + +00:14:37.800 --> 00:14:42.039 +with it afterwards. I can add this dictionary that stores + +00:14:42.040 --> 00:14:48.759 +this message. I'm updating the source code of this app, and + +00:14:48.760 --> 00:15:03.079 +when I run any of these endpoints again, you will see that + +00:15:03.080 --> 00:15:08.759 +the state changes, and the new inputs, I can grab and I can use + +00:15:08.760 --> 00:15:14.399 +them for quick evaluation or testing. This example is + +00:15:14.400 --> 00:15:18.519 +really simple. It was just an integer. For example, if you + +00:15:18.520 --> 00:15:23.519 +are sending a more complex object, maybe a pydantic schema + +00:15:23.520 --> 00:15:28.199 +or something, and you want to inspect what's coming in, and if + +00:15:28.200 --> 00:15:34.199 +you have some sort of validation that you want to test out. + +00:15:34.200 --> 00:15:38.399 +The configuration or the code that I wrote to make this work + +00:15:38.400 --> 00:15:44.159 +is a little bit different than just adding an overlay. I'm + +00:15:44.160 --> 00:15:50.999 +using this overlay just like with the IPython example, but in + +00:15:51.000 --> 00:15:57.839 +this case, when I change code, I have to think about where that + +00:15:57.840 --> 00:16:02.159 +code lives, because it's the app that runs the code. So it's + +00:16:02.160 --> 00:16:07.039 +in the app context I need to manipulate with the data. If you + +00:16:07.040 --> 00:16:11.919 +have started the app from maybe a main function and that + +00:16:11.920 --> 00:16:17.879 +module imports namespaces, then you need to, if you want to + +00:16:17.880 --> 00:16:22.359 +update a function or something like that, you need to update + +00:16:22.360 --> 00:16:26.679 +it in the correct namespace. What I did before in IPython + +00:16:26.680 --> 00:16:29.919 +by adding and changing things, everything ends up in the + +00:16:29.920 --> 00:16:34.439 +global namespace. But here, if you want the app to actually + +00:16:34.440 --> 00:16:38.479 +react to the changes, you need to put it in the right + +00:16:38.480 --> 00:16:43.479 +namespace. So that's what I do here. I do some lookups, where + +00:16:43.480 --> 00:16:49.139 +is this function, and then I do this reload of this function or + +00:16:49.140 --> 00:16:54.799 +module. And when I was developing this, I was thinking, hey, + +00:16:54.800 --> 00:16:59.319 +this is really ugly. I'm in this REPL and do some + +00:16:59.320 --> 00:17:03.559 +manipulation of the imports and things like that. That + +00:17:03.560 --> 00:17:09.759 +didn't feel good. Then I was reminded of the IPython. And + +00:17:09.760 --> 00:17:15.519 +IPython has this feature to reload any updated + +00:17:15.520 --> 00:17:19.119 +submodules. I was curious how do they do it. I looked in the + +00:17:19.120 --> 00:17:24.079 +IPython source code and saw that they also use importlib and + +00:17:24.080 --> 00:17:28.359 +reloading of this module. Once I've learned that, then I + +00:17:28.360 --> 00:17:32.599 +stopped thinking that my code was hacky. I thought it was + +00:17:32.600 --> 00:17:37.159 +good enough at least. + +NOTE Testing with an LLM + +00:17:37.160 --> 00:17:45.059 +But one thing that has bothered me for a long time is I quite + +00:17:45.060 --> 00:17:50.199 +often want to test out and evaluate individual rows that + +00:17:50.200 --> 00:17:58.559 +lives in a function. Quite often, this code uses the input + +00:17:58.560 --> 00:18:02.639 +to that function like the input parameters. To be able to + +00:18:02.640 --> 00:18:07.719 +do that, I need to manually type some fake data and set it to + +00:18:07.720 --> 00:18:12.279 +this variable, and then I can evaluate the code. But I think + +00:18:12.280 --> 00:18:17.779 +that takes... That slows me down. I was thinking, maybe I can + +00:18:17.780 --> 00:18:23.439 +do this in a quicker way, so I have this quicker feedback, so I + +00:18:23.440 --> 00:18:27.933 +can run this or evaluate this code much quicker. + +00:18:27.934 --> 00:18:29.439 +So my idea was maybe I + +00:18:29.440 --> 00:18:35.239 +can use an LLM for this. If I give it the parameters, maybe it + +00:18:35.240 --> 00:18:41.119 +can return some random data so I don't have to write it + +00:18:41.120 --> 00:18:44.119 +myself. I ended up doing that. I have this source code. + +00:18:44.120 --> 00:18:50.399 +I'm loading the REPL with the code. Then I select this + +00:18:50.400 --> 00:18:56.719 +function name and the parameters with its data type. I + +00:18:56.720 --> 00:19:02.839 +have this prompt that instructs the LLM to come up with fake + +00:19:02.840 --> 00:19:06.239 +data based on the tag name and on the data type. And then I can + +00:19:06.240 --> 00:19:10.099 +send that to the REPL. I do that with a key command. Then + +00:19:10.100 --> 00:19:16.019 +I can proceed by running the code within the function that + +00:19:16.020 --> 00:19:21.719 +uses these inputs. This works for all the data types. If + +00:19:21.720 --> 00:19:26.279 +there's a custom data type, you need to give the LLM extra + +00:19:26.280 --> 00:19:30.399 +context. So that's something to think about. Once it knows + +00:19:30.400 --> 00:19:35.679 +the context, it can generate this fake data that very often is + +00:19:35.680 --> 00:19:39.839 +good enough just to test out, you know, like I've done here, like + +00:19:39.840 --> 00:19:45.399 +string... sorry, list destructuring and parsing and things + +00:19:45.400 --> 00:19:51.879 +like that. I think that was all I had, and thank you for + +00:19:51.880 --> 00:19:52.920 +listening! diff --git a/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main--chapters.vtt b/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main--chapters.vtt new file mode 100644 index 00000000..dbd303e8 --- /dev/null +++ b/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main--chapters.vtt @@ -0,0 +1,32 @@ +WEBVTT + + +00:00:00.820 --> 00:01:43.599 +Introduction + +00:01:43.600 --> 00:04:00.919 +Capture + +00:04:00.920 --> 00:05:03.478 +Organizing + +00:05:03.479 --> 00:07:55.479 +Ebib + +00:07:55.480 --> 00:09:36.178 +Filters + +00:09:36.179 --> 00:12:50.539 +Dependent databases + +00:12:50.540 --> 00:15:02.439 +Reading lists + +00:15:02.440 --> 00:18:05.639 +Special org-roam-node-find + +00:18:05.640 --> 00:19:21.819 +Annotations + +00:19:21.820 --> 00:20:14.000 +Wrapping up diff --git a/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main.vtt b/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main.vtt new file mode 100644 index 00000000..33a06efa --- /dev/null +++ b/2025/captions/emacsconf-2025-reference--emacs-as-a-fullyfledged-reference-manager--vidianos-giannitsis--main.vtt @@ -0,0 +1,1035 @@ +WEBVTT captioned by vidianos + +NOTE Introduction + +00:00:00.820 --> 00:00:03.079 +So, hello everyone, welcome to EmacsConf. + +00:00:03.080 --> 00:00:06.519 +My name is Vidianos, and I'm a PhD student in KU Leuven, + +00:00:06.520 --> 00:00:08.279 +and today I'm going to be showing you + +00:00:08.280 --> 00:00:12.679 +how I managed to use Emacs as a reference manager, + +00:00:12.680 --> 00:00:17.219 +replacing what was for me Zotero, + +00:00:17.220 --> 00:00:24.439 +to a fully fledged approach inside Emacs. + +00:00:24.440 --> 00:00:26.639 +So, what is my typical reference workflow? + +00:00:26.640 --> 00:00:28.479 +First I need to find literature, + +00:00:28.480 --> 00:00:30.439 +then I need to collect and organize it, + +00:00:30.440 --> 00:00:32.599 +which I originally did with Zotero, + +00:00:32.600 --> 00:00:36.611 +but now with Emacs centered tools + +00:00:36.612 --> 00:00:39.879 +such as zotra and ebib. + +00:00:39.880 --> 00:00:41.279 +Then I create a reading list. + +00:00:41.280 --> 00:00:43.439 +This is a new addition to my workflow + +00:00:43.440 --> 00:00:47.679 +I started doing after moving this approach to Emacs + +00:00:47.680 --> 00:00:50.119 +because now everything is well integrated. + +00:00:50.120 --> 00:00:53.839 +I have made a very nice reading list implementation + +00:00:53.840 --> 00:00:57.759 +inside org-roam which I am going to be showing today. + +00:00:57.760 --> 00:01:01.199 +Then obviously I need to read the literature, take notes, + +00:01:01.200 --> 00:01:04.239 +organize the notes, and ensure I am actually learning + +00:01:04.240 --> 00:01:06.539 +from what I am reading. This is then done through packages + +00:01:06.540 --> 00:01:09.159 +such as org-noter and org-roam + +00:01:09.160 --> 00:01:11.359 +and is not going to be the focus of this talk. + +00:01:11.360 --> 00:01:14.239 +I already gave a talk about this part of my workflow, + +00:01:14.240 --> 00:01:17.959 +which I've been doing for many years now. + +00:01:17.960 --> 00:01:20.439 +You can find that + +00:01:20.440 --> 00:01:23.239 +or you can find many other people's approaches + +00:01:23.240 --> 00:01:25.999 +to reading literature and taking notes + +00:01:26.000 --> 00:01:31.039 +as it is quite a popular topic in the Emacs community. + +00:01:31.040 --> 00:01:32.759 +Lastly, I will have a short section + +00:01:32.760 --> 00:01:37.039 +about how I ensure that I can recall the knowledge + +00:01:37.040 --> 00:01:39.679 +from this literature very easily + +00:01:39.680 --> 00:01:43.599 +through this reference management system. + +NOTE Capture + +00:01:43.600 --> 00:01:48.199 +So, how can we capture an article in Emacs? + +00:01:48.200 --> 00:01:52.099 +One of the most commonly known packages is doi-utils + +00:01:52.100 --> 00:01:58.879 +where doi-utils has a lot of useful things to do + +00:01:58.880 --> 00:02:03.519 +and one of them is to capture a paper, + +00:02:03.520 --> 00:02:06.459 +but you need a DOI, and for me, that is a bit inconvenient, + +00:02:06.460 --> 00:02:08.879 +because what I want to do is that + +00:02:08.880 --> 00:02:11.999 +I have a URL here of a paper, + +00:02:12.000 --> 00:02:16.519 +I want to just copy this URL, not copy the DOI, + +00:02:16.520 --> 00:02:23.199 +and be able to save it immediately to my bib file. + +00:02:23.200 --> 00:02:24.311 +And that can now be done + +00:02:24.312 --> 00:02:27.719 +through this function zotra-add-entry. + +00:02:27.720 --> 00:02:28.679 +And as you can see here, + +00:02:28.680 --> 00:02:31.079 +there is also a zotra-download-attachment + +00:02:31.080 --> 00:02:34.159 +that sometimes works, but not always. + +00:02:34.160 --> 00:02:35.879 +I don't personally recommend it. + +00:02:35.880 --> 00:02:39.599 +But the problem is that due to articles being + +00:02:39.600 --> 00:02:42.639 +locked behind paywalls in many cases, + +00:02:42.640 --> 00:02:45.639 +downloading attachments doesn't work + +00:02:45.640 --> 00:02:47.479 +through Emacs. Sometimes it doesn't work + +00:02:47.480 --> 00:02:51.119 +through Zotero. Sometimes you just need to go to the browser, + +00:02:51.120 --> 00:02:54.259 +say download PDF, and that's the only solution + +00:02:54.260 --> 00:02:58.399 +that will properly work. + +00:02:58.400 --> 00:03:01.519 +So how do I then add this to the paper? + +00:03:01.520 --> 00:03:04.711 +I need to find this paper that is here + +00:03:04.712 --> 00:03:05.879 +through ivy-bibtex. + +00:03:05.880 --> 00:03:10.239 +This is the bib file manager I use. + +00:03:10.240 --> 00:03:11.759 +There is also others like citar. + +00:03:11.760 --> 00:03:15.559 +I think citar has much better coding and integration + +00:03:15.560 --> 00:03:18.159 +with other packages, but I haven't really bothered + +00:03:18.160 --> 00:03:19.639 +to move from ivy-bibtex + +00:03:19.640 --> 00:03:24.439 +because it does basically everything I want perfectly. + +00:03:24.440 --> 00:03:28.279 +So I go here, say add pdf to library. + +00:03:28.280 --> 00:03:31.719 +I find where I saved it, + +00:03:31.720 --> 00:03:35.399 +and it will suggest to automatically name it something + +00:03:35.400 --> 00:03:37.445 +which is in full integration + +00:03:37.446 --> 00:03:40.919 +with the rest of my Emacs packages, + +00:03:40.920 --> 00:03:43.839 +and all the literature management stuff + +00:03:43.840 --> 00:03:46.799 +knows to find it with this exact name. + +00:03:46.800 --> 00:03:49.045 +So it iss saved and now + +00:03:49.046 --> 00:03:53.319 +when I try to create a file from this, + +00:03:53.320 --> 00:03:55.199 +which I will show later, + +00:03:55.200 --> 00:04:00.919 +you will see that the file will already appear there. + +NOTE Organizing + +00:04:00.920 --> 00:04:06.519 +So now let's go to organizing. + +00:04:06.520 --> 00:04:09.119 +Organizing in Zotero is typically done through a + +00:04:09.120 --> 00:04:10.519 +hierarchical folder structure. + +00:04:10.520 --> 00:04:14.999 +This is very familiar to most people and generally works, + +00:04:15.000 --> 00:04:19.519 +but being someone that takes notes using the Zettelkasten method, + +00:04:19.520 --> 00:04:24.319 +which has a fully flat hierarchy, nothing goes in folders, + +00:04:24.320 --> 00:04:27.959 +everything is in the same folder, + +00:04:27.960 --> 00:04:31.959 +and you find everything because it's connected to other things. + +00:04:31.960 --> 00:04:34.839 +We have some basic indexes, + +00:04:34.840 --> 00:04:37.799 +from where you can jump to different points. + +00:04:37.800 --> 00:04:41.359 +I love this structure, so I also wanted to include it + +00:04:41.360 --> 00:04:42.919 +in my bibliography management, + +00:04:42.920 --> 00:04:45.579 +because with folders you have problems like, + +00:04:45.580 --> 00:04:48.359 +this article can go in that folder, can go in that folder, + +00:04:48.360 --> 00:04:51.079 +can go in that folder. Where do I actually put it? + +00:04:51.080 --> 00:04:54.439 +Do I put copies of it in different folders? + +00:04:54.440 --> 00:04:59.279 +It's just confusing and not really practical in my opinion. + +00:04:59.280 --> 00:05:03.478 +So I tried to do this approach inside Emacs. + +NOTE Ebib + +00:05:03.479 --> 00:05:08.239 +And how? With Ebib. + +00:05:08.240 --> 00:05:11.719 +Ebib is an amazing software built inside Emacs. + +00:05:11.720 --> 00:05:15.679 +It's a reference manager and it works absolutely amazing + +00:05:15.680 --> 00:05:18.199 +if you configure it to your liking. + +00:05:18.200 --> 00:05:21.119 +So let's open ebib first. + +00:05:21.120 --> 00:05:23.519 +This is the interface you will see when opening. + +00:05:23.520 --> 00:05:27.159 +Actually, by default you will not see anything, + +00:05:27.160 --> 00:05:30.799 +but I have open three bib files. + +00:05:30.800 --> 00:05:36.119 +These are opened by default on boot of ebib for me. + +00:05:36.120 --> 00:05:39.839 +These are my three main master bib files. + +00:05:39.840 --> 00:05:41.719 +This is the Zotero master bib file, + +00:05:41.720 --> 00:05:44.599 +which only Zotero can touch. If I change it, + +00:05:44.600 --> 00:05:47.959 +it will be overwritten. This is my new master bib, + +00:05:47.960 --> 00:05:53.759 +where I save all the files that I have now started using + +00:05:53.760 --> 00:05:55.599 +after switching to this approach. + +00:05:55.600 --> 00:05:59.119 +And then this is the master bib file + +00:05:59.120 --> 00:06:02.519 +for literature related to my PhD + +00:06:02.520 --> 00:06:07.479 +and things that I have already read. + +00:06:07.480 --> 00:06:09.759 +It's a very convenient interface. + +00:06:09.760 --> 00:06:14.140 +There is also search. There is one searching tool, + +00:06:14.141 --> 00:06:16.519 +the jump to entry, ebib-jump-to-entry, + +00:06:16.520 --> 00:06:23.039 +which searches through all open bib files for the title. + +00:06:23.040 --> 00:06:29.319 +So for example, I can search for membrane fabrication, + +00:06:29.320 --> 00:06:35.839 +because that's something I am currently doing, + +00:06:35.840 --> 00:06:41.919 +and go to this. There is another searching tool, + +00:06:41.920 --> 00:06:45.639 +the ebib-search, which searches through the database + +00:06:45.640 --> 00:06:48.359 +that you're on right now + +00:06:48.360 --> 00:06:50.399 +and it does a full text search, + +00:06:50.400 --> 00:06:53.459 +not only in the titles, but everywhere. + +00:06:53.460 --> 00:06:57.039 +So, for example, I see that in this paper + +00:06:57.040 --> 00:07:04.759 +if I go to the abstract and search for the word FTIR, + +00:07:04.760 --> 00:07:06.879 +which is a chemical analysis, + +00:07:06.880 --> 00:07:09.879 +it will tell me that it's here. + +00:07:09.880 --> 00:07:16.539 +Can it find it anywhere else? It cannot. That's okay. + +00:07:16.540 --> 00:07:18.111 +Let's search for something + +00:07:18.112 --> 00:07:21.679 +that we'll be able to find more easily, + +00:07:21.680 --> 00:07:24.799 +like, for example, membrane crystallization + +00:07:24.800 --> 00:07:27.539 +which is a main focus of my PhD. + +00:07:27.540 --> 00:07:30.319 +Then it will be able to find it many times, + +00:07:30.320 --> 00:07:35.159 +many, many times. + +00:07:35.160 --> 00:07:40.299 +I can also search on the next database + +00:07:40.300 --> 00:07:49.479 +or on this database and see where is everything that I want. + +00:07:49.480 --> 00:07:55.479 +So this is different searching tools which are very useful. + +NOTE Filters + +00:07:55.480 --> 00:08:01.700 +Then there's also another tool, that is, filters. + +00:08:01.701 --> 00:08:06.199 +So I can filter on any field. + +00:08:06.200 --> 00:08:09.739 +Like, for example, let's say on any field, + +00:08:09.740 --> 00:08:13.159 +and let's say I'm looking now again + +00:08:13.160 --> 00:08:16.279 +for membrane crystallization. + +00:08:16.280 --> 00:08:19.879 +This will now filter to all entries. + +00:08:19.880 --> 00:08:22.119 +You can see right now there's 18 entries here + +00:08:22.120 --> 00:08:28.019 +that mention these two words together in any field. + +00:08:28.020 --> 00:08:31.759 +Sometimes this is easier, because this is permanent. + +00:08:31.760 --> 00:08:33.699 +It's not like the search that we find one + +00:08:33.700 --> 00:08:35.218 +and then if you move, you've lost it, + +00:08:35.219 --> 00:08:37.885 +and you need to find it again. + +00:08:37.886 --> 00:08:39.679 +This is permanent until I say, + +00:08:39.680 --> 00:08:41.799 +okay, cancel the filter. + +00:08:41.800 --> 00:08:43.479 +I mostly work with filters, + +00:08:43.480 --> 00:08:47.059 +I think they're the most convenient. + +00:08:47.060 --> 00:08:49.679 +Then there's also the ebib-list-recent + +00:08:49.680 --> 00:08:51.479 +which is another very useful command. + +00:08:51.480 --> 00:08:53.359 +It asks you for a number of days + +00:08:53.360 --> 00:08:56.559 +and it will show you the files that were added + +00:08:56.560 --> 00:08:58.479 +in the most recent. + +00:08:58.480 --> 00:09:02.159 +So for example, show me the literature files + +00:09:02.160 --> 00:09:05.839 +that were added to this bib file in the last month. + +00:09:05.840 --> 00:09:09.799 +I will see five files in this case, + +00:09:09.800 --> 00:09:12.699 +because in this bib file, I have mostly entries + +00:09:12.700 --> 00:09:15.239 +that I have read, these are the files + +00:09:15.240 --> 00:09:18.959 +I have most recently read and added here. + +00:09:18.960 --> 00:09:21.599 +While if I go for example here and say that, + +00:09:21.600 --> 00:09:25.799 +these are files that maybe I haven't read yet, + +00:09:25.800 --> 00:09:31.099 +but I was planning to read. So this is something useful. + +00:09:31.100 --> 00:09:32.999 +Although for things I'm planning to read, + +00:09:33.000 --> 00:09:34.439 +I mostly use the reading list + +00:09:34.440 --> 00:09:36.178 +that I'm going to show next. + +NOTE Dependent databases + +00:09:36.179 --> 00:09:37.399 +But before that, + +00:09:37.400 --> 00:09:41.759 +a few more neat things that you can do in Ebib. + +00:09:41.760 --> 00:09:45.019 +So I have a list of dependent databases. + +00:09:45.020 --> 00:09:48.819 +For ease, I have already opened them here. + +00:09:48.820 --> 00:09:52.599 +These, as you can see, have two brackets here, + +00:09:52.600 --> 00:09:59.439 +indicating that they're dependent on phd_literature_1.bib, + +00:09:59.440 --> 00:10:06.378 +and these, in my case, act as the sort of index file, + +00:10:06.379 --> 00:10:10.911 +where I am tagging things based on the structure + +00:10:10.912 --> 00:10:14.651 +that I wanted to have for the organization. + +00:10:14.652 --> 00:10:16.478 +So all the organization is flat, + +00:10:16.479 --> 00:10:20.145 +all the literature is in phd_literature_1, + +00:10:20.146 --> 00:10:23.419 +however, I have this file + +00:10:23.420 --> 00:10:25.839 +that has 14 entries. + +00:10:25.840 --> 00:10:32.899 +I have another file here that has 20 entries. + +00:10:32.900 --> 00:10:35.719 +And these are smaller indexes + +00:10:35.720 --> 00:10:38.519 +where I can find things easier, + +00:10:38.520 --> 00:10:41.159 +but things are not limited to one of these. + +00:10:41.160 --> 00:10:45.599 +Things can be in all of these, or probably not all of these, + +00:10:45.600 --> 00:10:50.479 +but can be in three or four of these very easily. + +00:10:50.480 --> 00:10:55.219 +And how you add things is that I go here, + +00:10:55.220 --> 00:11:00.079 +and I say not r, it's... M for the dependent databases, + +00:11:00.080 --> 00:11:02.079 +and I add entry, and it will tell me + +00:11:02.080 --> 00:11:03.159 +"Where do you want to add this?" + +00:11:03.160 --> 00:11:05.159 +So when I read a new paper, + +00:11:05.160 --> 00:11:09.839 +I can say okay, this is related to these three tags, + +00:11:09.840 --> 00:11:11.239 +and this is sort of like, again, + +00:11:11.240 --> 00:11:13.185 +it's tagging it and it's putting it + +00:11:13.186 --> 00:11:15.059 +there, there, and there. + +00:11:15.060 --> 00:11:17.599 +And then this creates a flat structure + +00:11:17.600 --> 00:11:21.619 +that however has a great organization, + +00:11:21.620 --> 00:11:23.799 +similar to how Zettelkasten works + +00:11:23.800 --> 00:11:27.879 +and I really like working with something like this, + +00:11:27.880 --> 00:11:33.719 +with dependent databases. + +00:11:33.720 --> 00:11:36.539 +Another feature that I really like, + +00:11:36.540 --> 00:11:41.919 +another feature that exists by default... But if I tag, + +00:11:41.920 --> 00:11:45.279 +this tagging is done through "m", + +00:11:45.280 --> 00:11:49.019 +and then I can tag different files here, + +00:11:49.020 --> 00:11:52.159 +and this is to do different actions with these together, + +00:11:52.160 --> 00:11:53.585 +such as, for example, + +00:11:53.586 --> 00:11:55.585 +copy them to a different file, + +00:11:55.586 --> 00:11:59.459 +with "x" I can export the entries somewhere else, + +00:11:59.460 --> 00:12:00.685 +there are many things that you can do + +00:12:00.686 --> 00:12:01.439 +when you mark them. + +00:12:01.440 --> 00:12:05.319 +By the way, one of them is this function, + +00:12:05.320 --> 00:12:08.939 +which sees everything that I have marked + +00:12:08.940 --> 00:12:14.785 +and shows me an org-roam-node-find entry + +00:12:14.786 --> 00:12:18.052 +that is filtered to just these files. + +00:12:18.053 --> 00:12:20.885 +I can select one and it will take me + +00:12:20.886 --> 00:12:24.399 +to my notes on this specific paper. + +00:12:24.400 --> 00:12:25.719 +I find this very useful, + +00:12:25.720 --> 00:12:27.159 +because I can be looking for something + +00:12:27.160 --> 00:12:30.018 +and I can say, okay I remember, + +00:12:30.019 --> 00:12:31.399 +or I did some filtering, + +00:12:31.400 --> 00:12:34.099 +and I know it's in one of these files + +00:12:34.100 --> 00:12:37.239 +and now I want to see my in-depth notes on each one + +00:12:37.240 --> 00:12:41.079 +to remember where exactly I found it. + +00:12:41.080 --> 00:12:43.419 +So I find this kind of filtering, + +00:12:43.420 --> 00:12:50.539 +this org-roam related filtering, to be also very effective. + +NOTE Reading lists + +00:12:50.540 --> 00:12:54.079 +So now let's finally move to reading list. + +00:12:54.080 --> 00:12:55.399 +The reading list in ebib + +00:12:55.400 --> 00:13:00.259 +reminds me a lot of the philosophy that Emacs uses. + +00:13:00.260 --> 00:13:05.119 +By default, it is extremely bare bones, not very usable, + +00:13:05.120 --> 00:13:10.719 +but it is so customizable, to where you can do + +00:13:10.720 --> 00:13:13.019 +anything that you can imagine through it + +00:13:13.020 --> 00:13:16.479 +because the limit truly is your imagination. + +00:13:16.480 --> 00:13:21.039 +It's how much you can code into this + +00:13:21.040 --> 00:13:22.519 +that actually makes sense + +00:13:22.520 --> 00:13:25.479 +and you can actually imagine it working. + +00:13:25.480 --> 00:13:31.699 +Besides that, you can do anything really. + +00:13:31.700 --> 00:13:36.799 +So we can open ebib and try to find this paper + +00:13:36.800 --> 00:13:42.699 +that I just added here. + +00:13:42.700 --> 00:13:48.679 +Then we can create a reading list entry from it. + +00:13:48.680 --> 00:13:51.999 +Here, my reading list prompts me + +00:13:52.000 --> 00:13:53.418 +for a priority for this. + +00:13:53.419 --> 00:13:57.239 +How urgent it is for me to read this. + +00:13:57.240 --> 00:14:02.219 +It adds a TO-READ, which is a todo entry, + +00:14:02.220 --> 00:14:04.479 +which helps with organizing my reading list, + +00:14:04.480 --> 00:14:08.679 +because as you may also be able to see, this has an ID, + +00:14:08.680 --> 00:14:11.579 +because this is an org-roam node, + +00:14:11.580 --> 00:14:16.839 +so the TO-READ allows me to organize it inside org-roam. + +00:14:16.840 --> 00:14:21.839 +It saves the citekey, the link to the paper, + +00:14:21.840 --> 00:14:25.979 +and also tags it with the parent file node + +00:14:25.980 --> 00:14:30.379 +because I don't like having orphan nodes in my Zettelkasten. + +00:14:30.380 --> 00:14:33.839 +I like everything to be linked to at least one thing. + +00:14:33.840 --> 00:14:35.799 +So everything in the reading list + +00:14:35.800 --> 00:14:39.299 +is linked to the parent file. + +00:14:39.300 --> 00:14:47.519 +And now I can find this in the org-roam-node-find menu, + +00:14:47.520 --> 00:14:53.719 +here. However, that's not very interesting. + +00:14:53.720 --> 00:14:56.239 +In practice, my typical org-roam-node-find + +00:14:56.240 --> 00:14:59.759 +does not even include these reading list files + +00:14:59.760 --> 00:15:02.439 +because I don't really care to have them there. + +NOTE Special org-roam-node-find + +00:15:02.440 --> 00:15:06.159 +I have a special org-roam-node-find + +00:15:06.160 --> 00:15:13.439 +that is designed to find these in particular. + +00:15:13.440 --> 00:15:16.459 +And here these have 22. These are the amount of files + +00:15:16.460 --> 00:15:21.679 +that are currently in my reading list. + +00:15:21.680 --> 00:15:29.899 +So for example, let's try and press here. + +00:15:29.900 --> 00:15:33.479 +And magically, this prompts me to select a capture template, + +00:15:33.480 --> 00:15:39.119 +because what it's doing is that I selected this, + +00:15:39.120 --> 00:15:42.119 +and because of the citekey, it knows + +00:15:42.120 --> 00:15:46.539 +that it wants to create a new node for that. + +00:15:46.540 --> 00:15:47.959 +So I select the capture template. + +00:15:47.960 --> 00:15:56.099 +It knows that it wants to create this new node for this. + +00:15:56.100 --> 00:16:01.359 +And now, if I for a second close the reading list, + +00:16:01.360 --> 00:16:07.119 +now I can already go ahead and take notes on this. + +00:16:07.120 --> 00:16:09.759 +This is org-noter, in particular, + +00:16:09.760 --> 00:16:11.599 +and it makes it all very easy + +00:16:11.600 --> 00:16:15.719 +because it's all integrated in one place. + +00:16:15.720 --> 00:16:22.539 +If I then close this and open a new Emacs, + +00:16:22.540 --> 00:16:27.939 +we have this, and the reading list allows me to very quickly + +00:16:27.940 --> 00:16:32.539 +go from this being reading this item to initializing it. + +00:16:32.540 --> 00:16:35.039 +Another thing that is very useful is that + +00:16:35.040 --> 00:16:38.739 +everything is sorted by priority. + +00:16:38.740 --> 00:16:41.359 +So I need to increase the font size again + +00:16:41.360 --> 00:16:48.899 +because I closed the previous Emacs. + +00:16:48.900 --> 00:16:52.319 +So here, I can select what is high priority, + +00:16:52.320 --> 00:16:56.399 +what is low priority. I can also change the priority + +00:16:56.400 --> 00:16:58.799 +without needing to be in this file. + +00:16:58.800 --> 00:17:05.479 +Let's leave this file. I can say, okay, I decided + +00:17:05.480 --> 00:17:08.639 +that this file is priority B. It needs to be A, + +00:17:08.640 --> 00:17:11.899 +which is more urgent. + +00:17:11.900 --> 00:17:16.079 +In my system, there's five different priority levels. + +00:17:16.080 --> 00:17:17.999 +You can get away with less + +00:17:18.000 --> 00:17:20.959 +but I like to have the very much low ones + +00:17:20.960 --> 00:17:22.999 +as this is not urgent at all + +00:17:23.000 --> 00:17:27.159 +but I want to keep it somewhere. A is very urgent + +00:17:27.160 --> 00:17:31.779 +and B is urgent but just below A. + +00:17:31.780 --> 00:17:34.619 +And then the C in the middle is just + +00:17:34.620 --> 00:17:35.999 +I will eventually read this + +00:17:36.000 --> 00:17:41.919 +but not something I want to focus my attention on right now. + +00:17:41.920 --> 00:17:45.439 +So this is mostly about reading list. + +00:17:45.440 --> 00:17:46.639 +I can also show, for example, + +00:17:46.640 --> 00:17:50.679 +I have this if I finalize something, + +00:17:50.680 --> 00:17:56.799 +if I read it. For example, I created a note for this new thing. + +00:17:56.800 --> 00:17:59.079 +Let's say I finished reading it. + +00:17:59.080 --> 00:18:02.179 +I want to remove from my reading list. + +00:18:02.180 --> 00:18:05.639 +It's also just one command and it's done. + +NOTE Annotations + +00:18:05.640 --> 00:18:08.739 +If we now return to the presentation, + +00:18:08.740 --> 00:18:11.699 +the last thing I want to show is annotations. + +00:18:11.700 --> 00:18:17.119 +So for annotations, it's the idea that sometimes + +00:18:17.120 --> 00:18:23.139 +you just need to find something in Ebib quickly. + +00:18:23.140 --> 00:18:26.679 +So I'm here and I'm looking for something. + +00:18:26.680 --> 00:18:30.999 +I said I'm here and I'm looking for something. + +00:18:31.000 --> 00:18:35.199 +And as you can see, there's the annote file everywhere, + +00:18:35.200 --> 00:18:40.839 +the annote entry, which is a very very small description + +00:18:40.840 --> 00:18:43.799 +of things that I want to remember for this paper. + +00:18:43.800 --> 00:18:48.039 +So I can be scrolling here or scrolling + +00:18:48.040 --> 00:18:51.811 +in one of the smaller files + +00:18:51.812 --> 00:18:56.859 +and saying this was in this subsection, + +00:18:56.860 --> 00:18:59.519 +and which paper was it, and I can scroll, + +00:18:59.520 --> 00:19:01.711 +read all these annotes. + +00:19:01.712 --> 00:19:04.919 +Each annote takes like 15 seconds to read, + +00:19:04.920 --> 00:19:07.359 +and really decide, okay, + +00:19:07.360 --> 00:19:09.799 +it was this paper that I wanted, good. + +00:19:09.800 --> 00:19:21.819 +Now I can open the note for it, go there, and it's very easy. + +NOTE Wrapping up + +00:19:21.820 --> 00:19:25.719 +So I think that's all. I would like to thank you for your time. + +00:19:25.720 --> 00:19:29.319 +I would love to see your questions either in IRC, + +00:19:29.320 --> 00:19:32.199 +I will be, maybe I've already answered + +00:19:32.200 --> 00:19:35.039 +some of your questions there in the Etherpad, + +00:19:35.040 --> 00:19:37.599 +or right now, we're going to the live Q&A + +00:19:37.600 --> 00:19:42.159 +where I'd love to interact with everyone and have a discussion. + +00:19:42.160 --> 00:19:44.479 +However, if you don't have any questions right now, + +00:19:44.480 --> 00:19:48.559 +but you have a question later on, feel free to send me an email. + +00:19:48.560 --> 00:19:51.739 +My mail is also on the site. + +00:19:51.740 --> 00:19:54.599 +And if you're curious how all this "magic" worked, + +00:19:54.600 --> 00:20:00.839 +feel free to go to my Github and see the ebib section + +00:20:00.840 --> 00:20:04.039 +here that will also be linked in the doc page, + +00:20:04.040 --> 00:20:06.279 +where you can see all the configuration + +00:20:06.280 --> 00:20:10.919 +that I have done in Ebib for everything to work. + +00:20:10.920 --> 00:20:14.000 +Thank you again and have a wonderful EmacsConf! diff --git a/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main--chapters.vtt b/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main--chapters.vtt new file mode 100644 index 00000000..d266d618 --- /dev/null +++ b/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main--chapters.vtt @@ -0,0 +1,29 @@ +WEBVTT + + +00:02:07.200 --> 00:04:24.759 +The scope of the project + +00:04:24.760 --> 00:05:49.719 +Difference with Robin Templeton's project (Guile-Emacs) + +00:05:49.720 --> 00:07:28.039 +Progress made since last year + +00:07:28.040 --> 00:09:06.039 +Portable React-like GUI + +00:09:06.040 --> 00:11:48.699 +Demo + +00:11:48.700 --> 00:14:12.019 +Additional changes + +00:14:12.020 --> 00:17:06.199 +Other Scheme implementations + +00:17:06.200 --> 00:21:51.519 +GUI framework + +00:21:51.520 --> 00:23:14.113 +Wrapping up diff --git a/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main.vtt b/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main.vtt new file mode 100644 index 00000000..ed301c1c --- /dev/null +++ b/2025/captions/emacsconf-2025-schemacs--one-year-progress-update-schemacs-formerly-gypsum--ramin-honary--main.vtt @@ -0,0 +1,1183 @@ +WEBVTT captioned by sachac
+
+00:00:01.060 --> 00:00:05.639
+Hi, EmacsConf 2025. My name is Ramin Honary,
+
+00:00:05.640 --> 00:00:07.559
+and I'd like to talk to you today
+
+00:00:07.560 --> 00:00:10.399
+about my project called Schemacs
+
+00:00:10.400 --> 00:00:12.079
+which I presented last year.
+
+00:00:12.080 --> 00:00:13.879
+Back then it was called "Gypsum"
+
+00:00:13.880 --> 00:00:18.319
+and the name has since changed.
+
+00:00:18.320 --> 00:00:20.239
+So my name is Ramin Honary.
+
+00:00:20.240 --> 00:00:24.999
+I'm an Emacs enthusiast, have been since 2017 or so.
+
+00:00:25.000 --> 00:00:27.759
+I'm a full stack software developer.
+
+00:00:27.760 --> 00:00:29.300
+I love Haskell, Scheme,
+
+00:00:29.301 --> 00:00:31.433
+anything functional programming related,
+
+00:00:31.434 --> 00:00:32.959
+and of course Emacs.
+
+00:00:32.960 --> 00:00:36.199
+I started learning Scheme about three years ago,
+
+00:00:36.200 --> 00:00:37.999
+and this is my third time presenting
+
+00:00:38.000 --> 00:00:40.799
+at EmacsConf.
+
+00:00:40.800 --> 00:00:46.479
+So the Schemacs project that I'm talking to,
+
+00:00:46.480 --> 00:00:48.159
+I'm talking about to you today,
+
+00:00:48.160 --> 00:00:50.279
+was originally called "Gypsum".
+
+00:00:50.280 --> 00:00:53.359
+The reason I did not call it "Schemacs"
+
+00:00:53.360 --> 00:00:58.119
+was that the name "Schemacs" was taken on GitHub.
+
+00:00:58.120 --> 00:00:59.567
+But in the past year,
+
+00:00:59.568 --> 00:01:02.119
+I was able to get the permission
+
+00:01:02.120 --> 00:01:04.479
+of the author of GitHub's Schemacs
+
+00:01:04.480 --> 00:01:08.840
+project to name my project the same thing,
+
+00:01:08.841 --> 00:01:11.320
+even though it's a very similar project.
+
+00:01:11.321 --> 00:01:14.719
+So I changed the name.
+
+00:01:14.720 --> 00:01:19.700
+Let me see if I can quickly show the screen.
+
+00:01:19.701 --> 00:01:24.959
+So yeah, I have archived the old project.
+
+00:01:24.960 --> 00:01:27.719
+It's still there, but there's an explanation in
+
+00:01:27.720 --> 00:01:30.167
+the readme file and a screen grab
+
+00:01:30.168 --> 00:01:31.599
+of the conversation I had
+
+00:01:31.600 --> 00:01:36.119
+with the original author of GitHub Schemacs.
+
+00:01:36.120 --> 00:01:38.679
+My Schemacs is not on GitHub at all.
+
+00:01:38.680 --> 00:01:43.879
+It's only on Codeberg. So please don't get confused.
+
+00:01:43.880 --> 00:01:45.999
+But yes, I received permission
+
+00:01:46.000 --> 00:01:50.199
+to change the name and so I did.
+
+00:01:50.200 --> 00:01:52.839
+And I would like to give a quick shout-out to
+
+00:01:52.840 --> 00:01:57.239
+user "Tusharhero" for helping me with that.
+
+00:01:57.240 --> 00:02:02.639
+This person really helped make that name change happen.
+
+00:02:02.640 --> 00:02:07.199
+So back to the slides.
+
+NOTE The scope of the project
+
+00:02:07.200 --> 00:02:11.319
+And so now I'd like to clarify the scope of the project.
+
+00:02:11.320 --> 00:02:13.000
+I don't think I quite made it clear
+
+00:02:13.001 --> 00:02:14.919
+well enough last year,
+
+00:02:14.920 --> 00:02:18.400
+but... Although I'm definitely cloning
+
+00:02:18.401 --> 00:02:20.167
+the Emacs Lisp programming language,
+
+00:02:20.168 --> 00:02:25.267
+the actual scope of the Schemacs project
+
+00:02:25.268 --> 00:02:29.759
+is to make an Emacs-like app platform for Scheme.
+
+00:02:29.760 --> 00:02:32.959
+I don't consider Emacs to be a text editor.
+
+00:02:32.960 --> 00:02:38.199
+I consider Emacs to be a Lisp app platform.
+
+00:02:38.200 --> 00:02:39.839
+So it's similar to something like the
+
+00:02:39.840 --> 00:02:48.539
+World Wide Web, or Microsoft's .NET app platform, or Java.
+
+00:02:48.540 --> 00:02:52.559
+These are all examples of app platforms.
+
+00:02:52.560 --> 00:02:55.119
+I would like Schemacs to make it easy
+
+00:02:55.120 --> 00:02:59.399
+for not only people to use it for
+
+00:02:59.400 --> 00:03:03.479
+things like editing text or, you know, for
+
+00:03:03.480 --> 00:03:06.439
+using your computer through a command line
+
+00:03:06.440 --> 00:03:10.599
+or manipulating your Git repository.
+
+00:03:10.600 --> 00:03:13.159
+I'd also like you to be able to create simple
+
+00:03:13.160 --> 00:03:16.119
+GUIs or TUIs using Scheme.
+
+00:03:16.120 --> 00:03:19.319
+So that's also one of the goals of this project.
+
+00:03:19.320 --> 00:03:23.079
+It will of course have an Emacs-like text editor,
+
+00:03:23.080 --> 00:03:24.999
+and I will clone Emacs Lisp.
+
+00:03:25.000 --> 00:03:29.879
+So hopefully GNU Emacs users
+
+00:03:29.880 --> 00:03:32.779
+will feel comfortable moving over to Schemacs
+
+00:03:32.780 --> 00:03:35.679
+because they'll be able to use your init.
+
+00:03:35.680 --> 00:03:40.399
+You'll be able to use your init.el file.
+
+00:03:40.400 --> 00:03:43.239
+So configuring and scripting Schemacs
+
+00:03:43.240 --> 00:03:44.479
+should be done in Scheme.
+
+00:03:44.480 --> 00:03:47.679
+I'd like to encourage scripting in Scheme
+
+00:03:47.680 --> 00:03:51.199
+and creating new workflows and macros in Scheme.
+
+00:03:51.200 --> 00:03:54.267
+It will support Emacs Lisp depending on
+
+00:03:54.268 --> 00:03:59.319
+how much of the Emacs Lisp interpreter I can clone.
+
+00:03:59.320 --> 00:04:03.039
+That will be supported but not encouraged.
+
+00:04:03.040 --> 00:04:06.319
+But you should still be able to run your init.el.
+
+00:04:06.320 --> 00:04:08.839
+And I would like it to be good enough,
+
+00:04:08.840 --> 00:04:09.999
+this Emacs Lisp interpreter
+
+00:04:10.000 --> 00:04:12.999
+should be good enough to run packages from ELPA.
+
+00:04:13.000 --> 00:04:15.879
+Although it will probably be some time
+
+00:04:15.880 --> 00:04:17.559
+before it will be able to run
+
+00:04:17.560 --> 00:04:24.759
+something as large as Org Mode or Magit.
+
+NOTE Difference with Robin Templeton's project (Guile-Emacs)
+
+00:04:24.760 --> 00:04:27.439
+It is slightly different from the Guile-Emacs project.
+
+00:04:27.440 --> 00:04:30.333
+This is the work of Robin Templeton
+
+00:04:30.334 --> 00:04:32.219
+who presented last year.
+
+00:04:32.220 --> 00:04:36.033
+Guile-Emacs links the Guile runtime
+
+00:04:36.034 --> 00:04:38.500
+into the Emacs executable.
+
+00:04:38.501 --> 00:04:41.580
+It's not a Scheme application. Emacs,
+
+00:04:41.581 --> 00:04:44.200
+the core of Emacs is written in C.
+
+00:04:44.201 --> 00:04:48.120
+Guile, the core of Guile is written in C.
+
+00:04:48.121 --> 00:04:53.700
+What Robin Templeton has done is, at the C level, linked
+
+00:04:53.701 --> 00:04:56.833
+"libguile.so" into Emacs and then provided
+
+00:04:56.834 --> 00:04:59.500
+a programming layer where you can
+
+00:04:59.501 --> 00:05:04.759
+call the Scheme interpreter from Emacs Lisp
+
+00:05:04.760 --> 00:05:11.279
+so that you can run Scheme programs from within Emacs
+
+00:05:11.280 --> 00:05:13.919
+without having to launch a separate process
+
+00:05:13.920 --> 00:05:18.039
+and communicate over a channel such as a socket.
+
+00:05:18.040 --> 00:05:19.839
+You won't need "SLIME" or anything.
+
+00:05:19.840 --> 00:05:23.419
+The Guile interpreter is just right there inside of Emacs.
+
+00:05:23.420 --> 00:05:25.999
+But my project is not like this at all.
+
+00:05:26.000 --> 00:05:28.879
+Schemacs is written completely from the ground up
+
+00:05:28.880 --> 00:05:34.999
+in R7RS-compliant Scheme. And because it's R7RS-compliant,
+
+00:05:35.000 --> 00:05:37.999
+it's not bound to any one particular Scheme implementation,
+
+00:05:38.000 --> 00:05:39.879
+although Guile is the reference implementation.
+
+00:05:39.880 --> 00:05:42.359
+One goal of this project is to be able to run
+
+00:05:42.360 --> 00:05:49.719
+Schemacs on any R7RS-compliant Scheme implementation.
+
+NOTE Progress made since last year
+
+00:05:49.720 --> 00:05:56.259
+The work that I've done this past year mostly is internal.
+
+00:05:56.260 --> 00:06:00.939
+There's not much that you can see on the surface.
+
+00:06:00.940 --> 00:06:04.519
+But the most... One of the most important things that I
+
+00:06:04.520 --> 00:06:06.839
+did was I rewrote the parser in R7RS Scheme,
+
+00:06:06.840 --> 00:06:07.919
+so it no longer depends on
+
+00:06:07.920 --> 00:06:11.999
+the Guile regular expressions library.
+
+00:06:12.000 --> 00:06:14.959
+The parser now also provides source locations,
+
+00:06:14.960 --> 00:06:18.939
+so if an error occurs in Emacs Lisp,
+
+00:06:18.940 --> 00:06:20.567
+there will be a stack trace
+
+00:06:20.568 --> 00:06:23.633
+and it will show you where in the source code
+
+00:06:23.634 --> 00:06:28.319
+the error occured. This was not possible last year.
+
+00:06:28.320 --> 00:06:30.860
+And because it no longer depends on Guile,
+
+00:06:30.861 --> 00:06:34.520
+I can make it work on multiple Scheme implementations.
+
+00:06:34.521 --> 00:06:36.820
+So far, I've been able to get it to run on
+
+00:06:36.821 --> 00:06:38.920
+the Chibi Scheme interpreter
+
+00:06:38.921 --> 00:06:41.280
+and the Gauche Scheme interpreter, as well as
+
+00:06:41.281 --> 00:06:44.279
+Guile, which is the reference implementation.
+
+00:06:44.280 --> 00:06:48.559
+For a short time, it did work also on Chez Scheme,
+
+00:06:48.560 --> 00:06:53.179
+the Chez Scheme compiler, using Gwen Weinholt's "Akku,"
+
+00:06:53.180 --> 00:06:59.299
+which is a program that translates R7RS Scheme to R6RS Scheme.
+
+00:06:59.300 --> 00:07:04.519
+And with that translation, because Chez Scheme
+
+00:07:04.520 --> 00:07:07.319
+is pretty strictly an R6RS compiler,
+
+00:07:07.320 --> 00:07:11.519
+the translation allows you to run R7RS programs.
+
+00:07:11.520 --> 00:07:15.219
+But due to some change, I'm not sure where,
+
+00:07:15.220 --> 00:07:17.119
+it may have been changed in the Schemacs source code,
+
+00:07:17.120 --> 00:07:19.639
+or it may have been a change to Akku,
+
+00:07:19.640 --> 00:07:21.239
+but it no longer builds on Chez.
+
+00:07:21.240 --> 00:07:28.039
+It did at one point. I'd like to try to fix that.
+
+NOTE Portable React-like GUI
+
+00:07:28.040 --> 00:07:30.719
+The second most important thing that I've worked on is a
+
+00:07:30.720 --> 00:07:36.439
+portable React-like GUI. And so React,
+
+00:07:36.440 --> 00:07:40.999
+for anyone who has done web programming,
+
+00:07:41.000 --> 00:07:46.839
+is a very popular framework for programming web applications.
+
+00:07:46.840 --> 00:07:48.233
+And I've provided something
+
+00:07:48.234 --> 00:07:49.599
+very similar to that in Scheme now.
+
+00:07:49.600 --> 00:07:54.679
+So it works. I have constructed a DOM data structure
+
+00:07:54.680 --> 00:07:59.079
+in Scheme. It's just an ordinary Scheme data structure.
+
+00:07:59.080 --> 00:08:01.519
+It works like the web's "Document Object Model"
+
+00:08:01.520 --> 00:08:03.960
+or the "DOM" data structure.
+
+00:08:03.961 --> 00:08:09.999
+And then this Scheme DOM data structure can be rendered
+
+00:08:10.000 --> 00:08:13.059
+using any GUI framework that is convenient
+
+00:08:13.060 --> 00:08:16.239
+for the Scheme implementation that you're targeting.
+
+00:08:16.240 --> 00:08:18.879
+And you should be able to implement
+
+00:08:18.880 --> 00:08:22.919
+also rendering to a CLI as well.
+
+00:08:22.920 --> 00:08:24.600
+The current reference implementation
+
+00:08:24.601 --> 00:08:27.759
+is using a framework called Guile-GI.
+
+00:08:27.760 --> 00:08:30.639
+This is the "GObject Introspection" framework.
+
+00:08:30.640 --> 00:08:31.967
+It's a very simple
+
+00:08:31.968 --> 00:08:36.119
+GObject Introspection framework for Guile,
+
+00:08:36.120 --> 00:08:40.979
+and it binds to GTK3 on Linux.
+
+00:08:40.980 --> 00:08:42.919
+There's a similar framework called G-Golf
+
+00:08:42.920 --> 00:08:48.359
+which I'd like to begin using as well, also for Guile.
+
+00:08:48.360 --> 00:08:53.739
+G-Golf seems to be a bit more well-maintained, a bit...
+
+00:08:53.740 --> 00:08:57.799
+It has better features. G-Golf may be a
+
+00:08:57.800 --> 00:09:00.039
+better rendering backend for the reference
+
+00:09:00.040 --> 00:09:06.039
+implementation, but I would like to provide both.
+
+NOTE Demo
+
+00:09:06.040 --> 00:09:07.933
+I will give a demo of this now.
+
+00:09:07.934 --> 00:09:12.999
+Unfortunately not a whole lot
+
+00:09:13.000 --> 00:09:18.139
+to see compared to last year.
+
+00:09:18.140 --> 00:09:20.639
+First thing I'd like to show is
+
+00:09:20.640 --> 00:09:24.279
+that I now have a Makefile. You can look inside
+
+00:09:24.280 --> 00:09:28.400
+this Makefile and if you're able to read a Makefile,
+
+00:09:28.401 --> 00:09:31.967
+you can see that I have several targets now available.
+
+00:09:31.968 --> 00:09:35.000
+You can build Schemacs for Guile,
+
+00:09:35.001 --> 00:09:36.667
+you can build Schemacs for Gambit,
+
+00:09:36.668 --> 00:09:40.333
+or Stklos, or Chicken, or Chez,
+
+00:09:40.334 --> 00:09:42.900
+although none of these (except for Guile)
+
+00:09:42.901 --> 00:09:51.167
+currently works. These targets will actually
+
+00:09:51.168 --> 00:09:56.000
+build the source code, but then you would have to
+
+00:09:56.001 --> 00:09:59.433
+load it into the REPL separately.
+
+00:09:59.434 --> 00:10:02.467
+There are targets for launching
+
+00:10:02.468 --> 00:10:06.467
+a Gauche REPL and a Chibi REPL.
+
+00:10:06.468 --> 00:10:10.867
+You can also run the Emacs Lisp tests
+
+00:10:10.868 --> 00:10:13.067
+in Gauche and Chibi.
+
+00:10:13.068 --> 00:10:19.079
+You can also start a Guile REPL through this Makefile.
+
+00:10:19.080 --> 00:10:27.499
+So I will do that right now in the shell.
+
+00:10:27.500 --> 00:10:31.833
+(...make the text larger...there we go...)
+
+00:10:31.834 --> 00:10:38.479
+OK, so we have this directory of the source code.
+
+00:10:38.480 --> 00:10:44.559
+Let's just begin by running "guile.sh".
+
+00:10:44.560 --> 00:10:52.600
+This will launch a REPL and you can load "main-guile".
+
+00:10:52.601 --> 00:10:59.719
+This will launch the GUI. This is the basic
+
+00:10:59.720 --> 00:11:04.800
+proof of concept GUI that uses Guile-GI.
+
+00:11:04.801 --> 00:11:10.519
+So it may be hard to see.
+
+00:11:10.520 --> 00:11:14.559
+I cannot change the size of the text yet.
+
+00:11:14.560 --> 00:11:17.333
+I've implemented the M-: feature
+
+00:11:17.334 --> 00:11:24.067
+where you can eval in a minibuffer some Scheme code.
+
+00:11:24.068 --> 00:11:34.439
+(string-append "hello" ...)
+
+00:11:34.440 --> 00:11:38.279
+It outputs the result in the buffer.
+
+00:11:38.280 --> 00:11:40.959
+This is basically the "*Messages*" buffer.
+
+00:11:40.960 --> 00:11:42.619
+And that's all the more that I have.
+
+00:11:42.620 --> 00:11:45.479
+This is the same state it was in last year.
+
+00:11:45.480 --> 00:11:48.699
+It hasn't changed a whole lot since back then.
+
+NOTE Additional changes
+
+00:11:48.700 --> 00:11:52.819
+But I have made additional changes.
+
+00:11:52.820 --> 00:12:05.379
+So first of all, you can run
+
+00:12:05.380 --> 00:12:10.200
+(let me just go back into the Guile)... you can
+
+00:12:10.201 --> 00:12:16.619
+run the Emacs Lisp interpreter tests, so "elisp-tests".
+
+00:12:16.620 --> 00:12:18.919
+As you can see, it gives you a stack trace.
+
+00:12:18.920 --> 00:12:22.039
+So this is an error that I've been able to reproduce.
+
+00:12:22.040 --> 00:12:24.599
+I know exactly what the cause of this error is.
+
+00:12:24.600 --> 00:12:27.599
+It is not finding a variable
+
+00:12:27.600 --> 00:12:30.319
+because the closure is not correctly
+
+00:12:30.320 --> 00:12:35.199
+capturing its environment. So there should be a variable
+
+00:12:35.200 --> 00:12:38.719
+in the closure, but that variable has not been captured
+
+00:12:38.720 --> 00:12:43.459
+and so it is causing an error.
+
+00:12:43.460 --> 00:12:48.080
+It is currently loading "byte-run.el".
+
+00:12:48.081 --> 00:12:49.820
+Let me show you what code that is here.
+
+00:12:49.821 --> 00:12:53.500
+So I've copied into the source repository
+
+00:12:53.501 --> 00:12:58.760
+for Schemacs some of the Elisp code from GNU Emacs.
+
+00:12:58.761 --> 00:13:04.420
+So I have this "subr.el".
+
+00:13:04.421 --> 00:13:09.860
+This declares most of the core of Emacs Lisp
+
+00:13:09.861 --> 00:13:12.860
+that's not written in C.
+
+00:13:12.861 --> 00:13:18.999
+There's also "byte-run.el". Schemacs Emacs Lisp can now
+
+00:13:19.000 --> 00:13:24.379
+evaluate this. This is where functions like "defun" are
+
+00:13:24.380 --> 00:13:28.359
+defined, and "defmacro". So as you can see,
+
+00:13:28.360 --> 00:13:30.799
+defun itself is a defmacro defined right here.
+
+00:13:30.800 --> 00:13:34.859
+It's written in Emacs Lisp itself,
+
+00:13:34.860 --> 00:13:37.719
+defined in terms of defalias.
+
+00:13:37.720 --> 00:13:40.239
+So I can evaluate "byte-run",
+
+00:13:40.240 --> 00:13:42.739
+I can evaluate "macroexp",
+
+00:13:42.740 --> 00:13:46.019
+and the failure occurs somewhere in "subr.el".
+
+00:13:46.020 --> 00:13:48.959
+Although if you look at the stack trace,
+
+00:13:48.960 --> 00:13:51.159
+it doesn't provide all the necessary information.
+
+00:13:51.160 --> 00:13:56.439
+So it appears to be happening in byte-run.el.
+
+00:13:56.440 --> 00:14:00.619
+Really, it's an error that's occurring inside of a macro,
+
+00:14:00.620 --> 00:14:05.799
+and the macro call site is somewhere in subr.el.
+
+00:14:05.800 --> 00:14:08.639
+Anyway, take note of this stack trace.
+
+00:14:08.640 --> 00:14:12.019
+This was run from within Guile.
+
+NOTE Other Scheme implementations
+
+00:14:12.020 --> 00:14:14.199
+Now what I've done this past year
+
+00:14:14.200 --> 00:14:19.479
+is make it work on other Scheme implementations.
+
+00:14:19.480 --> 00:14:23.833
+Use "make" to launch a Gauche REPL.
+
+00:14:23.834 --> 00:14:25.999
+Now I'm inside of Gauche.
+
+00:14:26.000 --> 00:14:27.233
+This is the command that
+
+00:14:27.234 --> 00:14:30.079
+you would use to launch a Gauche REPL.
+
+00:14:30.080 --> 00:14:38.199
+And I can load the same program (load "elisp-tests.scm").
+
+00:14:38.200 --> 00:14:42.759
+You get the exact same result as Guile.
+
+00:14:42.760 --> 00:14:45.799
+So we have two different Scheme implementations
+
+00:14:45.800 --> 00:14:46.799
+producing the same result.
+
+00:14:46.800 --> 00:14:53.079
+Let's try "make" a Chibi REPL. This is Chibi Scheme.
+
+00:14:53.080 --> 00:15:00.219
+And you can (load "elisp-tests.scm").
+
+00:15:00.220 --> 00:15:04.080
+Chibi is a bit slower, but you get the exact same result.
+
+00:15:04.081 --> 00:15:07.400
+So we have three different Scheme implementations
+
+00:15:07.401 --> 00:15:11.539
+all running Emacs Lisp,
+
+00:15:11.540 --> 00:15:14.039
+and all producing the same result.
+
+00:15:14.040 --> 00:15:14.767
+I think that's...
+
+00:15:14.768 --> 00:15:17.099
+I'm fairly proud of that accomplishment.
+
+00:15:17.100 --> 00:15:21.200
+I was able to get the code written to the point
+
+00:15:21.201 --> 00:15:24.879
+where it actually runs on multiple implementations.
+
+00:15:24.880 --> 00:15:30.599
+You can also try making it for other Scheme compilers
+
+00:15:30.600 --> 00:15:35.959
+like "schemacs-mitscheme" for example,
+
+00:15:35.960 --> 00:15:40.019
+but this will fail.
+
+00:15:40.020 --> 00:15:46.679
+You can try building it for "schemacs-chez",
+
+00:15:46.680 --> 00:15:51.800
+Let's try Chez... there we go.
+
+00:15:51.801 --> 00:15:54.233
+And it will use Akku,
+
+00:15:54.234 --> 00:15:58.000
+and it will fetch the necessary dependencies.
+
+00:15:58.001 --> 00:16:03.433
+But it fails, and I haven't been able to
+
+00:16:03.434 --> 00:16:05.959
+debug that quite yet.
+
+00:16:05.960 --> 00:16:13.139
+Stklos fails for a similar reason.
+
+00:16:13.140 --> 00:16:21.699
+Gambit... Chicken still doesn't build all the way yet.
+
+00:16:21.700 --> 00:16:27.120
+The Makefile at least has places for it.
+
+00:16:27.121 --> 00:16:30.967
+If anyone can help me out and get Schemacs to compile
+
+00:16:30.968 --> 00:16:32.733
+on these other Scheme implementations,
+
+00:16:32.734 --> 00:16:34.000
+I'd appreciate it.
+
+00:16:34.001 --> 00:16:35.467
+I can probably figure it out myself,
+
+00:16:35.468 --> 00:16:37.000
+but that will take more time.
+
+00:16:37.001 --> 00:16:40.120
+And let me just show you quickly
+
+00:16:40.121 --> 00:16:41.720
+the test program.
+
+00:16:41.721 --> 00:16:44.500
+Basically this is the Emacs Lisp test program
+
+00:16:44.501 --> 00:16:48.580
+that I was just running, and it produces an error.
+
+00:16:48.581 --> 00:16:53.220
+All it does is it loads these files here in this order,
+
+00:16:53.221 --> 00:16:55.180
+and it fails right around here.
+
+00:16:55.181 --> 00:16:58.319
+So it's able to load these two.
+
+00:16:58.320 --> 00:17:01.360
+And yeah, that's what I've accomplished
+
+00:17:01.361 --> 00:17:06.199
+on the Emacs Lisp side of things.
+
+NOTE GUI framework
+
+00:17:06.200 --> 00:17:09.119
+The next thing I want to show you is the GUI framework
+
+00:17:09.120 --> 00:17:13.199
+that I've written, which I'm fairly proud of so far.
+
+00:17:13.200 --> 00:17:15.833
+So this is the GUI framework
+
+00:17:15.834 --> 00:17:19.919
+(oops, I better launch it again, OK...)
+
+00:17:19.920 --> 00:17:22.600
+and let me show you the tests.
+
+00:17:22.601 --> 00:17:25.700
+So here in the tests, you can start to see
+
+00:17:25.701 --> 00:17:29.067
+some examples of how you use it.
+
+00:17:29.068 --> 00:17:31.833
+So here is a "counter" test, and this is kind of like
+
+00:17:31.834 --> 00:17:35.940
+the "hello world" of reactive programming frameworks,
+
+00:17:35.941 --> 00:17:38.000
+where you have a state variable,
+
+00:17:38.200 --> 00:17:41.659
+sometimes called an "observable."
+
+00:17:41.660 --> 00:17:43.439
+I'm calling it "number",
+
+00:17:43.440 --> 00:17:47.838
+and it uses "=" to check if the state has updated.
+
+00:17:49.000 --> 00:17:52.820
+If an update occurs and the new value is different
+
+00:17:52.821 --> 00:17:55.032
+from the old value according to the "=" function,
+
+00:17:55.033 --> 00:17:59.232
+then trigger a state update in the GUI as well.
+
+00:17:59.233 --> 00:18:03.099
+Initialize to 0, bound to "number".
+
+00:18:03.100 --> 00:18:06.132
+I have a "button" function which creates a simple button.
+
+00:18:06.133 --> 00:18:07.832
+It takes a label and an action.
+
+00:18:07.833 --> 00:18:10.865
+Right here you see the "div" command.
+
+00:18:10.866 --> 00:18:13.039
+This is what creates a "div".
+
+00:18:13.040 --> 00:18:18.999
+Using the properties, I describe that this div is a
+
+00:18:19.000 --> 00:18:22.599
+push-button and the "on-button-push" is an action.
+
+00:18:22.600 --> 00:18:25.265
+The action is to update the variable "number"
+
+00:18:25.266 --> 00:18:30.399
+using whatever function or lambda was provided to it.
+
+00:18:30.400 --> 00:18:31.965
+And then the content that you see on screen,
+
+00:18:31.966 --> 00:18:34.033
+that you will see on screen when it runs,
+
+00:18:34.034 --> 00:18:38.667
+is here. You create a "div-pack cut-vertical".
+
+00:18:38.668 --> 00:18:43.233
+You declare two buttons and then you declare
+
+00:18:43.234 --> 00:18:46.800
+this "use-vars" which will take the content
+
+00:18:46.801 --> 00:18:48.833
+of this variable here, this observable,
+
+00:18:48.834 --> 00:18:51.733
+and place it into the GUI next to the buttons here.
+
+00:18:51.734 --> 00:18:54.233
+So what you will see on screen is
+
+00:18:54.234 --> 00:18:57.067
+a "plus" button which increments,
+
+00:18:57.068 --> 00:18:59.320
+here is the "increment" function,
+
+00:18:59.321 --> 00:19:02.779
+a "minus" button which decrements,
+
+00:19:02.780 --> 00:19:05.479
+and then the content of the variable that is
+
+00:19:05.480 --> 00:19:09.699
+being incremented and decremented.
+
+00:19:09.700 --> 00:19:11.865
+The advantage of these reactive frameworks is that
+
+00:19:11.866 --> 00:19:13.065
+with very few lines of code
+
+00:19:13.066 --> 00:19:16.032
+you can create fairly complex interfaces.
+
+00:19:16.033 --> 00:19:18.599
+The less code you have to write,
+
+00:19:18.600 --> 00:19:21.232
+the fewer chances you have to make mistakes.
+
+00:19:21.233 --> 00:19:23.967
+So let's just run this program.
+
+00:19:23.968 --> 00:19:33.292
+This was the "counter-test."
+
+00:19:33.293 --> 00:19:35.199
+And that is the debug window. Here's the "counter."
+
+00:19:35.200 --> 00:19:38.452
+I'm sorry it's not much larger than this.
+
+00:19:38.453 --> 00:19:41.132
+But here's the "plus" button, the "minus" button,
+
+00:19:41.133 --> 00:19:43.919
+and here's the "number", 0.
+
+00:19:43.920 --> 00:19:49.792
+And I can increment or decrement as much as I like.
+
+00:19:49.793 --> 00:19:51.100
+So yeah, that's kind of the hello world
+
+00:19:51.101 --> 00:19:55.239
+of reactive programming. (I'll reboot the REPL...)
+
+00:19:55.240 --> 00:19:58.599
+The next thing I want to show you is this layout test.
+
+00:19:58.600 --> 00:20:02.192
+And I'll just run the test first.
+
+00:20:02.193 --> 00:20:05.019
+So here we have basically
+
+00:20:05.020 --> 00:20:08.539
+a tiling window manager kind of thing,
+
+00:20:08.540 --> 00:20:11.279
+where you can resize the tiles
+
+00:20:11.280 --> 00:20:12.933
+and then by clicking on these buttons here,
+
+00:20:12.934 --> 00:20:16.939
+you can change the layout.
+
+00:20:16.940 --> 00:20:20.167
+So you can do two on the right,
+
+00:20:20.168 --> 00:20:27.100
+two up above, or three up above. So, yeah.
+
+00:20:27.101 --> 00:20:30.267
+Those tiling windows, as you can see,
+
+00:20:30.268 --> 00:20:33.000
+once I work this branch into the main branch of Schemacs,
+
+00:20:33.001 --> 00:20:35.273
+I can use that to implement
+
+00:20:35.274 --> 00:20:38.265
+the split window functionality for Schemacs,
+
+00:20:38.266 --> 00:20:42.466
+the editor. So here's what this split...
+
+00:20:42.466 --> 00:20:44.865
+Here's the layout test that you just saw.
+
+00:20:44.866 --> 00:20:46.732
+Let me make it a bit smaller
+
+00:20:46.733 --> 00:20:48.199
+so that it all fits on one screen.
+
+00:20:48.200 --> 00:20:51.365
+So basically we have the "button" command again,
+
+00:20:51.366 --> 00:20:54.132
+and then these are the button actions
+
+00:20:54.133 --> 00:20:56.999
+which basically just changes the layout,
+
+00:20:57.000 --> 00:21:01.459
+and then I have the layout. So this layout is a "div."
+
+00:21:01.460 --> 00:21:04.919
+The first div just places three buttons in a row.
+
+00:21:04.920 --> 00:21:07.532
+The next layout is a div within a div.
+
+00:21:07.533 --> 00:21:12.999
+So we have one div which places the button
+
+00:21:13.000 --> 00:21:18.632
+called "two right" buttons, and the div above it,
+
+00:21:18.633 --> 00:21:21.492
+which places the "three in a row" button
+
+00:21:21.493 --> 00:21:22.660
+or the "two above" buttons.
+
+00:21:22.661 --> 00:21:26.000
+And here's the next... So there's three different
+
+00:21:26.001 --> 00:21:30.600
+layouts, and clicking on one of their associated buttons
+
+00:21:30.601 --> 00:21:32.100
+will just change the layout.
+
+00:21:32.101 --> 00:21:33.433
+As you can see, very little code
+
+00:21:33.434 --> 00:21:38.500
+to create a somewhat complex user interface.
+
+00:21:38.501 --> 00:21:42.867
+That's the advantage of using reactive or declarative
+
+00:21:42.868 --> 00:21:44.233
+UI programming paradigms.
+
+00:21:44.234 --> 00:21:47.232
+So yeah, this has not been merged into Schemacs
+
+00:21:47.233 --> 00:21:48.700
+at the time of this recording,
+
+00:21:48.701 --> 00:21:51.519
+but will be soon hopefully.
+
+NOTE Wrapping up
+
+00:21:51.520 --> 00:21:55.165
+So yeah, I think I've already gone on for 20 minutes.
+
+00:21:55.166 --> 00:21:59.139
+So I guess I'll just end my presentation here.
+
+00:21:59.140 --> 00:22:00.465
+I have lots more to talk about.
+
+00:22:00.466 --> 00:22:03.979
+I guess I will say one last thing before I go:
+
+00:22:03.980 --> 00:22:07.065
+that I would very much like for others
+
+00:22:07.066 --> 00:22:09.199
+to try and contribute to this project.
+
+00:22:09.200 --> 00:22:14.232
+I will do my best to try and help teach anybody
+
+00:22:14.233 --> 00:22:16.832
+or work with anybody, especially even
+
+00:22:16.833 --> 00:22:18.599
+if you don't have much experience with Scheme.
+
+00:22:18.600 --> 00:22:22.759
+I'd like to help everybody try to contribute.
+
+00:22:22.760 --> 00:22:26.239
+Basically I want to get this proof of concept working.
+
+00:22:26.240 --> 00:22:30.673
+I want to get a stable user interface up and running,
+
+00:22:30.674 --> 00:22:33.065
+and then we can start working on
+
+00:22:33.066 --> 00:22:36.699
+improving the Emacs Lisp interpreter all together.
+
+00:22:36.700 --> 00:22:41.065
+There are close to 1,400 built-in functions
+
+00:22:41.066 --> 00:22:43.659
+which need to be implemented.
+
+00:22:43.660 --> 00:22:44.965
+We don't need to get all of them
+
+00:22:44.966 --> 00:22:48.465
+in order to be able to run probably most of ELPA,
+
+00:22:48.466 --> 00:22:50.865
+but as much as possible.
+
+00:22:50.866 --> 00:22:54.799
+We would like to clone Emacs Lisp and I need help.
+
+00:22:54.800 --> 00:23:02.579
+So get a hold of me. My project is on Codeberg.
+
+00:23:02.580 --> 00:23:06.919
+Well, (oh, I can't show this here),
+
+00:23:06.920 --> 00:23:14.113
+but I will end it there. Thank you for listening.
diff --git a/2025/captions/emacsconf-2025-swanky--swanky-python-interactive-development-for-python--scott-zimmermann--main.vtt b/2025/captions/emacsconf-2025-swanky--swanky-python-interactive-development-for-python--scott-zimmermann--main.vtt new file mode 100644 index 00000000..35eb7ce7 --- /dev/null +++ b/2025/captions/emacsconf-2025-swanky--swanky-python-interactive-development-for-python--scott-zimmermann--main.vtt @@ -0,0 +1,1108 @@ +WEBVTT captioned by sachac + +00:00:00.880 --> 00:00:02.439 +Hello everyone, I'm Scott + +00:00:02.440 --> 00:00:04.239 +and I'll be talking about Swanky Python, + +00:00:04.240 --> 00:00:06.199 +which is a development environment for Python + +00:00:06.200 --> 00:00:08.319 +based on Emacs' Slime package. + +00:00:08.320 --> 00:00:11.679 +So what is that and why might you find it interesting? + +00:00:11.680 --> 00:00:15.279 +SLIME is the Superior Lisp Interaction Mode for Emacs. + +00:00:15.280 --> 00:00:17.999 +It's an Emacs package for developing Common Lisp, + +00:00:18.000 --> 00:00:20.679 +and it's a bit different from the way we develop most languages + +00:00:20.680 --> 00:00:22.599 +in that you're always connected + +00:00:22.600 --> 00:00:25.399 +to a running instance of your application, + +00:00:25.400 --> 00:00:27.959 +and you kind of build up your application, piece by piece, + +00:00:27.960 --> 00:00:30.399 +modifying one expression at a time + +00:00:30.400 --> 00:00:34.559 +without ever having to restart your application. + +00:00:34.560 --> 00:00:36.679 +So why might you want to develop this way? + +00:00:36.680 --> 00:00:40.039 +One advantage is that you can get a faster feedback loop. + +00:00:40.040 --> 00:00:42.599 +For some kinds of software, it doesn't make a big difference. + +00:00:42.600 --> 00:00:43.919 +Like, if you're developing a web backend + +00:00:43.920 --> 00:00:48.039 +where all state is stored externally in a database, + +00:00:48.040 --> 00:00:50.279 +then you can have a file watcher + +00:00:50.280 --> 00:00:52.799 +that just restarts the whole Python process + +00:00:52.800 --> 00:00:54.639 +whenever you make any edit, + +00:00:54.640 --> 00:00:56.159 +and you're not really losing anything, + +00:00:56.160 --> 00:00:59.679 +because all the state is stored outside the Python process + +00:00:59.680 --> 00:01:01.719 +in a database. So it works great. + +00:01:01.720 --> 00:01:03.559 +But for other kinds of software, like + +00:01:03.560 --> 00:01:05.559 +let's say you're developing an Emacs package + +00:01:05.560 --> 00:01:07.279 +or a video game, + +00:01:07.280 --> 00:01:10.319 +then it can be a real pain to restart the application + +00:01:10.320 --> 00:01:12.679 +and recreate the state it was in before + +00:01:12.680 --> 00:01:17.279 +just to test the effect of each edit you want to make. + +00:01:17.280 --> 00:01:21.359 +Another advantage is the runtime introspection you have available. + +00:01:21.360 --> 00:01:22.679 +So since you're always connected + +00:01:22.680 --> 00:01:24.999 +to a running instance of your application, + +00:01:25.000 --> 00:01:27.799 +you can inspect the values of variables, + +00:01:27.800 --> 00:01:30.959 +you can trace functions, and all sorts of other information + +00:01:30.960 --> 00:01:36.279 +to help you understand your application better. + +00:01:36.280 --> 00:01:39.919 +And lastly, it's just a lot of fun to develop this way, + +00:01:39.920 --> 00:01:43.519 +or at least I find it fun developing with SLIME, + +00:01:43.520 --> 00:01:45.759 +so I wrote a SLIME backend for Python + +00:01:45.760 --> 00:01:48.799 +so I could have more fun when I'm coding in Python. + +00:01:48.800 --> 00:01:52.599 +As for the name swanky-python, within SLIME, + +00:01:52.600 --> 00:01:56.279 +swank is the name of the Common Lisp backend + +00:01:56.280 --> 00:01:59.199 +that runs within your Common Lisp application + +00:01:59.200 --> 00:02:02.919 +and connects to Emacs. So I'm not too creative. + +00:02:02.920 --> 00:02:07.999 +swanky-python is just a swank implementation in Python. + +NOTE Demo + +00:02:08.000 --> 00:02:15.279 +So let's see it in action. So we started up with M-x slime. + +00:02:15.280 --> 00:02:19.639 +And what that does is it starts a Python process, + +00:02:19.640 --> 00:02:25.039 +starts swanky-python within it, and connects to it from Emacs. + +00:02:25.040 --> 00:02:29.039 +And you can configure how exactly it runs Python. + +00:02:29.040 --> 00:02:32.479 +Or you can start swanky python manually + +00:02:32.480 --> 00:02:35.119 +within a Python application running on a remote server + +00:02:35.120 --> 00:02:36.313 +and forward the port locally + +00:02:36.614 --> 00:02:40.919 +and connect to it in Emacs, from Emacs remotely. + +00:02:40.920 --> 00:02:43.239 +Within the README, there's more documentation + +00:02:43.240 --> 00:02:45.519 +on other ways to start it. + +00:02:45.520 --> 00:02:52.159 +But just M-x slime is the basic way that works most of the time. + +00:02:52.160 --> 00:02:55.759 +So within the REPL, the first thing you'll notice is that + +00:02:55.760 --> 00:02:58.839 +REPL outputs are clickable buttons, + +00:02:58.840 --> 00:03:02.119 +what SLIME calls presentations. + +00:03:02.120 --> 00:03:04.759 +So you can do things like inspect them. + +00:03:04.760 --> 00:03:09.759 +And for each presentation, in the Python backend, + +00:03:09.760 --> 00:03:12.479 +it holds on to the reference to the object. + +00:03:12.480 --> 00:03:14.559 +So for an int, it's not too interesting, + +00:03:14.560 --> 00:03:20.239 +but let's do a more complex object like a file. + +00:03:20.240 --> 00:03:22.519 +Then we can inspect the file. + +00:03:22.520 --> 00:03:26.599 +We can describe it, which will bring up documentation + +00:03:26.600 --> 00:03:33.759 +on that class. We can use it in further expressions + +00:03:33.760 --> 00:03:39.431 +like if we copy it, it will use the actual Python object + +00:03:39.432 --> 00:03:43.399 +in this expression. + +00:03:43.400 --> 00:03:48.319 +We can assign it to a variable. + +00:03:48.320 --> 00:03:50.999 +SLIME uses presentations everywhere + +00:03:51.000 --> 00:03:53.239 +that a Python object would be displayed. + +00:03:53.240 --> 00:03:56.559 +So instead of just their string representation, + +00:03:56.560 --> 00:04:00.239 +when you have a backtrace on an exception, + +00:04:00.240 --> 00:04:03.965 +or you... within the inspector or anywhere else really, + +00:04:03.966 --> 00:04:06.019 +anywhere that the string representation + +00:04:06.020 --> 00:04:07.940 +of an object would be displayed, + +00:04:07.941 --> 00:04:10.740 +it displays a presentation that you can go on to + +00:04:10.741 --> 00:04:14.960 +inspect, reuse, or send to the REPL and so on. + +00:04:14.961 --> 00:04:23.039 +One useful utility function is pp for print presentation. + +00:04:23.040 --> 00:04:25.119 +We haven't imported it yet. + +00:04:25.120 --> 00:04:29.159 +So when we get a name error exception + +00:04:29.160 --> 00:04:33.879 +and SLIME sees that that name is available for import somewhere, + +00:04:33.880 --> 00:04:38.279 +it'll give us the option of importing it. + +00:04:38.280 --> 00:04:40.599 +Since it's available for import from multiple modules, + +00:04:40.600 --> 00:04:43.919 +it'll prompt us for which one we want to import it from. + +00:04:43.920 --> 00:04:45.519 +We want to import it from swanky-python, + +00:04:45.520 --> 00:04:48.479 +not from the standard library. + +00:04:48.480 --> 00:04:52.599 +Then it will print a presentation of that object. + +00:04:52.600 --> 00:04:55.559 +Within the REPL, this is not really useful + +00:04:55.560 --> 00:04:58.919 +because all REPL outputs are already presentations. + +00:04:58.920 --> 00:05:02.799 +But I use this now whenever I would use print debugging, + +00:05:02.800 --> 00:05:05.639 +just whenever I would use insert print statements in my program + +00:05:05.640 --> 00:05:08.399 +to see what's going on, I have it print a presentation + +00:05:08.400 --> 00:05:11.199 +because that way I can go back and inspect it later, + +00:05:11.200 --> 00:05:16.599 +copy it to the REPL and further manipulate it and so on. + +NOTE Inspector + +00:05:16.600 --> 00:05:20.119 +Next up, let's look at the inspector more. + +00:05:20.120 --> 00:05:25.579 +If we go back and inspect the file object, + +00:05:25.580 --> 00:05:27.239 +you can write custom inspector views + +00:05:27.240 --> 00:05:28.839 +for different kinds of objects. + +00:05:28.840 --> 00:05:32.519 +So far, I just have a couple. One for sequences, + +00:05:32.520 --> 00:05:36.919 +one for mappings, and one for every other kind of object. + +00:05:36.920 --> 00:05:45.979 +Like if we inspect a mapping, there's a shortcut + +00:05:45.980 --> 00:05:48.639 +inspect last result, which is what I normally use + +00:05:48.640 --> 00:05:52.379 +to open the inspector. Then we see the values, + +00:05:52.380 --> 00:05:56.319 +and each value in the inspector is a presentation + +00:05:56.320 --> 00:05:58.419 +that we can go on to inspect, and so on. + +00:05:58.420 --> 00:06:03.979 +Let's go back to inspecting the file object. + +00:06:03.980 --> 00:06:06.039 +Again, we can inspect each of the values, + +00:06:06.040 --> 00:06:10.239 +we can copy them back to the REPL and so on. + +00:06:10.240 --> 00:06:13.839 +It just displays all the attributes for the class + +00:06:13.840 --> 00:06:15.399 +and their values. + +00:06:15.400 --> 00:06:18.119 +We can configure what attributes we want to show. + +00:06:18.120 --> 00:06:21.119 +There's a transient menu where we can toggle + +00:06:21.120 --> 00:06:23.359 +if we want to show private attributes, dunder attributes, + +00:06:23.360 --> 00:06:26.439 +doc strings, so on, or everything, + +00:06:26.440 --> 00:06:28.519 +which is a bit much to show by default. + +00:06:28.520 --> 00:06:33.719 +So we'll reset it to the default. + +00:06:33.720 --> 00:06:37.839 +In the future, I want to add graphical inspector views + +00:06:37.840 --> 00:06:40.679 +for different kinds of objects, and also support + +00:06:40.680 --> 00:06:42.999 +showing plots in both the inspector and the REPL, + +00:06:43.000 --> 00:06:47.719 +but that's future work I haven't started on yet. + +NOTE Evaluating Python + +00:06:47.720 --> 00:06:51.999 +Let's look at the different options for evaluating Python. + +00:06:52.000 --> 00:06:59.099 +So we can evaluate a whole file. + +00:06:59.100 --> 00:07:00.639 +We can evaluate just a class. + +00:07:00.640 --> 00:07:03.479 +We can evaluate just the method we're working on. + +00:07:03.480 --> 00:07:06.359 +We can evaluate a Python statement, + +00:07:06.360 --> 00:07:11.839 +and it will show the result in an overlay next to the cursor. + +00:07:11.840 --> 00:07:17.919 +We can select some code and just evaluate the highlighted region. + +00:07:17.920 --> 00:07:24.799 +We can sync the REPL to the active file. + +00:07:24.800 --> 00:07:27.319 +So now everything we evaluate in the REPL will be in the + +00:07:27.320 --> 00:07:29.639 +context of the eval_demo module. + +00:07:29.640 --> 00:07:35.399 +We can also set the module that the REPL is in. + +00:07:35.400 --> 00:07:38.279 +We can go back to main. + +00:07:38.280 --> 00:07:43.679 +But let's go back to the eval_demo module for now. + +NOTE Updating + +00:07:43.680 --> 00:07:49.799 +One useful thing is when you update a class or a function, + +00:07:49.800 --> 00:07:54.539 +it updates old instances of that class or function. + +00:07:54.540 --> 00:07:58.479 +So right now, f.bar is foobar. + +00:07:58.480 --> 00:08:03.719 +But if we edit that class, it will actually edit the code + +00:08:03.720 --> 00:08:05.239 +for the old instance of that class. + +00:08:05.240 --> 00:08:07.599 +And that's provided by code I copied + +00:08:07.600 --> 00:08:12.079 +from IPython's autoreload extension. + +00:08:12.080 --> 00:08:14.639 +It helps when you're trying to develop in Python + +00:08:14.640 --> 00:08:16.498 +without having to restart the Python process + +00:08:16.499 --> 00:08:20.039 +whenever you make a change. + +00:08:20.040 --> 00:08:22.599 +Auto reload in Python is a big topic + +00:08:22.600 --> 00:08:26.519 +that I don't really have time to go into here, + +00:08:26.520 --> 00:08:29.479 +but right now it is more limited + +00:08:29.480 --> 00:08:32.559 +than what is done in Common Lisp. + +00:08:32.560 --> 00:08:35.759 +Like for example, if you have a data class in Python + +00:08:35.760 --> 00:08:37.619 +and you add a new field to the data class, + +00:08:37.620 --> 00:08:41.039 +it won't automatically update old instances + +00:08:41.040 --> 00:08:43.399 +of the data class with a new field. + +00:08:43.400 --> 00:08:46.599 +So there's more that needs to be done with that, + +00:08:46.600 --> 00:08:50.359 +but I am perhaps naively optimistic + +00:08:50.360 --> 00:08:54.279 +that Python's runtime is quite dynamic and flexible, + +00:08:54.280 --> 00:08:59.799 +and that I can fully implement autoreload in Python, + +00:08:59.800 --> 00:09:02.119 +but there's still work to be done, + +00:09:02.120 --> 00:09:05.419 +and it's a big topic to go into. + +00:09:05.420 --> 00:09:08.959 +Next up, let's look at the backtrace buffer. + +00:09:08.960 --> 00:09:12.839 +But as it is right now, autoreload is actually useful. + +00:09:12.840 --> 00:09:16.959 +I mostly develop in Python without having to restart the process + +00:09:16.960 --> 00:09:19.599 +and without running into issues from old state + +00:09:19.600 --> 00:09:22.899 +that hasn't been updated properly. + +NOTE Backtraces + +00:09:22.900 --> 00:09:25.999 +So if we go on to look at the backtrace buffer, + +00:09:26.000 --> 00:09:32.819 +whenever we get an exception in Python... + +00:09:32.820 --> 00:09:37.079 +Let's go back to it. + +00:09:37.080 --> 00:09:41.419 +Whenever we get an exception, it will... + +00:09:41.420 --> 00:09:43.698 +let's change the code so that it actually + +00:09:43.699 --> 00:09:49.965 +gets an exception... + +00:09:49.966 --> 00:09:52.519 +we will get an interactive backtrace buffer + +00:09:52.520 --> 00:09:57.599 +where we can browse the source code for the different stack frames + +00:09:57.600 --> 00:10:00.199 +and the local variables within the stack frames, + +00:10:00.200 --> 00:10:03.439 +which are all presentations that we can inspect and so on. + +00:10:04.340 --> 00:10:10.619 +We can also open a REPL in the context of any stack frame. + +00:10:10.620 --> 00:10:16.439 +Or we can, when we go to the source for a given stack frame, + +00:10:16.440 --> 00:10:20.359 +we can select some Python code and evaluate it + +00:10:20.360 --> 00:10:25.959 +within the context of that stack frame. + +00:10:25.960 --> 00:10:30.699 +One major limitation compared to SLIME for Common Lisp + +00:10:30.700 --> 00:10:33.759 +is that in Common Lisp, you have the option to + +00:10:33.760 --> 00:10:38.159 +restart or resume execution from a given stack frame + +00:10:38.160 --> 00:10:42.439 +after an exception happens, where in Python, + +00:10:42.440 --> 00:10:45.799 +what we have right now is pretty much equivalent to + +00:10:45.800 --> 00:10:47.159 +the postmortem debugger. + +00:10:47.160 --> 00:10:50.839 +You can view the state that the call stack was in + +00:10:50.840 --> 00:10:51.959 +at the time of the exception, + +00:10:51.960 --> 00:10:55.659 +but you can't actually resume execution, + +00:10:55.660 --> 00:10:57.559 +which you often might want to do, + +00:10:57.560 --> 00:10:59.919 +because when you're coding in a dynamic language, + +00:10:59.920 --> 00:11:01.479 +you're going to get runtime errors. + +00:11:01.480 --> 00:11:04.119 +So if you're writing a script that does like some sort of + +00:11:04.120 --> 00:11:07.999 +long-running computation or processes a ton of files + +00:11:08.000 --> 00:11:11.939 +and gets an exception parsing one file halfway through, + +00:11:11.940 --> 00:11:16.919 +normally you'd have to fix the script, and then rerun it + +00:11:16.920 --> 00:11:19.759 +and have it process all the same files all over again, + +00:11:19.760 --> 00:11:23.839 +and lose a bunch of time for every bug you run into + +00:11:23.840 --> 00:11:24.879 +and fix you have to make. + +00:11:24.880 --> 00:11:28.679 +So right now we've got a kind of mediocre workaround + +00:11:28.680 --> 00:11:34.019 +which is you can add the restart decorator to a function + +00:11:34.020 --> 00:11:37.239 +and then... where in the case of a script + +00:11:37.240 --> 00:11:38.879 +processing a bunch of files, + +00:11:38.880 --> 00:11:41.799 +you would add the restart decorator to the function + +00:11:41.800 --> 00:11:43.599 +that processes a single file. + +00:11:43.600 --> 00:11:45.439 +You'd add it to the function + +00:11:45.440 --> 00:11:47.879 +that represents kind of the smallest unit of work + +00:11:47.880 --> 00:11:50.219 +that might fail with an exception, + +00:11:50.220 --> 00:11:54.359 +Then, when you get an exception, + +00:11:54.360 --> 00:11:57.479 +you can actually edit the function. + +00:11:57.480 --> 00:12:01.019 +Like, if we edit it so it doesn't throw an error, + +00:12:01.020 --> 00:12:07.199 +and then we can resume execution, + +00:12:07.200 --> 00:12:12.799 +then it will return from foo using the + +00:12:12.800 --> 00:12:15.040 +the new version of baz, + +00:12:15.041 --> 00:12:18.559 +without having to run the script from the beginning again. + +00:12:18.560 --> 00:12:22.379 +So in the example of a script that processes a bunch of files, + +00:12:22.380 --> 00:12:24.299 +that would let you, + +00:12:24.300 --> 00:12:27.619 +as you run into files that cause an exception, + +00:12:27.620 --> 00:12:29.079 +fix your code to deal with it + +00:12:29.080 --> 00:12:31.880 +and resume execution without having to restart the script + +00:12:31.881 --> 00:12:33.080 +from the beginning. + +00:12:33.081 --> 00:12:36.120 +But this is obviously a pretty terrible hack, + +00:12:36.121 --> 00:12:38.840 +having to add the restart decorator to the function. + +00:12:38.841 --> 00:12:46.739 +I would like it to be able to restart from any function. + +00:12:46.740 --> 00:12:49.631 +without needing the decorator, as you can in Common Lisp, + +00:12:49.632 --> 00:12:54.031 +but I think that will require patching CPython + +00:12:54.032 --> 00:12:56.579 +and I really have no idea how to do that. + +00:12:56.580 --> 00:13:00.531 +So if you do know anything about CPython internals + +00:13:00.532 --> 00:13:03.720 +and are interested in helping, please reach out. + +NOTE pydumpling + +00:13:03.721 --> 00:13:07.119 +Another feature we have with the backtrace buffer is + +00:13:07.120 --> 00:13:09.079 +there's this library called PyDumpling + +00:13:09.080 --> 00:13:14.659 +which can serialize a traceback and store it to a file. + +00:13:14.660 --> 00:13:17.859 +So you can use PyDumpling with your applications running in + +00:13:17.860 --> 00:13:21.239 +production to serialize a traceback + +00:13:21.240 --> 00:13:24.899 +whenever they have an exception and save it to a file. + +00:13:24.900 --> 00:13:28.599 +Then you can transfer the file locally + +00:13:28.600 --> 00:13:38.859 +and load it into your local Emacs with slime-py-load-pydumpling. + +00:13:38.860 --> 00:13:41.839 +This will load the same backtrace buffer, + +00:13:41.840 --> 00:13:44.559 +and you see all the same local variables + +00:13:44.560 --> 00:13:45.759 +at the time of the exception. + +00:13:45.760 --> 00:13:48.199 +You can inspect them and get a REPL + +00:13:48.200 --> 00:13:50.999 +in the context of the stack frame. + +00:13:51.000 --> 00:13:54.199 +Well, this will only work for variables + +00:13:54.200 --> 00:13:57.619 +that can be serialized with pickle. + +00:13:57.620 --> 00:13:59.519 +Or actually, the library uses dill, + +00:13:59.520 --> 00:14:03.039 +which can serialize a bit more than pickle can. + +00:14:03.040 --> 00:14:10.200 +But yeah so this can help you inspect and debug errors + +00:14:10.201 --> 00:14:12.880 +for applications running in production remotely + +00:14:12.881 --> 00:14:20.059 +that you don't want to have SLIME connected to 24-7. + +NOTE Documentation browser + +00:14:20.060 --> 00:14:24.859 +Next up, let's look at the documentation browser. + +00:14:24.860 --> 00:14:29.919 +We can bring up documentation for any module, + +00:14:29.920 --> 00:14:33.079 +and all this information is generated + +00:14:33.080 --> 00:14:34.999 +from runtime introspection, + +00:14:35.000 --> 00:14:37.079 +from the doc strings for the module + +00:14:37.080 --> 00:14:39.159 +and the classes and so on. + +00:14:39.160 --> 00:14:41.879 +So you won't see documentation for libraries + +00:14:41.880 --> 00:14:43.159 +that you don't have actually loaded + +00:14:43.160 --> 00:14:45.939 +into your running Python process. + +00:14:45.940 --> 00:14:50.119 +Then you can go browse to classes. + +00:14:50.120 --> 00:14:54.719 +It'll show all the attributes, their methods, and so on. + +00:14:54.720 --> 00:14:57.239 +By each method to the right, it will show + +00:14:57.240 --> 00:15:02.599 +the base class where the method was originally inherited from. + +00:15:02.600 --> 00:15:09.079 +You can also bring up a screen with all the Python packages + +00:15:09.080 --> 00:15:14.439 +that are installed, and browse that with imenu, + +00:15:14.440 --> 00:15:20.359 +and bring up information on any package and so on. + +NOTE Thread view + +00:15:20.360 --> 00:15:28.499 +Next up, let's take a look at the thread view. + +00:15:28.500 --> 00:15:31.839 +So let's run this and then bring up the thread view + +00:15:31.840 --> 00:15:35.559 +and this will show information on all running threads. + +00:15:35.560 --> 00:15:38.799 +You can configure it to refresh after a given interval, + +00:15:38.800 --> 00:15:41.959 +like every second, but I don't have that set up right now, + +00:15:41.960 --> 00:15:45.659 +so I have to manually refresh it. + +00:15:45.660 --> 00:15:47.639 +Probably the most useful thing is that + +00:15:47.640 --> 00:15:49.739 +you can bring up a backtrace for any thread + +00:15:49.740 --> 00:15:51.759 +which won't pause the thread or anything, + +00:15:51.760 --> 00:15:53.879 +but will just give you the call stack + +00:15:53.880 --> 00:15:55.879 +at the time you requested the backtrace. + +00:15:55.880 --> 00:15:59.199 +You can again view the stack frames, local variables, + +00:15:59.200 --> 00:16:04.139 +open a REPL in the context of the thread, and so on. + +00:16:04.140 --> 00:16:07.839 +There's also a viewer for async tasks, + +00:16:07.840 --> 00:16:09.999 +but I'm not going to demo that right now, + +00:16:10.000 --> 00:16:14.159 +because for that to work, you have to start swanky-python + +00:16:14.160 --> 00:16:16.599 +after the async event loop has started, + +00:16:16.600 --> 00:16:18.519 +from within the same thread. + +00:16:18.520 --> 00:16:20.279 +If you go to the project readme, + +00:16:20.280 --> 00:16:23.919 +there's a demo of how to use the async task viewer + +00:16:23.920 --> 00:16:27.439 +with a fastapi project. + +NOTE Tracing functions + +00:16:27.440 --> 00:16:33.879 +Next up, let's look at tracing functions. + +00:16:33.880 --> 00:16:36.279 +So here we got some random error, + +00:16:36.280 --> 00:16:39.879 +because this is still very much a work in progress. + +00:16:39.880 --> 00:16:42.359 +But it looks like it executed + +00:16:42.360 --> 00:16:43.199 +correctly this time. + +00:16:43.200 --> 00:16:47.565 +So now let's mark the fibonacci function + +00:16:47.566 --> 00:16:50.239 +for tracing and execute it. + +00:16:50.240 --> 00:16:56.079 +We can see, every time the function is called, + +00:16:56.080 --> 00:16:58.239 +all its arguments and return values. + +00:16:58.240 --> 00:17:02.899 +Again, there are presentations that we can inspect and so on. + +00:17:02.900 --> 00:17:06.079 +But let's inspect a more complex object, like a file object. + +00:17:06.080 --> 00:17:11.339 +If we trace the count_lines function and run that code, + +00:17:11.340 --> 00:17:15.319 +then we can inspect the file it was passed, or the file object. + +00:17:15.320 --> 00:17:21.039 +One pitfall is that in Python, objects are mutable. + +00:17:21.040 --> 00:17:25.559 +So in the trace buffer, the string representation + +00:17:25.560 --> 00:17:27.879 +that's printed is the string representation + +00:17:27.880 --> 00:17:31.219 +at the time it was passed to the function. + +00:17:31.220 --> 00:17:32.639 +But when we go to inspect it, + +00:17:32.640 --> 00:17:34.919 +we're inspecting the object as it is right now, + +00:17:34.920 --> 00:17:37.639 +which can be different than it was at the time + +00:17:37.640 --> 00:17:41.559 +the function saw it. So for this file object, for example, + +00:17:41.560 --> 00:17:44.279 +it's closed now, when it was open at the time + +00:17:44.280 --> 00:17:47.799 +the function used it. + +NOTE AI integrations + +00:17:47.800 --> 00:17:50.479 +Next up, let's look at AI integrations. + +00:17:50.480 --> 00:17:54.519 +So if you're used to SLIME with Common Lisp, + +00:17:54.520 --> 00:18:09.479 +Emacs actually has a built-in AI that can help with the transition. + +00:18:09.480 --> 00:18:14.559 +So it's just a joke, I actually really like Python. + +00:18:14.560 --> 00:18:18.119 +And for more serious AI integrations, + +00:18:18.120 --> 00:18:19.959 +I have some ideas for the future + +00:18:19.960 --> 00:18:21.919 +but I haven't implemented anything yet. + +00:18:21.920 --> 00:18:27.319 +I think right now, people are mostly passing source code to LLMs + +00:18:27.320 --> 00:18:32.679 +but since we're embedded in the Python process at runtime, + +00:18:32.680 --> 00:18:35.639 +we have a lot of more information available, + +00:18:35.640 --> 00:18:39.439 +like maybe we can trace all calls to functions, + +00:18:39.440 --> 00:18:41.799 +and when we have a bug, + +00:18:41.800 --> 00:18:46.479 +we can feed the trace to the LLM, + +00:18:46.480 --> 00:18:48.719 +and the LLM can point out maybe + +00:18:48.720 --> 00:18:51.959 +when this function was called with these arguments, + +00:18:51.960 --> 00:18:53.879 +its return value doesn't make sense, + +00:18:53.880 --> 00:18:55.679 +so maybe that's the root cause of your bug. + +00:18:55.680 --> 00:19:02.359 +If you have any ideas of potential LLM or AI integrations, + +00:19:02.360 --> 00:19:05.999 +let me know. I'm happy to discuss. + +NOTE LSP-type features + +00:19:06.000 --> 00:19:09.919 +Next up, let's look at standard LSP-type features. + +00:19:09.920 --> 00:19:14.439 +So we've got completions. It's fuzzy completions right now, + +00:19:14.440 --> 00:19:16.319 +so it's showing everything with a PR in the name. + +00:19:16.320 --> 00:19:21.779 +We can bring up documentation for each one. + +00:19:21.780 --> 00:19:26.759 +When we start calling a method in the minibuffer at the bottom + +00:19:26.760 --> 00:19:28.859 +it'll show the signature. + +00:19:28.860 --> 00:19:33.719 +There's some refactoring available. + +00:19:33.720 --> 00:19:37.399 +We can extract a function or variable, + +00:19:37.400 --> 00:19:39.499 +or rename something, + +00:19:39.500 --> 00:19:42.919 +like, let's rename fib to fib2, + +00:19:42.920 --> 00:19:47.479 +and it will rename all the uses of it. + +00:19:47.480 --> 00:19:49.759 +All these features are based on Jedi, + +00:19:49.760 --> 00:19:55.399 +which is the Python library used by IPython. + +00:19:55.400 --> 00:19:56.999 +But as it is right now, + +00:19:57.000 --> 00:20:02.039 +if you want the most complete Python development experience + +00:20:02.040 --> 00:20:05.579 +in Emacs, I'd probably recommend using LSP + +00:20:05.580 --> 00:20:10.439 +for everything LSP can do, and then just using swanky-python + +00:20:10.440 --> 00:20:13.679 +for the object inspector and backtrace buffer, + +00:20:13.680 --> 00:20:15.359 +and the interactive features it has + +00:20:15.360 --> 00:20:18.031 +that an LSP can't provide. + +NOTE Wrapping up + +00:20:18.032 --> 00:20:23.339 +And that's it really. + +00:20:23.340 --> 00:20:25.865 +Shortly we'll have questions and answers + +00:20:25.866 --> 00:20:28.799 +as part of EmacsConf, and later on, + +00:20:28.800 --> 00:20:31.199 +if you have any questions, ideas, or issues + +00:20:31.200 --> 00:20:34.639 +feel free to reach out over email + +00:20:34.640 --> 00:20:37.999 +or create an issue on the repository. + +00:20:38.000 --> 00:20:39.331 +I should probably warn you, + +00:20:39.332 --> 00:20:41.119 +if you want to try out the project: + +00:20:41.120 --> 00:20:45.279 +so far I'm probably the only user of it + +00:20:45.280 --> 00:20:48.279 +and I've only tested it on my own Emacs setup, + +00:20:48.280 --> 00:20:50.839 +so it's quite likely you'll run into issues + +00:20:50.840 --> 00:20:53.479 +trying to get it installed and working. + +00:20:53.480 --> 00:20:56.119 +But if you do run into problems, please reach out, + +00:20:56.120 --> 00:20:59.279 +let me know. I'm happy to help and try and fix them. + +00:20:59.280 --> 00:21:03.640 +So that's it. Thanks for listening. |