1 files changed, 724 insertions, 0 deletions

diff --git a/2024/captions/emacsconf-2024-literate--literate-programming-for-the-21st-century--howard-abrams--main.vtt b/2024/captions/emacsconf-2024-literate--literate-programming-for-the-21st-century--howard-abrams--main.vtt
new file mode 100644
index 00000000..986320d0
--- /dev/null
+++ b/2024/captions/emacsconf-2024-literate--literate-programming-for-the-21st-century--howard-abrams--main.vtt
@@ -0,0 +1,724 @@
+WEBVTT captioned by sachac
+
+NOTE Introduction
+
+00:00:00.000 --> 00:00:09.359
+Can you believe it's been a decade since I started
+
+00:00:09.360 --> 00:00:12.358
+pontificating on literate programming?
+
+00:00:12.359 --> 00:00:17.542
+I am Howard Abrams. In 2015, I spoke at this EmacsConf
+
+00:00:17.543 --> 00:00:21.705
+where I described my challenges I called Literate DevOps.
+
+00:00:21.706 --> 00:00:25.634
+The conference wasn't completely virtual, even though I was.
+
+00:00:25.635 --> 00:00:29.317
+My city of Portland was suffering a citywide electrical outage
+
+00:00:29.318 --> 00:00:33.479
+and I was without power, so I gave the talk in a corner of my
+
+00:00:33.480 --> 00:00:37.439
+friend's living room. People online asking questions and
+
+00:00:37.440 --> 00:00:41.439
+wondering about literate programming... I also see comments
+
+00:00:41.440 --> 00:00:44.599
+explaining why literate programming hasn't caught on in
+
+00:00:44.600 --> 00:00:49.079
+corporate practice. I often don't engage. I mean, is the
+
+00:00:49.080 --> 00:00:51.599
+online arguments and chatter over ignorance or
+
+00:00:51.600 --> 00:00:56.719
+preference? Sure, we're wired differently. I mean, my
+
+00:00:56.720 --> 00:00:59.559
+favorite programming languages put the parentheses
+
+00:00:59.560 --> 00:01:01.939
+before the function name.
+
+00:01:01.940 --> 00:01:03.800
+Literate programming has come a long way
+
+00:01:03.801 --> 00:01:08.519
+since Knuth proposed it in the 19th century. I feel
+
+00:01:08.520 --> 00:01:12.999
+it's come a long way just in the last 10 years. Obviously,
+
+00:01:13.000 --> 00:01:16.399
+this interest is due to Org. I don't think I would bother if
+
+00:01:16.400 --> 00:01:21.359
+all I had was Knuth's original preprocessor. But since I'm
+
+00:01:21.360 --> 00:01:24.839
+talking to fellow nerds about an open source project
+
+00:01:24.840 --> 00:01:27.919
+without corporate backing, let me change the title of my
+
+00:01:27.920 --> 00:01:32.919
+talk and re-pitch Literate Programming in the 24th and a
+
+00:01:32.920 --> 00:01:35.252
+Half Century!
+
+NOTE Do I still literate?
+
+00:01:35.253 --> 00:01:36.653
+People often ask if I still program that way.
+
+00:01:36.654 --> 00:01:42.759
+I guess they want to know if there's any long-term benefits,
+
+00:01:42.760 --> 00:01:45.919
+for many of our tools and our workflows, while initially
+
+00:01:45.920 --> 00:01:51.079
+tantalizing, often don't last. But yes, when I sit down to
+
+00:01:51.080 --> 00:01:57.759
+write a program, I create a file with an extension of .org.
+
+00:01:57.760 --> 00:02:03.799
+I guess you can say I program literally.
+
+00:02:03.800 --> 00:02:07.359
+Let me be transparent. Do I use literate programming during
+
+00:02:07.360 --> 00:02:12.599
+my day job? Yes, but only for personal tools or for initial
+
+00:02:12.600 --> 00:02:16.759
+investigation. At the end of the sprint, I tangle the file
+
+00:02:16.760 --> 00:02:21.079
+and git commit that. My personal projects, on the other
+
+00:02:21.080 --> 00:02:25.679
+hand, are Org files. Since I can't show you the code from
+
+00:02:25.680 --> 00:02:27.839
+my day job, I'm afraid my example code will have a lot of
+
+00:02:27.840 --> 00:02:31.159
+parentheses.
+
+00:02:31.160 --> 00:02:33.955
+I'm sure you won't mind.
+
+00:02:33.956 --> 00:02:37.356
+I like having my Emacs configuration in Org.
+
+00:02:37.357 --> 00:02:40.359
+It's pretty bling. It has over 8,000
+
+00:02:40.360 --> 00:02:44.559
+lines of code. I know, I can hear the screams and gasps over
+
+00:02:44.560 --> 00:02:49.439
+the network. However, the surrounding prose in Org adds
+
+00:02:49.440 --> 00:02:53.410
+10,000 lines, and those lines are non-wrapped paragraphs.
+
+00:02:53.411 --> 00:02:58.119
+I mean, is that large? Sure, we've all worked on
+
+00:02:58.120 --> 00:03:03.639
+larger, so I guess it's not huge. Come on, it's still
+
+00:03:03.640 --> 00:03:06.331
+significant.
+
+NOTE Advantages
+
+00:03:06.332 --> 00:03:09.799
+Advantages? Look who I'm talking to. I'm sure
+
+00:03:09.800 --> 00:03:14.279
+you know the advantages, but indulge me. I feel that one
+
+00:03:14.280 --> 00:03:16.799
+advantage of literate programming, especially with large
+
+00:03:16.800 --> 00:03:20.279
+code bases, is how you can organize and manage the
+
+00:03:20.280 --> 00:03:24.839
+complexity. Most programming languages tame large bases
+
+00:03:24.840 --> 00:03:29.119
+by putting code in separate files. While Org can too, with
+
+00:03:29.120 --> 00:03:32.279
+Org, we can group related functions together under
+
+00:03:32.280 --> 00:03:35.043
+expandable headlines.
+
+00:03:35.044 --> 00:03:37.279
+Here's one. You can see that
+
+00:03:37.280 --> 00:03:40.706
+I've got different sections grouped together.
+
+00:03:40.707 --> 00:03:43.759
+In my original talk, I mentioned how I would attempt to organize
+
+00:03:43.760 --> 00:03:47.839
+my thoughts before coding. I appreciate how I can look back
+
+00:03:47.840 --> 00:03:53.599
+at my notes. In my Emacs configuration, I review the prose to
+
+00:03:53.600 --> 00:03:57.799
+help memorize key bindings.
+
+00:03:57.800 --> 00:04:01.039
+My section on getting email working with Emacs using
+
+00:04:01.040 --> 00:04:04.079
+notmuch means creating small collections of scripts and
+
+00:04:04.080 --> 00:04:08.199
+configuration files. I can tangle them all from one Org
+
+00:04:08.200 --> 00:04:16.799
+file. I like that I can explain each part separately.
+
+00:04:16.800 --> 00:04:20.879
+You just can't beat having links back to Stack Overflow or
+
+00:04:20.880 --> 00:04:25.519
+that GitHub repo where you stole, I mean, became inspired to
+
+00:04:25.520 --> 00:04:28.719
+write your code.
+
+NOTE Disadvantages
+
+00:04:28.720 --> 00:04:34.279
+Literate programming may push the boundaries of our
+
+00:04:34.280 --> 00:04:38.119
+workflows and revealing some abrasion, but we aren't
+
+00:04:38.120 --> 00:04:41.239
+solely working with Org. We have the flexibility of a Lisp
+
+00:04:41.240 --> 00:04:45.119
+engine to file down those rough parts. You may have your
+
+00:04:45.120 --> 00:04:48.159
+concerns. Perhaps you could reach out to me, and with
+
+00:04:48.160 --> 00:04:54.239
+particular issues, maybe we can figure something out.
+
+00:04:54.240 --> 00:04:57.439
+Here is my list of frictions, and the rest of my talk
+
+00:04:57.440 --> 00:05:02.159
+demonstrates my answers and my hacks. The goal in literate
+
+00:05:02.160 --> 00:05:05.039
+programming with Org is that it should not require more
+
+00:05:05.040 --> 00:05:08.679
+effort than non-literate programming. For instance, I
+
+00:05:08.680 --> 00:05:12.119
+shouldn't have to type much more than regular programming
+
+00:05:12.120 --> 00:05:15.719
+to get my code literate. I also shouldn't have to worry about
+
+00:05:15.720 --> 00:05:20.799
+the state between my Org file and the source code. I want
+
+00:05:20.800 --> 00:05:24.132
+to be able to jump around my code just as easily.
+
+NOTE Ease of typing
+
+00:05:24.133 --> 00:05:28.654
+Let me explain more. I've created some templates using
+
+00:05:28.655 --> 00:05:34.679
+yasnippet. Since I was used to the old org-tempo feature,
+
+00:05:34.680 --> 00:05:37.145
+my habit has all the snippets starting with a
+
+00:05:37.146 --> 00:05:40.759
+< character. I'm not sure if I should demonstrate all of them
+
+00:05:40.760 --> 00:05:45.999
+as you may be doing something similar. I like to build on top
+
+00:05:46.000 --> 00:05:49.999
+of characters to remind me that if I just enter a <s, I
+
+00:05:50.000 --> 00:05:53.519
+need to put in the language. But if I append a mnemonic, I can
+
+00:05:53.520 --> 00:05:56.839
+get a full language. Why not do that with a full function
+
+00:05:56.840 --> 00:06:01.199
+definition? In this case, I'm smooshing one yasnippet
+
+00:06:01.200 --> 00:06:11.679
+inside another one in order to save myself some typing.
+
+00:06:11.680 --> 00:06:15.159
+My point here is to pay attention to what slows you down or
+
+00:06:15.160 --> 00:06:24.719
+hinders you from getting the advantages you want.
+
+NOTE Keep tangled code sync'd
+
+00:06:24.720 --> 00:06:28.399
+Do you ever forget to tangle your code? You can append this
+
+00:06:28.400 --> 00:06:31.519
+code to the bottom of your Org file so that it gets tangled
+
+00:06:31.520 --> 00:06:36.159
+every time you save. I've written a function so I can visit
+
+00:06:36.160 --> 00:06:40.559
+that tangled file and then return. I've grouped all my
+
+00:06:40.560 --> 00:06:45.119
+functions together. I've taken a cue from Charles Choi, you
+
+00:06:45.120 --> 00:06:48.639
+know, kickingvegas, and his Casual feature set. But
+
+00:06:48.640 --> 00:06:52.374
+instead of Transient, I've just made a hydra using
+
+00:06:52.375 --> 00:06:57.399
+the major-mode-hydra package. Anyway, this allows me to use and
+
+00:06:57.400 --> 00:07:00.136
+remember my micro-optimizations.
+
+00:07:00.137 --> 00:07:03.697
+If you set the :comments property to link,
+
+00:07:03.698 --> 00:07:06.999
+the tangled output is back-connected.
+
+00:07:07.000 --> 00:07:11.479
+This allows us to edit the tangled code and have it update the
+
+00:07:11.480 --> 00:07:16.879
+Org file. Personally, I don't like this. My source of truth
+
+00:07:16.880 --> 00:07:22.500
+is the Org file, and I tangle as a one-way diode.
+
+NOTE Code evaluation
+
+00:07:22.501 --> 00:07:25.603
+Often a block of code will reference a variable
+
+00:07:25.604 --> 00:07:29.046
+or call a function to find in another block of code.
+
+00:07:29.047 --> 00:07:31.508
+In my original literate DevOps talk,
+
+00:07:31.509 --> 00:07:34.519
+I discussed how to use the output from one block into
+
+00:07:34.520 --> 00:07:37.799
+another block by naming the first block and referencing it
+
+00:07:37.800 --> 00:07:42.159
+with a :var for the second. However, if all the blocks use the
+
+00:07:42.160 --> 00:07:46.039
+same language, you can use sessions, which create a
+
+00:07:46.040 --> 00:07:51.479
+persistent REPL behind the scenes. Let's evaluate the
+
+00:07:51.480 --> 00:07:53.199
+blocks of Python code in this file.
+
+00:07:53.200 --> 00:08:00.119
+The evaluation created a Python REPL. It's available in
+
+00:08:00.120 --> 00:08:04.279
+another buffer. This buffer matches the name of the
+
+00:08:04.280 --> 00:08:07.959
+session, but with surrounding asterisks. Evaluating a
+
+00:08:07.960 --> 00:08:11.399
+code block sends it into the REPL, and now I can work with my
+
+00:08:11.400 --> 00:08:19.959
+code blocks interactively. (That's not quite right.)
+
+NOTE Has that block been eval'd?
+
+00:08:19.960 --> 00:08:24.039
+I primarily hack on Emacs Lisp, and textual changes to
+
+00:08:24.040 --> 00:08:28.199
+variables, functions, or macros--unless you habitually
+
+00:08:28.200 --> 00:08:31.679
+type C-c C-c--may not represent the state of your
+
+00:08:31.680 --> 00:08:35.439
+machine. A similar effect happens in any language that
+
+00:08:35.440 --> 00:08:39.319
+uses sessions. Sure, I can move the point to a block and
+
+00:08:39.320 --> 00:08:42.799
+evaluate, but I have three functions that allow me to
+
+00:08:42.800 --> 00:08:44.734
+evaluate all blocks in a buffer or all blocks in a subtree,
+
+00:08:44.735 --> 00:08:50.199
+or I can, without moving the point, evaluate any block I see.
+
+00:08:50.200 --> 00:08:54.919
+Now, this function here evaluates all blocks in a buffer.
+
+00:08:54.920 --> 00:08:58.279
+Someone mentioned calling this function when you first
+
+00:08:58.280 --> 00:09:02.359
+load a file. I'm not sure that's a good policy. I mean, have
+
+00:09:02.360 --> 00:09:05.238
+you not written a bug?
+
+NOTE Evaluating code in a subtree
+
+00:09:05.239 --> 00:09:08.559
+Since this function right here
+
+00:09:08.560 --> 00:09:12.039
+evaluates only visible blocks, we can limit what Emacs
+
+00:09:12.040 --> 00:09:18.799
+evaluates to a single Org mode section. For instance, with
+
+00:09:18.800 --> 00:09:23.759
+the cursor in one section, I can evaluate just the blocks in
+
+00:09:23.760 --> 00:09:26.871
+that header section.
+
+NOTE Evaluating code from a distance
+
+00:09:26.872 --> 00:09:29.399
+If I can see a block, why clumsily
+
+00:09:29.400 --> 00:09:33.079
+navigate to it when I can extend the avy project to just jump to
+
+00:09:33.080 --> 00:09:40.479
+it? For instance, let's pull this file up. I can jump to any of
+
+00:09:40.480 --> 00:09:41.639
+the four blocks.
+
+00:09:41.640 --> 00:09:50.319
+I think that's quite slick. Now why navigate to a code block
+
+00:09:50.320 --> 00:09:55.799
+solely to evaluate it? Yes, this is a terrible example, but
+
+00:09:55.800 --> 00:09:59.679
+these three blocks set a variable to different values. So
+
+00:09:59.680 --> 00:10:02.599
+without moving the point, I can evaluate any one of them.
+
+00:10:02.600 --> 00:10:09.719
+To be honest, the reason why I wrote this is because I often
+
+00:10:09.720 --> 00:10:13.999
+forget to evaluate a block after editing it. I've moved on,
+
+00:10:14.000 --> 00:10:17.839
+and I just don't want to jump back. Now, I can just evaluate
+
+00:10:17.840 --> 00:10:22.359
+from a distance. I apologize for the previous terrible
+
+00:10:22.360 --> 00:10:26.019
+examples, but I'm quite pleased with this feature.
+
+NOTE Navigating by headers
+
+00:10:26.020 --> 00:10:30.119
+As I mentioned earlier, in a large code base, we organize code by
+
+00:10:30.120 --> 00:10:33.839
+library or module, and each file contains a class composed
+
+00:10:33.840 --> 00:10:37.119
+of methods, functions, variables, fields, et cetera.
+
+00:10:37.120 --> 00:10:39.999
+Literate programming in Org files allows me to add a
+
+00:10:40.000 --> 00:10:43.159
+semantic organization layer where I can group related
+
+00:10:43.160 --> 00:10:46.919
+concepts under headlines. Now, while this isn't specific
+
+00:10:46.920 --> 00:10:50.799
+to literate programming, I wrote a little user interface to
+
+00:10:50.800 --> 00:10:54.296
+allow me to jump to any heading in any Org file
+
+00:10:54.297 --> 00:10:57.679
+in a particular project.
+
+00:10:57.680 --> 00:11:02.879
+These are the headings in my Emacs configuration project.
+
+00:11:02.880 --> 00:11:06.559
+Notice the file name beforehand, before the colon
+
+00:11:06.560 --> 00:11:09.759
+character. The header name and its parent headers are
+
+00:11:09.760 --> 00:11:14.799
+after. Let me search for the LSP sections. Maybe I only want
+
+00:11:14.800 --> 00:11:20.039
+the one for Python. Now I use ripgrep to search the files and
+
+00:11:20.040 --> 00:11:24.559
+then some Lisp to parse the output. Unless someone has
+
+00:11:24.560 --> 00:11:26.793
+already done this, I should package this up on MELPA.
+
+NOTE Navigating by function names
+
+00:11:26.794 --> 00:11:32.199
+What about jumping directly to the definition of a function,
+
+00:11:32.200 --> 00:11:36.799
+variable, or what have you? We can use Emacs's built-in xref
+
+00:11:36.800 --> 00:11:39.879
+library, but these functions don't understand that the
+
+00:11:39.880 --> 00:11:45.319
+source code is in Org files. When I started using Emacs
+
+00:11:45.320 --> 00:11:49.479
+30-something years ago, I would pre-index my source into
+
+00:11:49.480 --> 00:11:53.799
+tag files, but the dumb-jump project uses the newfangled and
+
+00:11:53.800 --> 00:11:58.319
+faster text search programs like ripgrep to find a symbol in
+
+00:11:58.320 --> 00:12:02.319
+real time. I followed this pattern and wrote an extension
+
+00:12:02.320 --> 00:12:08.119
+to the xref API. Now, I want to jump around my code from both
+
+00:12:08.120 --> 00:12:14.519
+code block or in the surrounding prose. I'm sure it
+
+00:12:14.520 --> 00:12:18.199
+comes as no surprise that my presentation is just an Org
+
+00:12:18.200 --> 00:12:23.919
+file. Let's suppose my cursor is on this symbol. I wrote this
+
+00:12:23.920 --> 00:12:28.079
+function for this demonstration. We can jump to the
+
+00:12:28.080 --> 00:12:30.759
+definition and I can jump back.
+
+00:12:30.760 --> 00:12:37.639
+Notice it jumped into an Org file and back out. References,
+
+00:12:37.640 --> 00:12:42.279
+unlike definitions, is where something is defined and
+
+00:12:42.280 --> 00:12:46.919
+where it's used. Well, you know how the xref system works.
+
+00:12:46.920 --> 00:12:52.679
+Here, I can jump to the definition or where it's
+
+00:12:52.680 --> 00:12:59.519
+used. Of course, and jump back. I think this is cool. This
+
+00:12:59.520 --> 00:13:04.319
+should be a nifty package on MELPA. But my code is specific to
+
+00:13:04.320 --> 00:13:08.799
+Lisp, and I'm not completely sure how to make it general. For
+
+00:13:08.800 --> 00:13:13.399
+instance, what is a symbol? If you know the language, this is
+
+00:13:13.400 --> 00:13:17.679
+obvious. But what should the language be when your cursor is
+
+00:13:17.680 --> 00:13:22.639
+in the prose of an Org file? Python only supports sequences
+
+00:13:22.640 --> 00:13:25.559
+of alphanumeric and underscores, but in Lisp, a symbol can
+
+00:13:25.560 --> 00:13:30.399
+be almost any character sequence. I've been stewing on how
+
+00:13:30.400 --> 00:13:34.479
+to do this. I have ideas like prompting during the first
+
+00:13:34.480 --> 00:13:37.719
+query or scanning the language based on the nearest code
+
+00:13:37.720 --> 00:13:40.479
+block. I think I'm babbling.
+
+NOTE Why literate programming?
+
+00:13:40.480 --> 00:13:47.199
+In true geek fashion, I dived into the details before
+
+00:13:47.200 --> 00:13:52.079
+answering some better questions. In my original Literate
+
+00:13:52.080 --> 00:13:55.479
+DevOps talk, I explained the advantages of initially
+
+00:13:55.480 --> 00:13:58.959
+writing down your thoughts, your plans, goals... the
+
+00:13:58.960 --> 00:14:02.879
+user requirements. But what do you do with all that luscious
+
+00:14:02.880 --> 00:14:06.359
+prose afterwards? Well, you do the same thing you do to your
+
+00:14:06.360 --> 00:14:09.279
+initial code. You refactor that prose.
+
+00:14:09.280 --> 00:14:14.759
+Just because the tech surrounding your code is now a
+
+00:14:14.760 --> 00:14:18.799
+first-class citizen doesn't excuse bad code. You want
+
+00:14:18.800 --> 00:14:23.165
+something more from both your code and your prose.
+
+NOTE LP prose isn't comments
+
+00:14:23.166 --> 00:14:25.586
+The prose of your literate program isn't
+
+00:14:25.587 --> 00:14:28.667
+just regurgitation of the code in the block.
+
+00:14:28.668 --> 00:14:31.527
+You want something more helpful.
+
+00:14:31.528 --> 00:14:35.736
+You're really writing a research paper to yourself.
+
+00:14:35.737 --> 00:14:38.577
+I know what you're thinking. You've seen my Git repos.
+
+00:14:38.578 --> 00:14:41.858
+I'm guilty and not always the best example.
+
+00:14:41.859 --> 00:14:44.559
+However, I do get great joy
+
+00:14:44.560 --> 00:14:48.680
+when I see someone ask about something in Emacs
+
+00:14:48.681 --> 00:14:51.041
+and my response is little more than a link
+
+00:14:51.042 --> 00:14:55.799
+to my online repo that I've rendered as a website.
+
+NOTE Summary
+
+00:14:55.800 --> 00:15:01.199
+I'm out of time. I hope this has been interesting
+
+00:15:01.200 --> 00:15:04.359
+philosophically as well as practically, as I think
+
+00:15:04.360 --> 00:15:08.559
+literate programming is the cat's meow. I'm afraid this
+
+00:15:08.560 --> 00:15:11.879
+summary slide is about my home-baked solutions that fit my
+
+00:15:11.880 --> 00:15:15.119
+needs, but hopefully you can recognize your pain points and
+
+00:15:15.120 --> 00:15:17.839
+address them. If you don't need my Literate
+
+00:15:17.840 --> 00:15:21.479
+DevOps-specific techniques for connecting code blocks, I
+
+00:15:21.480 --> 00:15:25.799
+suggest using sessions by default. I highly recommend
+
+00:15:25.800 --> 00:15:28.399
+looking at your workflow and writing snippets to give you
+
+00:15:28.400 --> 00:15:33.159
+less typing for Org blocks. I now jump by headlines in my
+
+00:15:33.160 --> 00:15:37.479
+projects, but extending xref to support Org files made
+
+00:15:37.480 --> 00:15:40.159
+literate programming as easy as programming the
+
+00:15:40.160 --> 00:15:44.319
+old-fashioned way. I do need to make it more general to put up
+
+00:15:44.320 --> 00:15:47.722
+on MELPA, though. Thanks for watching.
+
+00:15:47.723 --> 00:15:51.240
+Happy hacking, my friends.