WEBVTT 00:00.000 --> 00:04.880 Hi, my name is Eduard Duax, and the title of this talk is 00:04.880 --> 00:09.680 Rapples in Strange Places, Lua, LaTeX, LPEG, LPEG-REX, and TIX. 00:10.720 --> 00:17.200 I'm the author of an MX package called EEV, and this is a talk at the MXConf 2023 00:17.200 --> 00:20.880 that is happening in December 2023 at the Internets. 00:21.680 --> 00:25.920 And this is one of the examples of diagrams that we are going to see. 00:25.920 --> 00:27.680 Let me show how I generate it. 00:28.640 --> 00:32.240 One second. I have to use a smaller font here. 00:35.600 --> 00:41.920 This is a file called parse32.lua. Let me go back to this block of tests again. 00:42.720 --> 00:50.320 And now if I run this, we get these outputs here at the right. 00:52.000 --> 00:54.960 And then in this line here, it generates a PDF. 00:55.520 --> 01:01.440 And if I type f8 here, it shows the PDF in the lower right window. 01:04.240 --> 01:07.600 Let me start by explaining briefly what is EEV. 01:09.920 --> 01:14.560 First, it's something that appeared by accident in the mid-90s. 01:14.560 --> 01:19.920 I explained this story in my presentation at the MXConf 2019. 01:20.720 --> 01:25.520 It's a package. It's an MX package that is part of ELPA. 01:25.520 --> 01:31.440 It has at least 10 users. Those are the ones that I know by name. 01:33.360 --> 01:36.320 EEV means MX Execute Verbosely. 01:37.360 --> 01:42.640 EEV is something that treats eval as the central feature of MX. 01:43.680 --> 01:46.800 EEV blurs the distinction between programmers and users, 01:47.520 --> 01:50.080 and it replaces the slogan, 01:50.080 --> 01:55.360 users should not be forced to see Lisp, that is something that Richard Stallman told me once, 01:55.360 --> 01:59.280 by users should see Lisp instead of buttons, 01:59.280 --> 02:02.400 and new users should see Lisp in the first five minutes. 02:04.240 --> 02:07.120 I'm going to show some examples of that soon. 02:08.800 --> 02:12.880 EEV uses code in comments a lot, and also tests in comments. 02:13.600 --> 02:21.040 I changed my way of presenting it, and it became very REPL-centric in the last few years, 02:21.040 --> 02:28.240 in the sense that I start by explaining its main features by its support for REPLs. 02:30.320 --> 02:34.320 EEV supposes that we want to keep executable notes of everything. 02:34.320 --> 02:37.040 I'm also going to show examples of this in a second. 02:38.080 --> 02:41.600 EEV has lots of videos for people who hate videos, 02:42.240 --> 02:47.520 and it tries to do everything with very little magic and without black boxes. 02:47.520 --> 02:50.240 I'm going to explain many of these things very soon. 02:52.880 --> 02:58.160 This is a figure that I'm going to show in detail soon, 02:58.160 --> 03:01.520 that is about something important about Lua. 03:01.520 --> 03:05.280 Oops, the font is very bad now, so let me change the font. 03:05.280 --> 03:07.840 The figure is this one, and 03:08.560 --> 03:13.440 what most people do when they visit a file with something interesting on it 03:13.440 --> 03:16.640 is that they just go there, and they set a bookmark there, 03:16.640 --> 03:19.760 or they put the position in the register. 03:21.760 --> 03:28.480 But I prefer to keep links to everything that is interesting as a list of hyperlinks. 03:28.480 --> 03:32.640 So, for example, this is an at least hyperlink to the file 03:32.640 --> 03:36.480 that goes to this anchor here, and to this string I've added. 03:36.480 --> 03:38.560 And to this string after this anchor. 03:39.760 --> 03:45.760 This is a variant that opens that file in the window at the right here. 03:47.440 --> 03:53.520 And this is a sexpid that changes the font. 03:53.520 --> 03:57.680 I have a command with a very short name that does that, 03:58.400 --> 04:02.880 but I prefer to keep that as a one-liner. 04:03.840 --> 04:11.440 About the videos, we can see the list of first-class videos of Eevee by executing this, 04:11.440 --> 04:16.960 meta-x, find first-class videos, or by running this alias here, 04:16.960 --> 04:20.320 meta-x1c, and then what we see is this. 04:21.520 --> 04:25.440 The first sexpid here regenerates this buffer, 04:25.440 --> 04:28.960 so we can make a mess here and then run this, 04:29.040 --> 04:33.040 and the original buffer is regenerated again in a clean way. 04:34.880 --> 04:42.000 Each of these things here opens a buffer with information about a video. 04:42.000 --> 04:44.400 Let me take a specific example here. 04:46.080 --> 04:50.720 This video here is about one of the ancestors of this talk, 04:50.720 --> 04:52.960 that is a library that I wrote for 04:53.920 --> 05:01.120 creating diagrams in LaTeX using a package called pic2e, using repls. 05:02.320 --> 05:10.720 Anyway, the thing is that if we run a sexpid like this one, 05:10.720 --> 05:13.440 and we don't have a local copy of the video, 05:14.080 --> 05:16.320 Eevee will try to load the local copy, 05:16.320 --> 05:19.600 and instead of doing that and by asking something like, 05:19.680 --> 05:20.720 asking something like, 05:20.720 --> 05:24.560 do you want me to download the local copy, blah, blah, blah, blah, 05:25.680 --> 05:28.720 it simply opens a buffer like this. 05:28.720 --> 05:32.480 I mean, if we don't have a local copy yet, 05:32.480 --> 05:34.400 it will open a buffer like this one, 05:35.440 --> 05:41.760 in which these things here in comments are links to the documentation. 05:41.760 --> 05:48.240 I mean, this thing here explains the idea of local copies of files from the internet, 05:48.880 --> 05:52.960 uh, there are more details here and here, 05:53.840 --> 05:59.280 and this is a script that we can execute line by line. 05:59.280 --> 06:04.080 So, instead of this script being hidden behind the button that we just 06:04.960 --> 06:07.360 press after a question like, 06:07.360 --> 06:09.920 do you want me to do something, blah, blah, blah, yes or no, 06:11.200 --> 06:15.120 the script is visible here, and we can execute it step by step. 06:15.200 --> 06:22.080 It creates a terminal with a shell here in the right window, 06:22.080 --> 06:28.880 and when we type f8 in these lines here, the lines are sent to this line. 06:30.080 --> 06:33.920 So, this is going to download the copy of the video, 06:33.920 --> 06:39.360 the wget says that I already have a copy of the video in its subtitles, and so on. 06:39.920 --> 06:47.600 And after getting a copy of the video, we can run this exp here, and it displays the video. 06:52.880 --> 06:57.840 I said that Eevee has lots of videos for people who hate videos, 06:57.840 --> 07:02.960 and the idea is that very few people are going to watch the videos in real time, 07:03.920 --> 07:10.480 and most of the people that I know, or most of the people that are interested in Eevee in some way, 07:10.480 --> 07:15.280 they are going to watch just small sections of the video, 07:15.280 --> 07:18.720 and most of the time they're just going to read the subtitles of the video. 07:19.680 --> 07:25.760 So, for each one of the videos, we have a page about the video. 07:25.760 --> 07:28.400 Let me see if I have internet here. 07:28.400 --> 07:29.680 Yes, this is a page, 07:29.920 --> 07:36.080 and usually these pages have a link to another page, 07:36.080 --> 07:42.880 the page that has all the subtitles of the video, wherever. 07:42.880 --> 07:46.960 In this one, it's not so visible, but anyway, 07:46.960 --> 07:50.480 there are several ways of accessing the subtitles of the video, 07:50.480 --> 07:54.720 and one of the ways is by running this exp here, 07:55.040 --> 07:57.280 is by running this exp here, 07:59.440 --> 08:03.680 that opens a file in Lua that is what I use to generate the subtitles. 08:04.880 --> 08:13.760 Anyway, by the way, each one of these things here is hyperlinked to a position of the video, 08:14.400 --> 08:19.280 so if I type this in the right way, it goes to that position. 08:20.160 --> 08:21.280 Anyway, let me go back. 08:21.920 --> 08:26.640 Also, the tutorials of Eevee, the intros of Eevee, 08:26.640 --> 08:30.560 let's start with find, and with intro, 08:30.560 --> 08:35.760 they have lots of blocks that say video links, like this one, 08:36.800 --> 08:40.080 and these blocks have links to the positions in videos, 08:40.080 --> 08:43.600 and if we don't have a local copy of the video yet, 08:44.480 --> 08:48.800 the thing shows us a script that lets us download the local copy. 08:48.880 --> 08:56.720 Anyway, I said that I was going to explain what I mean by magic and black boxes. 08:59.360 --> 09:02.080 This is something that I've been trying to explain for a long time, 09:02.080 --> 09:05.840 and I think that I got a very good explanation about that 09:05.840 --> 09:09.440 in a video that I made about something called Eevee wconfig, 09:09.440 --> 09:15.120 that is a tool for configuring Eevee on Windows without magic, 09:15.840 --> 09:20.320 on Windows without magic, without buttons that do things 09:21.040 --> 09:22.720 without explaining what they are doing. 09:24.080 --> 09:26.720 This is a part of the subtitles of the video. 09:26.720 --> 09:27.680 Let me read that. 09:30.080 --> 09:32.640 Eevee wconfig is an attempt to solve the problem 09:32.640 --> 09:35.040 of how to install these things on Windows, 09:35.040 --> 09:38.240 both without magic and with very little magic. 09:39.520 --> 09:40.640 Remember this slogan, 09:41.200 --> 09:45.760 any sufficiently advanced technology is indistinguishable from magic. 09:47.280 --> 09:51.600 Here in this video, I'm going to use the term magic as a shorthand 09:51.600 --> 09:55.520 for sufficiently advanced technology, 09:55.520 --> 09:59.040 that is something that is complex and non-obvious, 09:59.040 --> 10:01.840 and that is indistinguishable from magic 10:01.840 --> 10:04.640 in the sense of being almost impossible to understand. 10:05.600 --> 10:10.720 And I'm also going to use the term black box as a near synonym for magic, 10:11.360 --> 10:15.360 and sometimes the term black box is more convenient, 10:15.360 --> 10:18.800 even though it's a bit longer, it has more letters, 10:18.800 --> 10:20.880 because when I use the term black box, 10:20.880 --> 10:25.440 it invites us to use expressions like opening the black box, 10:25.440 --> 10:27.840 and I'm going to use that expression a lot. 10:34.640 --> 10:37.360 Now, let me try to explain what is... 10:37.360 --> 10:40.400 Sorry, let me change the font. 10:44.000 --> 10:44.800 What is Lua? 10:46.000 --> 10:51.840 Lua is a minimalistic language in the sense of batteries not included. 10:52.880 --> 10:56.640 It uses associative tables for most of its data structures, 10:58.800 --> 11:02.240 and it's so minimalistic that it's the default print function. 11:03.120 --> 11:09.280 When we create an associative table and we ask it to print... 11:11.760 --> 11:14.560 We ask print to print an associative table, 11:14.560 --> 11:17.280 it just prints the address of the table. 11:17.280 --> 11:18.480 Here are some examples. 11:19.920 --> 11:23.760 Here is a table, and when we ask print to print it, 11:23.760 --> 11:26.720 it just says that it's the table at this address here. 11:27.440 --> 11:32.400 So, one of the things that most people do when they start using Lua 11:32.400 --> 11:36.320 is that either they download a package with a print to printing function, 11:36.320 --> 11:38.720 or they write their own print to printing functions. 11:39.360 --> 11:42.000 My own print to printing function is called pp, 11:42.000 --> 11:45.840 with uppercase letters, and it works like this, 11:47.600 --> 11:50.880 and it prints associative tables in a way like this. 11:50.880 --> 11:54.800 It says that for the key 1, 11:55.200 --> 11:57.280 the value associated to it is 2, 11:57.280 --> 11:59.200 for the key 2, the value is 3, 11:59.200 --> 12:01.760 and for the key 3, the value is 5. 12:08.960 --> 12:12.160 When I started using Lua, one of my favorite languages 12:12.160 --> 12:15.600 was also a language that used associative tables a lot. 12:15.600 --> 12:16.720 It was called Icon, 12:18.400 --> 12:22.640 and I had to write my own print to printing functions for Icon. 12:22.720 --> 12:28.800 So, I just had to port my print to printing functions to Lua, 12:28.800 --> 12:31.600 and my first version looked something like this. 12:32.240 --> 12:36.560 It just had some global functions, lots of them actually, 12:39.680 --> 12:41.840 and after a while I rewrote it, 12:41.840 --> 12:44.320 and I rewrote it again, and again, and again, 12:44.320 --> 12:49.200 and this is one of the versions that is not even the default at this point. 12:50.000 --> 12:53.920 TOS is for toString, 12:56.800 --> 12:57.920 and this is a demo. 12:58.800 --> 13:04.560 It's very modular, so it's easy to replace parts of it or to toggle flags, 13:04.560 --> 13:05.760 and this is an example. 13:05.760 --> 13:10.320 If I try to print the table of methods for a certain class, 13:11.760 --> 13:13.040 I'll need a smaller font, 13:14.320 --> 13:16.480 it prints the table like this, 13:16.480 --> 13:18.480 with the names of the methods, 13:18.560 --> 13:21.520 and then links to the source code of the functions. 13:22.080 --> 13:25.360 These links only make sense in Emacs and in Eevee, 13:26.000 --> 13:28.560 and when we run a link like this one, 13:29.840 --> 13:33.920 it shows the source code in the window at the right. 13:33.920 --> 13:37.760 So, for some functions, the source code is three lines, 13:37.760 --> 13:40.400 for other ones it's one line, and whatever. 13:41.920 --> 13:43.200 Anyway, let me go back. 13:45.360 --> 13:48.000 Lua can be used in many different styles. 13:48.000 --> 13:50.160 Most people hate other people's styles. 13:51.920 --> 13:56.080 When I started using it in the year 2000, 13:56.080 --> 13:59.040 I learned most of the basic language in a single day. 13:59.040 --> 14:02.000 It was very similar to things that I was already using, 14:02.800 --> 14:08.560 and then I rewrote the mini language that I was using 14:08.560 --> 14:14.800 to generate the HTML for my pages in Lua. 14:14.800 --> 14:16.960 Actually, I had to rewrite it many times, 14:16.960 --> 14:20.880 but the first version I certainly did in my first weeks 14:20.880 --> 14:22.480 or first months using Lua. 14:25.520 --> 14:29.040 In the beginning, I was just using it for writing programs 14:29.040 --> 14:32.160 that either didn't take in any input at all, 14:32.160 --> 14:35.280 because the input was already in the source file, 14:35.280 --> 14:39.200 or that worked as a Unix function, 14:39.200 --> 14:42.960 the Unix programs that would read files 14:42.960 --> 14:46.160 and process these files in some way and output something. 14:47.840 --> 14:52.480 I mentioned the basic language here. 14:52.480 --> 14:55.520 I only learned how to use closures, metatables, 14:55.520 --> 14:58.240 and coroutines many years later. 15:00.320 --> 15:02.240 In the beginning, when I started using Lua, 15:02.240 --> 15:04.080 it didn't have a package manager. 15:04.080 --> 15:05.200 It appeared later. 15:05.200 --> 15:06.560 It is called Lua-rocks. 15:07.680 --> 15:11.600 It has had this package manager for several years. 15:12.720 --> 15:15.520 Most of the rocks for Lua-rocks are poorly documented, 15:16.480 --> 15:19.040 documented, and hacker-unfriendly, 15:19.040 --> 15:21.600 so you can't rely just on the documentation, 15:22.160 --> 15:24.320 and you can't rely just on the source code, 15:24.320 --> 15:28.320 because, I mean, if you're a genius, of course you can, 15:28.320 --> 15:32.320 but for people who are either lazy or dumb or whatever, 15:32.320 --> 15:33.920 like me, or unfocused, 15:35.120 --> 15:37.120 the source code is hard to understand 15:37.120 --> 15:38.800 and hard to tinker with. 15:41.440 --> 15:43.040 Some rocks are excellent. 15:43.040 --> 15:46.640 The best rocks are well documented, 15:46.640 --> 15:48.640 but they are hacker-unfriendly, 15:48.640 --> 15:51.840 in the sense that I hope that I'll be able to explain soon. 15:53.440 --> 15:58.560 The best rocks use local variables and metatables a lot, 15:59.280 --> 16:03.120 so if you are a beginner learning Lua, 16:03.120 --> 16:06.080 you're not going to understand what their source codes do. 16:06.080 --> 16:08.320 They use lots of dirty tricks. 16:08.640 --> 16:12.240 Let me talk a bit about object orientation in Lua. 16:12.240 --> 16:13.840 It can be done in many ways. 16:15.360 --> 16:18.000 The main book about Lua, called Programming in Lua, 16:18.000 --> 16:20.240 by one of the authors of the language, 16:21.120 --> 16:22.320 Roberto Ierzalimschi, 16:23.200 --> 16:26.960 presents several ways of doing object orientation in Lua. 16:26.960 --> 16:29.040 I hated all of these ways, 16:29.840 --> 16:32.720 and also the ways that I tried from the rocks. 16:34.400 --> 16:36.880 And then I wrote my own way, 16:37.360 --> 16:40.160 wrote my own way of doing object orientation in Lua. 16:40.160 --> 16:41.440 It's very minimalistic. 16:42.000 --> 16:45.120 It's in this file here, eoo.lua. 16:47.040 --> 16:49.440 The main code is just these five lines here, 16:50.640 --> 16:53.120 and here's an example of how it works. 16:56.080 --> 17:02.640 Here we define a class vector with some metamethods. 17:02.640 --> 17:07.920 This metamethod here will tell Lua what to do 17:07.920 --> 17:11.040 when the user asks to add two vectors. 17:11.600 --> 17:14.320 This one here tells Lua what to do 17:14.320 --> 17:18.320 when the user asks Lua to convert a vector to a string, 17:18.880 --> 17:20.640 and whatever. 17:20.640 --> 17:23.920 This one is something that I'm going to explain in a second. 17:25.360 --> 17:28.240 So here we create a vector with these coordinates, 17:28.240 --> 17:29.360 three and four. 17:29.360 --> 17:31.360 Here we create another vector. 17:31.360 --> 17:35.200 If we print here, then Lua uses the function here 17:35.200 --> 17:36.160 in the toString. 17:37.200 --> 17:40.880 If we add two vectors, it uses the function here 17:40.880 --> 17:42.400 in the add metamethod. 17:42.960 --> 17:45.280 And if we run the method norm, 17:45.280 --> 17:49.760 it is defined here in the table index. 17:51.280 --> 17:51.780 Anyway. 17:52.740 --> 17:54.740 Even this thing being so small, 17:54.740 --> 17:58.500 I used to forget how it's in odds worked all the time. 17:58.500 --> 18:01.700 Actually, I always forget how things work, 18:01.700 --> 18:03.940 and I have to remember them somehow. 18:03.940 --> 18:07.940 And I have to have tricks for remembering 18:07.940 --> 18:11.940 and tricks for summarizing things and diagrams and so on. 18:13.540 --> 18:15.940 And every time that I forgot how to do it, 18:15.940 --> 18:17.220 I just go back to the code. 18:17.220 --> 18:18.420 I just go back to the code. 18:18.500 --> 18:23.140 And every time that I forgot how this thing worked, 18:23.140 --> 18:25.540 I went back to the source code, 18:25.540 --> 18:27.380 and then I looked at the diagrams. 18:27.380 --> 18:29.940 Or, of course, in the first time, 18:29.940 --> 18:31.460 I had to draw the diagrams. 18:32.260 --> 18:34.420 And I run the examples. 18:34.420 --> 18:35.780 And, of course, in the beginning, 18:35.780 --> 18:37.460 I thought that the code was clear. 18:37.460 --> 18:38.900 My examples were very brief. 18:38.900 --> 18:42.820 And so I had to rewrite the examples many times 18:42.820 --> 18:46.420 until they became, let's say, perfect. 18:48.660 --> 18:53.220 And I was saying that Lua can be used in many ways. 18:53.780 --> 18:57.700 And in my way of using Lua, my favorite way, 18:57.700 --> 19:02.020 everything can be inspected and modified from REPLs, 19:03.140 --> 19:07.140 like we can do in MX and in Smalltalk, or sort of. 19:07.860 --> 19:09.940 So in my favorite way of using Lua, 19:09.940 --> 19:11.860 there's no security at all. 19:11.860 --> 19:14.740 Everything can be changed at all times. 19:15.140 --> 19:16.980 Of course, most people hate that. 19:17.780 --> 19:20.180 My init file has lots of classes. 19:20.820 --> 19:24.100 And, by the way, instead of keeping many small files 19:24.100 --> 19:25.140 with many things, 19:25.140 --> 19:29.300 I put lots of stuff in just one big init file. 19:31.060 --> 19:33.860 My init file has lots of classes 19:33.860 --> 19:36.660 and lots of global functions and lots of crafts. 19:36.660 --> 19:40.020 So people hate that, of course. 19:41.380 --> 19:42.580 This is an example. 19:43.460 --> 19:44.500 This is an example. 19:44.500 --> 19:48.260 This is the index at the top of my init file. 19:50.580 --> 19:52.820 The classes start here. 19:55.060 --> 19:57.220 And then we have some functions. 19:57.780 --> 20:02.500 And then we have functions that load certain packages. 20:02.500 --> 20:03.540 And then we have craft. 20:04.260 --> 20:04.760 Whatever. 20:06.180 --> 20:08.580 Most people think that my style of using Lua 20:08.580 --> 20:10.340 is dirty and dangerous. 20:10.340 --> 20:11.860 And they wouldn't touch my Lua code. 20:12.100 --> 20:13.140 With a 10-feet pole. 20:14.740 --> 20:18.100 But most of the things that I'm going to present here 20:18.100 --> 20:20.740 in this presentation are ideas that should be easy 20:20.740 --> 20:24.180 to port to other environments and other languages. 20:24.180 --> 20:25.780 Especially the diagrams. 20:25.780 --> 20:27.620 So the code is not so important. 20:30.740 --> 20:33.140 Now let me talk a bit about LaTeX. 20:33.140 --> 20:33.860 Lua LaTeX. 20:33.860 --> 20:38.820 That is LaTeX with the Lua interpreter embedded inside. 20:39.380 --> 20:43.300 And two ways of generating pictures in LaTeX. 20:43.300 --> 20:44.820 Text that is very famous. 20:44.820 --> 20:47.300 And picture that is not very famous. 20:47.300 --> 20:48.900 And that is very low level. 20:48.900 --> 20:51.700 And I think that not many people use. 20:53.700 --> 20:57.300 I said before that when I learned Lua, 20:57.300 --> 21:01.540 I realized that it was very good for writing little languages. 21:03.220 --> 21:05.220 I was doing my PhD at the time. 21:05.300 --> 21:11.220 And typesetting the diagrams for my PhD thesis was very boring. 21:11.220 --> 21:14.100 So one of the things that I did was that I created 21:14.980 --> 21:17.540 a little language for typesetting the diagrams for me. 21:19.140 --> 21:20.900 It was called DEDNOT. 21:20.900 --> 21:25.300 Because initially it only generated diagrams 21:25.300 --> 21:26.820 for natural deduction. 21:26.820 --> 21:28.580 And then it had several versions. 21:30.180 --> 21:34.420 These are the slides for my presentation about DEDNOT6. 21:34.420 --> 21:39.140 And DEDNOT6 is an extensible semi-preprocessor for Lua LaTeX 21:39.140 --> 21:41.780 that understands diagrams in ASCII art. 21:42.340 --> 21:48.740 In the sense that when I have a LaTeX file that has this. 21:49.380 --> 21:51.860 And when DEDNOT6 is loaded. 21:53.220 --> 21:58.820 When I give the right commands, DEDNOT6 interprets this block here 21:58.820 --> 22:00.980 as something that defines this diagram. 22:02.260 --> 22:03.540 Oops, sorry. 22:05.380 --> 22:07.540 It interprets this diagram here. 22:08.340 --> 22:10.900 This diagram in the comments here 22:11.460 --> 22:14.260 as something that defines a diagram called foo, 22:14.260 --> 22:16.100 a deduction called foo. 22:16.100 --> 22:19.140 And it generates this code here. 22:19.940 --> 22:27.300 So that we can just invoke the definition of the deduction 22:27.300 --> 22:30.260 by typing backslash DED foo. 22:31.220 --> 22:37.140 And DEDNOT6 also supports another language 22:37.140 --> 22:39.860 for typesetting bi-dimensional diagrams 22:39.860 --> 22:42.660 with arrows and stuff for category theory and blah blah. 22:44.180 --> 22:46.900 The specifications of these diagrams look like this. 22:49.540 --> 22:53.540 Here is a very good example. 22:55.540 --> 22:56.980 This is a huge diagram. 22:58.420 --> 22:59.460 Sorry, one second. 23:00.740 --> 23:03.620 So the source code that generates this diagram here 23:03.620 --> 23:05.140 is just this thing at the left. 23:07.220 --> 23:08.340 So it's very visual. 23:09.220 --> 23:12.340 We can typeset the diagram in ASCII art here. 23:12.340 --> 23:13.780 And then in this part here, 23:13.780 --> 23:16.660 we tell how the nodes are to be joined, 23:17.620 --> 23:20.660 which arrows have to have annotations and so on. 23:22.420 --> 23:25.460 And this language is extensible in the sense that... 23:25.780 --> 23:26.580 Where is that? 23:32.020 --> 23:32.520 Here. 23:34.420 --> 23:36.660 Comments that start with percent colon 23:37.940 --> 23:41.620 are interpreted as definitions for three diagrams. 23:43.620 --> 23:47.220 Lines that start with percent uppercase D 23:47.220 --> 23:52.260 define 2D diagrams with arrows and stuff. 23:52.740 --> 23:58.020 And lines that start with comment uppercase L 23:58.660 --> 24:00.580 contain blocks of Lua code 24:00.580 --> 24:04.180 that we can use to extend the interpreter on the flag. 24:06.020 --> 24:09.860 Anyway, here are some recent examples of diagrams 24:09.860 --> 24:14.580 that I used DEDNOT6 to typeset. 24:15.780 --> 24:17.220 This diagram here 24:17.780 --> 24:20.580 was generated by this specification here. 24:23.460 --> 24:26.980 And this diagram here with the curved arrows 24:27.780 --> 24:30.580 was generated by this specification here. 24:34.180 --> 24:37.220 So DEDNOT6 was very easy to extend. 24:37.220 --> 24:38.260 At that some point, 24:38.260 --> 24:42.820 I started to use it to generate diagrams using Peaked Chewy, 24:42.820 --> 24:45.540 mainly for the classes that I gave you. 24:45.540 --> 24:48.420 For the classes that I give at the university. 24:48.420 --> 24:50.900 I teach mathematics and whatever. 24:51.860 --> 24:52.900 In a bad place. 24:52.900 --> 24:53.400 Whatever. 24:56.260 --> 24:58.500 Let me show an animation. 24:59.060 --> 25:02.420 Here is a diagram that I generated with DEDNOT6. 25:02.980 --> 25:05.300 And it is a flipbook animation. 25:05.300 --> 25:08.740 Like we type page up and page down 25:08.740 --> 25:10.660 and we go to the next page of the book 25:10.660 --> 25:12.340 and to the previous page of the book. 25:13.060 --> 25:15.940 And here is the source code that generates that. 25:16.980 --> 25:19.140 This source code is not very visual. 25:19.140 --> 25:22.740 So it's quite clumsy to edit a diagram 25:22.740 --> 25:25.460 directly in the tag file like that. 25:28.020 --> 25:31.860 These diagrams were inspired by something called Manim. 25:32.500 --> 25:33.000 That's... 25:34.580 --> 25:36.340 Oh, I've forgotten the name of the guy. 25:36.340 --> 25:39.300 But it's a guy that makes many videos about mathematics. 25:39.300 --> 25:41.780 And he created this library called Manim 25:41.780 --> 25:43.460 for generating his animations. 25:45.540 --> 25:51.460 Other people adapted his library to make it more accessible. 25:52.580 --> 25:53.700 I tried to learn it. 25:53.700 --> 25:55.700 But each animation, 25:57.060 --> 25:59.220 even an animation with very few frames, 25:59.220 --> 26:01.300 each animation took ages to render. 26:01.300 --> 26:03.060 So it wasn't fun. 26:03.700 --> 26:07.060 And animations and PDFs can be rendered in seconds. 26:07.940 --> 26:11.060 So these things were fun for me 26:11.060 --> 26:15.300 because my laptop is very slow and Manim was not fun. 26:20.100 --> 26:26.020 Anyway, writing code like this inside the attack file 26:26.020 --> 26:29.620 was not very fun because it was hard to debug. 26:30.500 --> 26:35.860 So in 2022, I started to play with ways 26:35.860 --> 26:40.340 of generating these diagrams from REPLs. 26:40.980 --> 26:44.500 And I found a way for PIC2E and a way for TIX. 26:45.380 --> 26:47.780 Each one of these ways became a video. 26:48.740 --> 26:53.780 If you go to the list of first class videos of EEV, 26:53.780 --> 26:57.540 you're going to see that there's a video about PIC2E here 26:57.540 --> 26:58.740 and a video about TIX. 27:00.340 --> 27:07.540 Here you have some information like length and explanation, etc. 27:08.180 --> 27:10.420 And here are the pages for these videos. 27:12.420 --> 27:15.860 My page about the video, about PIC2E is like this. 27:15.860 --> 27:18.740 It has some diagrams, whatever. 27:18.740 --> 27:20.980 And this one is much nicer. 27:20.980 --> 27:26.100 And a lot of people watched that video. 27:26.100 --> 27:30.260 I mean, I thought that 250 people watched it. 27:30.900 --> 27:33.940 For me, that's a million of people. 27:36.660 --> 27:42.900 And this video is about how to extract diagrams from the TIX manual 27:42.900 --> 27:47.700 and how to run those examples in a REPL 27:47.700 --> 27:49.780 and modify them bit by bit. 27:49.780 --> 27:51.380 This is a screenshot. 27:52.100 --> 27:53.620 But let me go back. 27:56.420 --> 27:59.700 At that point, these things were just prototypes. 27:59.700 --> 28:01.460 The code was not very nice. 28:02.020 --> 28:10.500 And in this year, I was able to unify those two ways of generating PDFs, 28:11.380 --> 28:14.100 the one for TIX and the one for PIC2E. 28:14.100 --> 28:18.740 And I unified them with many other things that generated diagrams. 28:19.220 --> 28:27.460 The basis of these things is something called showchew.lua. 28:27.460 --> 28:31.380 I'm not going to show its details now. 28:32.100 --> 28:40.420 But its extension that generates TIX code is just this. 28:40.420 --> 28:45.540 So we can specify a diagram with just a block like this. 28:46.260 --> 28:54.180 And then if we run show00 and it returns a string 28:54.180 --> 28:58.580 that is just the inner body of the TIX file. 28:59.220 --> 29:02.020 If we run this, we see the whole TIX file. 29:02.020 --> 29:04.340 And if we run this, we save the TIX file 29:04.340 --> 29:08.420 and we compile the TIX file to generate a PDF. 29:08.420 --> 29:14.180 And if we run this, we show the PDF in the lower right window. 29:16.020 --> 29:20.740 And that's the same thing for all my recent programs that generate PDFs. 29:21.620 --> 29:22.980 They are all integrated. 29:24.500 --> 29:30.660 Here is the one that the basis for all my modules that generate diagrams with PIC2E. 29:33.060 --> 29:34.980 Its demos are not very interesting. 29:34.980 --> 29:39.700 So let me show some demos of extensions that do interesting things. 29:40.660 --> 29:47.220 So this is a diagram that I created by editing it in a REPL. 29:48.820 --> 29:51.220 I create several PIC2E objects here. 29:52.500 --> 29:58.660 And if I execute this, it compiles the object, generates a PDF. 29:58.660 --> 30:03.860 And if I tap this, here is the PDF. 30:04.740 --> 30:10.020 And if I just ask Lua to display what is books here, 30:11.700 --> 30:17.220 it shows the source code in PIC2E of the diagram. 30:17.940 --> 30:20.340 And the nice thing is that it is indented. 30:20.340 --> 30:23.460 So it's easy to debug the PIC2E code. 30:24.340 --> 30:30.180 If anyone is interested, the module that does the tricks for indentation 30:30.180 --> 30:31.460 is very easy to understand. 30:31.540 --> 30:34.500 It has lots of tests and test blocks. 30:34.500 --> 30:39.060 And I think that its data structures are easy to understand. 30:42.500 --> 30:44.820 Anyway, here is another example. 30:50.260 --> 30:51.620 The show is here. 30:53.460 --> 30:56.340 It generates a 3D diagram. 31:02.420 --> 31:08.500 Now let me talk about parsers and REPLs in a very strange place. 31:09.140 --> 31:18.340 I mean, using REPLs to build parsers step by step and replacing parts by more complex parts. 31:21.060 --> 31:23.780 So I said that Lua is very minimalistic. 31:25.780 --> 31:30.660 And everybody knows that implementations of regular expressions 31:30.740 --> 31:31.780 are big and complex. 31:33.060 --> 31:39.220 So instead of coming with full regular expressions, Lua comes with something called patterns 31:39.220 --> 31:43.620 and a library function called string.match. 31:45.380 --> 31:54.420 Here is a copy of the part of the manual that explains a part of the syntax of patterns. 31:56.260 --> 32:00.100 Here's how string.match is described in the manual. 32:01.220 --> 32:02.500 It's just this. 32:02.500 --> 32:06.260 Looks for the first match of pattern in the string as blah, blah, blah. 32:07.220 --> 32:11.300 And then we have to go to the other section of the manual that explains patterns. 32:17.780 --> 32:26.500 Lua patterns are so simple, so limited, that they don't even have the alternation operator. 32:27.220 --> 32:31.460 Here is how it is described in the ELLISP manual. 32:33.300 --> 32:37.780 Backslash pipe specifies an alternative, blah, blah, blah. 32:40.820 --> 32:47.940 When we want to build more complex regular expressions, patterns, grammars, etc., 32:47.940 --> 32:51.780 we have to use an external library for that. 32:52.740 --> 32:59.140 No, sorry, a library that is external, but that was written by one of the authors of Lua itself. 33:00.340 --> 33:05.380 This library is called LPEG, and its manual says, 33:06.180 --> 33:11.940 LPEG is a new pattern matching library for Lua based on parsing expression grammars, PEGs. 33:14.900 --> 33:17.460 The manual is very terse. 33:18.180 --> 33:20.500 I found it incredibly hard to read. 33:20.900 --> 33:22.740 It doesn't have any diagrams. 33:22.740 --> 33:24.420 It has some examples, though. 33:26.180 --> 33:34.420 And the Lua wiki has a big page called LPEG tutorial with lots of examples. 33:35.700 --> 33:42.020 But it also doesn't have diagrams, and I found some things incredibly hard to understand. 33:42.020 --> 33:48.500 For example, this is something that is in the manual of LPEG that I saw and I thought, 33:49.460 --> 33:53.060 wow, great, this makes all sense, and this is going to be very useful. 33:53.060 --> 33:57.300 It's a way to build grammars that can be recursive, 33:57.300 --> 34:02.020 and they sort of can encode BNF grammars. 34:02.020 --> 34:06.980 We just have to translate the BNF a bit to get rid of some recursions 34:06.980 --> 34:08.820 and to translate them to something else. 34:10.580 --> 34:13.620 And the manual also has some things that I thought, 34:13.620 --> 34:16.580 oh, no, I don't have any idea of what this thing does. 34:17.380 --> 34:22.020 And in fact, I saw these things for the first time more than 10 years ago, 34:22.020 --> 34:26.020 and they only started to make sense one year ago. 34:28.420 --> 34:30.340 One example is group captures. 34:33.380 --> 34:37.620 LPEG also comes with a module called the RE module. 34:37.620 --> 34:40.180 Let me pronounce it in Portuguese, the RE module. 34:40.660 --> 34:45.540 And its manual says, the RE module provided by the file repo.luen 34:45.540 --> 34:48.100 in the distribution supports a somewhat conventional 34:48.980 --> 34:54.420 regular expression syntax for pattern usage within LPEG. 34:55.140 --> 34:57.460 And this is a quick reference. 35:00.500 --> 35:02.820 And this thing is very brief. 35:02.820 --> 35:06.420 It has some nice examples, but it's hard to understand in a way. 35:06.660 --> 35:12.500 And here are some comments about my attempts to learn HerdotLua. 35:14.020 --> 35:15.220 This is a class. 35:15.220 --> 35:17.060 In this case, it's a very small class. 35:18.180 --> 35:21.940 And this file implements a PM method. 35:23.540 --> 35:26.980 I'm going to show examples of other PM methods very soon. 35:28.020 --> 35:33.380 So this is a PM method for HerdotLua that lets us compare the syntax 35:33.620 --> 35:39.460 of Lua patterns, LPEG, and HerdotLua. 35:41.620 --> 35:43.060 See this example here. 35:43.700 --> 35:49.940 So if we run this, it loads my version of LPEG. 35:50.660 --> 35:52.740 No, sorry, my version of LPEG-REX. 35:54.820 --> 36:00.180 And it shows that when we apply the PM method to this Lua pattern, 36:00.740 --> 36:07.380 this LPEG pattern, and this RE pattern, they all get the same results. 36:07.380 --> 36:12.820 So we can use this thing, this kind of thing here to show how to translate from 36:14.420 --> 36:20.180 Lua patterns that are familiar because they are similar to regular expressions, only weaker, 36:22.260 --> 36:27.620 to LPEG that is super weird, and to RE that is not so weird. 36:28.580 --> 36:37.940 Anyway, the comment says that in 2012, I had a project that needed a precedence parser that 36:37.940 --> 36:42.900 could parse arithmetical expressions with the right precedences. 36:44.580 --> 36:50.180 And at that point, I was still struggling with pure LPEG, and I couldn't do much with it. 36:50.180 --> 36:55.300 So I tried to learn HerdotLua instead, and I wrote this old class here. 36:56.260 --> 36:59.620 That allowed me to use preprocessor on patterns for Lua. 36:59.620 --> 37:04.980 And the thing is that with this preprocessor, I could specify precedence grammars using this 37:04.980 --> 37:12.900 thing here that worked, but it was super clumsy, and I gave up after a few attempts. 37:14.900 --> 37:21.300 And in 2022, I heard about something called LPEG-REX that was an 37:22.260 --> 37:28.740 a kind of extension of Re, and it was much more powerful than HerdotLua, but after a while, 37:28.740 --> 37:32.660 I realized that it had the same defects as HerdotLua. 37:32.660 --> 37:41.620 And let me explain that because it has all to do with the things about black boxes and magic 37:41.620 --> 37:42.980 that I told in the beginning. 37:43.380 --> 37:50.100 Both, I mean, sorry, neither HerdotLua or LPEG-REX had some features that I needed. 37:51.620 --> 37:57.940 They didn't let us explain, sorry, they received a pattern that was specified as a string, 37:57.940 --> 38:04.740 and it converted that into an LPEG pattern, but it didn't let us explore the LPEG patterns 38:04.740 --> 38:05.540 that it was using. 38:05.540 --> 38:12.900 So I had to use the LPEG-REX, and it didn't let me explore the LPEG patterns that it was 38:12.900 --> 38:13.540 generated. 38:16.420 --> 38:21.060 Their code was written in a way that was REPL unfriendly. 38:21.060 --> 38:28.580 I couldn't modify parts of the code bit by bit in a REPL and try to change the code 38:29.700 --> 38:31.620 without changing the original file, say. 38:33.300 --> 38:37.220 The code was very hard to explore, to hack, and to extend, in my opinion. 38:37.780 --> 38:39.700 The documentation was not very clear. 38:40.580 --> 38:50.660 And I sent one or two messages to the developer of LPEG-REX, and he was too busy to help me. 38:50.660 --> 38:55.460 He answered very briefly, and to be honest, I felt rejected. 38:55.460 --> 38:58.580 I felt that I wasn't doing anything interesting. 38:58.580 --> 38:59.380 Whatever, whatever. 39:00.340 --> 39:11.940 So, in 2022, I was trying to learn LPEG-REX, because I was thinking that it would solve 39:11.940 --> 39:13.780 my problems, but it didn't. 39:14.500 --> 39:20.900 It didn't have the features that I needed, and it was hard to extend, and hard to explore, 39:20.900 --> 39:22.100 and hard to debug. 39:22.980 --> 39:32.420 I decided to rewrite it in a more hacker-friendly way, in the sense that it was modular, and 39:32.420 --> 39:35.460 I could replace any part of the module from a REPL. 39:37.300 --> 39:48.340 My version of it was called lpeg1.lua, and I decided that in my version, I wouldn't have 39:49.060 --> 39:55.460 the part that receives a grammar specified as a string and converts that to LPEG. 39:55.460 --> 40:03.780 I would just have the backend part that are the functions in LPEG that let us specify 40:04.340 --> 40:05.380 powerful grammars. 40:10.340 --> 40:11.540 So, let me go back. 40:12.260 --> 40:14.260 Let me explain a bit about LPEG. 40:14.900 --> 40:16.180 Lua has coercions. 40:18.660 --> 40:22.740 The plus expects to receive two numbers. 40:22.740 --> 40:29.860 If one of its arguments, or both of them, are strings, it converts the strings to numbers. 40:29.860 --> 40:39.220 So, in this case here, 2 plus string 3 returns the number 5, and this is the concatenation 40:39.220 --> 40:39.860 operator. 40:40.820 --> 40:42.820 It expects to receive strings. 40:43.780 --> 40:49.860 So, in this case, it will convert the number 2 to the string 2, and the concatenation of 40:49.860 --> 40:51.700 these two things will be 23. 40:52.420 --> 40:54.420 Sorry, 23 as a string. 40:56.260 --> 40:58.420 LPEG also has some coercions. 41:00.260 --> 41:09.620 I usually set these globals to let me write my grammars in a very compact way. 41:10.260 --> 41:17.460 So, instead of lpeg.p, lpeg.c, etc., I use these globals like uppercase B, uppercase 41:17.460 --> 41:18.420 C, and so on. 41:19.540 --> 41:27.540 And with these globals, I can write things like this, c1 times string underscore. 41:28.420 --> 41:40.820 And LPEG knows that lpeg.c, sorry, it sort of expands these to lpeg.c, but lpeg.c expects 41:40.820 --> 41:47.380 to receive an LPEG pattern, and one is not yet an LPEG pattern, so it is coerced into 41:47.380 --> 41:50.900 an LPEG pattern by calling lpeg.p. 41:51.220 --> 42:02.740 So, this shorting here becomes equivalent to lpeg.c, lpeg.p1, and the multiplication, 42:02.740 --> 42:09.940 when at least one of its arguments is an LPEG pattern, it expects to receive two LPEG patterns, 42:09.940 --> 42:15.700 and in this case, the one at the right is just a string, so it is coerced to an LPEG 42:15.700 --> 42:17.620 pattern by using lpeg.p. 42:18.180 --> 42:22.180 With this idea, we can sort of understand this comparison here. 42:22.900 --> 42:24.980 I mean, let me run it again. 42:24.980 --> 42:33.620 This first part is very similar to a regular expression here at the left, and when we apply 42:33.620 --> 42:45.940 this LPEG, sorry, this Lua pattern to this subject here, the result is that the Lua pattern 42:46.740 --> 42:53.540 the result is this thing here, this thing, this thing, and this thing. 42:53.540 --> 43:00.580 I'm going to call each one of these results captures, so each of these things between 43:00.580 --> 43:08.020 parentheses captures a substring of the original string, and these captured substrings are 43:08.020 --> 43:09.460 returned in a certain order. 43:10.420 --> 43:12.740 Here is how to express the same thing in LPEG. 43:13.540 --> 43:22.660 It's very cryptic, but it's a good way to understand some basic operators of LPEG. 43:22.660 --> 43:34.900 I mean, we can look at the menu and understand what C, S, and R do, and also exponentiation. 43:35.860 --> 43:43.060 And this strange thing here receives this string here, runs a function that I have defined 43:43.060 --> 43:50.260 that converts it to an object of a certain class, and that class represents He patterns. 43:50.260 --> 43:56.900 So this thing is treated as a pattern for He.Lua, and it is matched against the string, 43:56.900 --> 43:59.300 and it returns the same thing as the other one. 43:59.860 --> 44:06.260 Also, this thing here also has a comparison with LPEG-REGS, but these patterns are very 44:06.260 --> 44:06.820 trivial. 44:06.820 --> 44:10.180 They don't do anything very strange. 44:10.180 --> 44:14.260 So let's go back and see what kinds of very strange things there are. 44:16.660 --> 44:23.620 Here is the page of LPEG-REGS at GitHub. 44:24.260 --> 44:25.860 Here's the documentation. 44:27.940 --> 44:29.380 It's relatively brief. 44:29.380 --> 44:35.300 It explains LPEG-REGS as being an extension of He.Lua. 44:36.500 --> 44:40.100 So it explains mainly the additional features. 44:40.100 --> 44:44.180 Here is a quick reference that explains only the additional features. 44:46.820 --> 44:53.380 Some of these things I was able to understand by using the LPEG-REGS. 44:54.340 --> 45:02.820 I was struggling a lot, and some I wasn't able to, even by spending several evenings 45:02.820 --> 45:04.420 trying to build examples. 45:08.260 --> 45:10.900 And this is something very nice. 45:12.100 --> 45:19.700 LPEG-REGS comes with some example parsers, and here is a parser that parses the Lua grammar. 45:19.700 --> 45:28.100 I mean, this is the grammar for Lua 5.4 at the end of the reference manual. 45:28.660 --> 45:35.860 It's just this, and this is a kind of the BNF, and this is the BNF translated to 45:37.300 --> 45:39.860 the language of LPEG-REGS. 45:40.420 --> 45:47.300 So this thing uses many constructions that are in He.Lua and some extra constructions 45:47.300 --> 45:49.460 that are described here. 45:50.260 --> 45:58.980 And with these examples, I was able to understand some of these things here that are described 45:58.980 --> 46:02.660 here in the quick reference, but not all. 46:06.660 --> 46:15.300 So I wasn't able to use LPEG-REGS by itself because some things didn't make much sense, 46:15.300 --> 46:18.900 and I decided to reimplement it in my own style. 46:20.900 --> 46:27.860 Because that would be a way to map, at the very least, map what I understood and what 46:27.860 --> 46:32.980 I didn't, and learn one feature at a time, do comparisons, and so on. 46:35.380 --> 46:38.900 Here, I pointed to two features of LPEG. 46:38.900 --> 46:44.820 One, I said, oh great, this thing can be used to define grammars, even recursive 46:44.900 --> 46:46.020 grammars, and so on. 46:46.740 --> 46:52.820 And this is an oh-no feature, one thing that didn't make any sense at all, group captures. 46:55.380 --> 47:01.620 One thing that I did to understand group captures was to represent them as diagrams. 47:01.620 --> 47:08.020 Of course, in the beginning, I was drawing these diagrams by hand, but then I realized 47:08.020 --> 47:15.220 that I could use the bits of LPEG that I already knew to build a grammar that would 47:15.220 --> 47:19.380 parse a little language and generate these diagrams in LaTeX. 47:20.340 --> 47:21.940 And I was able to make this. 47:24.180 --> 47:34.500 In this diagram here, this thing above the arrow is a piece of Lua code that specifies 47:34.500 --> 47:36.100 an LPEG pattern. 47:36.820 --> 47:43.540 This thing here at the top is the string that is being matched, and the things below the 47:43.540 --> 47:51.220 under braces are the captures that each thing, sorry, each thing captures. 47:54.580 --> 48:02.100 And for example, this under brace here corresponds to this pattern here that parses a single 48:02.100 --> 48:05.060 character but doesn't return any captures. 48:05.140 --> 48:10.420 This thing here parses a single B and doesn't return any captures. 48:10.420 --> 48:16.740 This thing here parses a single character and captures it, and this thing here parses 48:17.620 --> 48:19.700 the character D and captures it. 48:20.340 --> 48:29.220 And this other thing here that transforms this pattern into another pattern returns 48:29.780 --> 48:36.500 first a capture with all the string that was parsed by this pattern here, and then all 48:36.500 --> 48:41.380 the captures returned by this thing here before the column. 48:43.380 --> 48:50.900 So this was a way to build concrete examples for things that the LPEG manual was explaining 48:50.900 --> 48:53.940 in a very terse way, and it worked for me. 48:54.740 --> 49:01.540 Some things that were very mysterious started to make sense, and I started to have intelligent 49:01.540 --> 49:03.300 questions to ask in the mailing list. 49:07.700 --> 49:16.980 And with that, I was able to understand what are group captures and group captures that 49:16.980 --> 49:17.860 receive a name. 49:20.900 --> 49:23.140 Well, let me explain what this does. 49:23.220 --> 49:30.180 This thing here captures, sorry, parses the empty string and returns this as a constant. 49:30.180 --> 49:35.860 So this is something that doesn't exist in regular expressions. 49:36.980 --> 49:40.180 It parses nothing and returns this as a capture. 49:40.820 --> 49:48.580 Then this thing here returns these two constants here and parses the empty string, and this 49:48.660 --> 49:56.500 thing here, d, converts the results of this thing here into a group capture and stores 49:56.500 --> 49:58.020 it in the label d. 50:01.380 --> 50:03.860 And then here's another constant capture. 50:03.860 --> 50:11.460 And I realized that these things here were similar to how Lua specifies building lists. 50:11.780 --> 50:16.580 When we build, sorry, a table, when we build a table and we say that the first element 50:16.580 --> 50:20.420 of the table is here, this element is put at the end of the table. 50:21.140 --> 50:30.820 When after that we say d equals to, say, 42, we are putting the 42 in the slot whose key 50:30.820 --> 50:31.380 is d. 50:33.380 --> 50:37.860 This was happening with LPEG captures, but there was something very strange. 50:37.860 --> 50:46.260 These group captures could hold more than one capture, more than one value. 50:46.260 --> 50:50.340 So there was something between lists and tables. 50:51.300 --> 51:00.740 I started to use this notation to explain in my notation what they were doing. 51:02.340 --> 51:04.900 Many things start, things stop. 51:05.220 --> 51:08.900 Many things start, things started to make sense. 51:08.900 --> 51:14.420 Many mysterious sentences in the manual started to make sense, but some didn't. 51:16.100 --> 51:25.060 But at least I was able to send some intelligent questions to the mailing list, and the author 51:25.060 --> 51:27.540 of Lua and LPEG answered some of them. 51:29.700 --> 51:31.860 He was not very happy about my questions. 51:32.020 --> 51:40.100 He told me that those diagrams were a waste of time, that the manual was perfectly clear, 51:40.100 --> 51:42.900 and so on, whatever. 51:42.900 --> 51:51.620 But I was able to, so it was weird, but I was able to understand lots of things from 51:51.620 --> 51:52.340 his answers. 51:53.940 --> 51:57.380 So this is a copy of one of my messages. 51:57.380 --> 51:59.300 Then there's another one, another one. 51:59.300 --> 52:04.580 Some were the diagrams, then he complained about these diagrams. 52:04.580 --> 52:12.980 He said that these things here that look like table constructions do not exist, whatever. 52:15.380 --> 52:23.220 Anyway, once I understood group captures, many features were very, very easy to understand, 52:23.780 --> 52:28.260 and I started to be able to use LPEG to build some very interesting things. 52:30.260 --> 52:36.340 I was able to reproduce some of the features that I saw in LPEG-REX. 52:38.420 --> 52:40.980 Remember that this, where is that? 52:42.420 --> 52:44.020 This is a syntax of Lua. 52:45.860 --> 52:54.820 Here, I was able to understand how these things here were translated to LPEG code, to LPEG 52:54.820 --> 52:58.340 patterns by using group captures in a certain way. 52:59.460 --> 53:05.060 And I was able to implement them in lpeg1.lua. 53:06.340 --> 53:14.180 And after some time, I was able to use lpeg1.lua to build grammars that were able to parse 53:17.300 --> 53:19.860 arithmetical expressions with the right precedence. 53:19.860 --> 53:25.700 And here's an example in which I built the grammar step by step, and I test the current 53:25.700 --> 53:29.300 grammar, and I replace a bit, and then I test the new grammar, and so on. 53:31.780 --> 53:38.580 And you can see that the result is always a tree that is drawn in a nice two-dimensional way. 53:42.580 --> 53:52.340 At this point, these powers here are returned as a list, as an operation 53:52.900 --> 53:56.260 pow with several arguments here. 53:56.820 --> 54:04.820 And then I apply a kind of parsing combinator here that transforms these trees into other trees. 54:05.540 --> 54:14.180 And with these combinators here, I can specify that the power is associative in a certain 54:14.180 --> 54:20.340 direction, the division is associative in another direction, the minus is associative, 54:20.420 --> 54:24.420 so it uses the same direction as a division, and so on. 54:24.420 --> 54:26.020 And they have the right precedences. 54:28.660 --> 54:30.020 So here are the tests. 54:33.700 --> 54:37.140 So here's my file lpeg1.lua. 54:37.140 --> 54:38.340 It has several classes. 54:38.980 --> 54:42.980 Each class has tests after it. 54:43.940 --> 54:51.780 I was able to implement something that lpeg-reqs has that's called keywords. 54:51.780 --> 54:56.980 That's very useful for parsing programs in programming languages. 54:56.980 --> 55:06.580 I was able to implement something similar to the debugger, to the PEG debugger that lpeg uses, 55:07.220 --> 55:12.740 but I was frustrated by some limitations of that debugger, 55:12.740 --> 55:16.180 and I implemented my own that is much better. 55:17.700 --> 55:23.860 And let me show something else. 55:23.860 --> 55:32.900 I was able to translate a good part of the Lua parser here to lpeg1.lua. 55:33.700 --> 55:39.940 I haven't finished yet, but I have most of the translation here. 55:43.140 --> 55:50.420 And after having all that, I was able to build other grammars very quickly. 55:51.380 --> 55:59.220 Writing new parsers finally became fun, and here's one example that I showed in the beginning. 56:03.620 --> 56:11.220 If I remember correctly, I took a figure from the Wikipedia. 56:11.220 --> 56:17.220 I don't have its link now, but I specified a grammar that parses 56:18.500 --> 56:21.140 exactly the example that appears in the Wikipedia. 56:21.140 --> 56:28.100 So with my grammar, considering that the top-level entry is statement, 56:28.100 --> 56:36.900 when I parse this string here, the result is this tree. 56:38.100 --> 56:41.780 And I can do some operations on that. 56:41.780 --> 56:45.780 I can define how this thing is to be converted into LaTeX. 56:47.460 --> 56:51.700 I can define other operations that convert trees into other trees. 56:52.580 --> 56:54.980 And here are some tests of these operations. 56:58.340 --> 57:00.660 This is what I showed in the beginning. 57:00.660 --> 57:04.100 I'm not going to explain all the details of this thing now. 57:07.460 --> 57:14.260 This show converts this thing into LaTeX in the way specified by these instructions here. 57:15.380 --> 57:16.340 Let's say that... 57:19.540 --> 57:20.260 Well, whatever. 57:22.740 --> 57:23.540 Really, whatever. 57:24.020 --> 57:28.500 And here's the result, the LaTeX result. 57:34.900 --> 57:46.260 And these diagrams here are generated by this file here that defines a simple grammar that parses 57:46.260 --> 57:53.460 this thing here, and then LaTeX it in a certain way, and then also tests this thing. 57:54.100 --> 58:00.900 To check if this code here, that is Lua code that generates an LPEG grammar, 58:02.900 --> 58:08.740 parses this subject here and returns the expected result. 58:12.740 --> 58:15.140 So this is the code that I wanted to show. 58:15.700 --> 58:21.700 I wanted to show many more things, but I wasn't able to prepare them before the conference. 58:22.580 --> 58:29.060 And I hope that soon, for some value of soon, I'll be able to create 58:29.620 --> 58:37.060 REPL-based tutorials for LPEG here and lpeg1.lua, where LPEG is something very famous. 58:37.060 --> 58:39.460 Here is a module of LPEG. 58:40.500 --> 58:48.100 I could also do something like this for LPEG-REX, and lpeg1.lua is the thing that I wrote, 58:48.820 --> 58:58.900 the one that has tests in comments, and the tests usually generate trees, 58:58.900 --> 59:00.900 and sometimes they generate tag code. 59:03.300 --> 59:04.420 Yeah, so that's it. 59:04.420 --> 59:07.860 I wanted to present much more, but I wasn't able to prepare it. 59:07.860 --> 59:08.660 So sorry. 59:08.660 --> 59:09.140 Thanks. 59:09.140 --> 59:10.260 Bye.