diff options
Diffstat (limited to '2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt')
| -rw-r--r-- | 2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt | 2219 |
1 files changed, 2219 insertions, 0 deletions
diff --git a/2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt b/2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt new file mode 100644 index 00000000..3bd5b01b --- /dev/null +++ b/2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt @@ -0,0 +1,2219 @@ +WEBVTT + +00:00.000 --> 00:04.880 +Hi, my name is Eduard Duax, and the title of this talk is + +00:04.880 --> 00:09.680 +Rapples in Strange Places, Lua, LaTeX, LPEG, LPEG-REX, and TIX. + +00:10.720 --> 00:17.200 +I'm the author of an MX package called EEV, and this is a talk at the MXConf 2023 + +00:17.200 --> 00:20.880 +that is happening in December 2023 at the Internets. + +00:21.680 --> 00:25.920 +And this is one of the examples of diagrams that we are going to see. + +00:25.920 --> 00:27.680 +Let me show how I generate it. + +00:28.640 --> 00:32.240 +One second. I have to use a smaller font here. + +00:35.600 --> 00:41.920 +This is a file called parse32.lua. Let me go back to this block of tests again. + +00:42.720 --> 00:50.320 +And now if I run this, we get these outputs here at the right. + +00:52.000 --> 00:54.960 +And then in this line here, it generates a PDF. + +00:55.520 --> 01:01.440 +And if I type f8 here, it shows the PDF in the lower right window. + +01:04.240 --> 01:07.600 +Let me start by explaining briefly what is EEV. + +01:09.920 --> 01:14.560 +First, it's something that appeared by accident in the mid-90s. + +01:14.560 --> 01:19.920 +I explained this story in my presentation at the MXConf 2019. + +01:20.720 --> 01:25.520 +It's a package. It's an MX package that is part of ELPA. + +01:25.520 --> 01:31.440 +It has at least 10 users. Those are the ones that I know by name. + +01:33.360 --> 01:36.320 +EEV means MX Execute Verbosely. + +01:37.360 --> 01:42.640 +EEV is something that treats eval as the central feature of MX. + +01:43.680 --> 01:46.800 +EEV blurs the distinction between programmers and users, + +01:47.520 --> 01:50.080 +and it replaces the slogan, + +01:50.080 --> 01:55.360 +users should not be forced to see Lisp, that is something that Richard Stallman told me once, + +01:55.360 --> 01:59.280 +by users should see Lisp instead of buttons, + +01:59.280 --> 02:02.400 +and new users should see Lisp in the first five minutes. + +02:04.240 --> 02:07.120 +I'm going to show some examples of that soon. + +02:08.800 --> 02:12.880 +EEV uses code in comments a lot, and also tests in comments. + +02:13.600 --> 02:21.040 +I changed my way of presenting it, and it became very REPL-centric in the last few years, + +02:21.040 --> 02:28.240 +in the sense that I start by explaining its main features by its support for REPLs. + +02:30.320 --> 02:34.320 +EEV supposes that we want to keep executable notes of everything. + +02:34.320 --> 02:37.040 +I'm also going to show examples of this in a second. + +02:38.080 --> 02:41.600 +EEV has lots of videos for people who hate videos, + +02:42.240 --> 02:47.520 +and it tries to do everything with very little magic and without black boxes. + +02:47.520 --> 02:50.240 +I'm going to explain many of these things very soon. + +02:52.880 --> 02:58.160 +This is a figure that I'm going to show in detail soon, + +02:58.160 --> 03:01.520 +that is about something important about Lua. + +03:01.520 --> 03:05.280 +Oops, the font is very bad now, so let me change the font. + +03:05.280 --> 03:07.840 +The figure is this one, and + +03:08.560 --> 03:13.440 +what most people do when they visit a file with something interesting on it + +03:13.440 --> 03:16.640 +is that they just go there, and they set a bookmark there, + +03:16.640 --> 03:19.760 +or they put the position in the register. + +03:21.760 --> 03:28.480 +But I prefer to keep links to everything that is interesting as a list of hyperlinks. + +03:28.480 --> 03:32.640 +So, for example, this is an at least hyperlink to the file + +03:32.640 --> 03:36.480 +that goes to this anchor here, and to this string I've added. + +03:36.480 --> 03:38.560 +And to this string after this anchor. + +03:39.760 --> 03:45.760 +This is a variant that opens that file in the window at the right here. + +03:47.440 --> 03:53.520 +And this is a sexpid that changes the font. + +03:53.520 --> 03:57.680 +I have a command with a very short name that does that, + +03:58.400 --> 04:02.880 +but I prefer to keep that as a one-liner. + +04:03.840 --> 04:11.440 +About the videos, we can see the list of first-class videos of Eevee by executing this, + +04:11.440 --> 04:16.960 +meta-x, find first-class videos, or by running this alias here, + +04:16.960 --> 04:20.320 +meta-x1c, and then what we see is this. + +04:21.520 --> 04:25.440 +The first sexpid here regenerates this buffer, + +04:25.440 --> 04:28.960 +so we can make a mess here and then run this, + +04:29.040 --> 04:33.040 +and the original buffer is regenerated again in a clean way. + +04:34.880 --> 04:42.000 +Each of these things here opens a buffer with information about a video. + +04:42.000 --> 04:44.400 +Let me take a specific example here. + +04:46.080 --> 04:50.720 +This video here is about one of the ancestors of this talk, + +04:50.720 --> 04:52.960 +that is a library that I wrote for + +04:53.920 --> 05:01.120 +creating diagrams in LaTeX using a package called pic2e, using repls. + +05:02.320 --> 05:10.720 +Anyway, the thing is that if we run a sexpid like this one, + +05:10.720 --> 05:13.440 +and we don't have a local copy of the video, + +05:14.080 --> 05:16.320 +Eevee will try to load the local copy, + +05:16.320 --> 05:19.600 +and instead of doing that and by asking something like, + +05:19.680 --> 05:20.720 +asking something like, + +05:20.720 --> 05:24.560 +do you want me to download the local copy, blah, blah, blah, blah, + +05:25.680 --> 05:28.720 +it simply opens a buffer like this. + +05:28.720 --> 05:32.480 +I mean, if we don't have a local copy yet, + +05:32.480 --> 05:34.400 +it will open a buffer like this one, + +05:35.440 --> 05:41.760 +in which these things here in comments are links to the documentation. + +05:41.760 --> 05:48.240 +I mean, this thing here explains the idea of local copies of files from the internet, + +05:48.880 --> 05:52.960 +uh, there are more details here and here, + +05:53.840 --> 05:59.280 +and this is a script that we can execute line by line. + +05:59.280 --> 06:04.080 +So, instead of this script being hidden behind the button that we just + +06:04.960 --> 06:07.360 +press after a question like, + +06:07.360 --> 06:09.920 +do you want me to do something, blah, blah, blah, yes or no, + +06:11.200 --> 06:15.120 +the script is visible here, and we can execute it step by step. + +06:15.200 --> 06:22.080 +It creates a terminal with a shell here in the right window, + +06:22.080 --> 06:28.880 +and when we type f8 in these lines here, the lines are sent to this line. + +06:30.080 --> 06:33.920 +So, this is going to download the copy of the video, + +06:33.920 --> 06:39.360 +the wget says that I already have a copy of the video in its subtitles, and so on. + +06:39.920 --> 06:47.600 +And after getting a copy of the video, we can run this exp here, and it displays the video. + +06:52.880 --> 06:57.840 +I said that Eevee has lots of videos for people who hate videos, + +06:57.840 --> 07:02.960 +and the idea is that very few people are going to watch the videos in real time, + +07:03.920 --> 07:10.480 +and most of the people that I know, or most of the people that are interested in Eevee in some way, + +07:10.480 --> 07:15.280 +they are going to watch just small sections of the video, + +07:15.280 --> 07:18.720 +and most of the time they're just going to read the subtitles of the video. + +07:19.680 --> 07:25.760 +So, for each one of the videos, we have a page about the video. + +07:25.760 --> 07:28.400 +Let me see if I have internet here. + +07:28.400 --> 07:29.680 +Yes, this is a page, + +07:29.920 --> 07:36.080 +and usually these pages have a link to another page, + +07:36.080 --> 07:42.880 +the page that has all the subtitles of the video, wherever. + +07:42.880 --> 07:46.960 +In this one, it's not so visible, but anyway, + +07:46.960 --> 07:50.480 +there are several ways of accessing the subtitles of the video, + +07:50.480 --> 07:54.720 +and one of the ways is by running this exp here, + +07:55.040 --> 07:57.280 +is by running this exp here, + +07:59.440 --> 08:03.680 +that opens a file in Lua that is what I use to generate the subtitles. + +08:04.880 --> 08:13.760 +Anyway, by the way, each one of these things here is hyperlinked to a position of the video, + +08:14.400 --> 08:19.280 +so if I type this in the right way, it goes to that position. + +08:20.160 --> 08:21.280 +Anyway, let me go back. + +08:21.920 --> 08:26.640 +Also, the tutorials of Eevee, the intros of Eevee, + +08:26.640 --> 08:30.560 +let's start with find, and with intro, + +08:30.560 --> 08:35.760 +they have lots of blocks that say video links, like this one, + +08:36.800 --> 08:40.080 +and these blocks have links to the positions in videos, + +08:40.080 --> 08:43.600 +and if we don't have a local copy of the video yet, + +08:44.480 --> 08:48.800 +the thing shows us a script that lets us download the local copy. + +08:48.880 --> 08:56.720 +Anyway, I said that I was going to explain what I mean by magic and black boxes. + +08:59.360 --> 09:02.080 +This is something that I've been trying to explain for a long time, + +09:02.080 --> 09:05.840 +and I think that I got a very good explanation about that + +09:05.840 --> 09:09.440 +in a video that I made about something called Eevee wconfig, + +09:09.440 --> 09:15.120 +that is a tool for configuring Eevee on Windows without magic, + +09:15.840 --> 09:20.320 +on Windows without magic, without buttons that do things + +09:21.040 --> 09:22.720 +without explaining what they are doing. + +09:24.080 --> 09:26.720 +This is a part of the subtitles of the video. + +09:26.720 --> 09:27.680 +Let me read that. + +09:30.080 --> 09:32.640 +Eevee wconfig is an attempt to solve the problem + +09:32.640 --> 09:35.040 +of how to install these things on Windows, + +09:35.040 --> 09:38.240 +both without magic and with very little magic. + +09:39.520 --> 09:40.640 +Remember this slogan, + +09:41.200 --> 09:45.760 +any sufficiently advanced technology is indistinguishable from magic. + +09:47.280 --> 09:51.600 +Here in this video, I'm going to use the term magic as a shorthand + +09:51.600 --> 09:55.520 +for sufficiently advanced technology, + +09:55.520 --> 09:59.040 +that is something that is complex and non-obvious, + +09:59.040 --> 10:01.840 +and that is indistinguishable from magic + +10:01.840 --> 10:04.640 +in the sense of being almost impossible to understand. + +10:05.600 --> 10:10.720 +And I'm also going to use the term black box as a near synonym for magic, + +10:11.360 --> 10:15.360 +and sometimes the term black box is more convenient, + +10:15.360 --> 10:18.800 +even though it's a bit longer, it has more letters, + +10:18.800 --> 10:20.880 +because when I use the term black box, + +10:20.880 --> 10:25.440 +it invites us to use expressions like opening the black box, + +10:25.440 --> 10:27.840 +and I'm going to use that expression a lot. + +10:34.640 --> 10:37.360 +Now, let me try to explain what is... + +10:37.360 --> 10:40.400 +Sorry, let me change the font. + +10:44.000 --> 10:44.800 +What is Lua? + +10:46.000 --> 10:51.840 +Lua is a minimalistic language in the sense of batteries not included. + +10:52.880 --> 10:56.640 +It uses associative tables for most of its data structures, + +10:58.800 --> 11:02.240 +and it's so minimalistic that it's the default print function. + +11:03.120 --> 11:09.280 +When we create an associative table and we ask it to print... + +11:11.760 --> 11:14.560 +We ask print to print an associative table, + +11:14.560 --> 11:17.280 +it just prints the address of the table. + +11:17.280 --> 11:18.480 +Here are some examples. + +11:19.920 --> 11:23.760 +Here is a table, and when we ask print to print it, + +11:23.760 --> 11:26.720 +it just says that it's the table at this address here. + +11:27.440 --> 11:32.400 +So, one of the things that most people do when they start using Lua + +11:32.400 --> 11:36.320 +is that either they download a package with a print to printing function, + +11:36.320 --> 11:38.720 +or they write their own print to printing functions. + +11:39.360 --> 11:42.000 +My own print to printing function is called pp, + +11:42.000 --> 11:45.840 +with uppercase letters, and it works like this, + +11:47.600 --> 11:50.880 +and it prints associative tables in a way like this. + +11:50.880 --> 11:54.800 +It says that for the key 1, + +11:55.200 --> 11:57.280 +the value associated to it is 2, + +11:57.280 --> 11:59.200 +for the key 2, the value is 3, + +11:59.200 --> 12:01.760 +and for the key 3, the value is 5. + +12:08.960 --> 12:12.160 +When I started using Lua, one of my favorite languages + +12:12.160 --> 12:15.600 +was also a language that used associative tables a lot. + +12:15.600 --> 12:16.720 +It was called Icon, + +12:18.400 --> 12:22.640 +and I had to write my own print to printing functions for Icon. + +12:22.720 --> 12:28.800 +So, I just had to port my print to printing functions to Lua, + +12:28.800 --> 12:31.600 +and my first version looked something like this. + +12:32.240 --> 12:36.560 +It just had some global functions, lots of them actually, + +12:39.680 --> 12:41.840 +and after a while I rewrote it, + +12:41.840 --> 12:44.320 +and I rewrote it again, and again, and again, + +12:44.320 --> 12:49.200 +and this is one of the versions that is not even the default at this point. + +12:50.000 --> 12:53.920 +TOS is for toString, + +12:56.800 --> 12:57.920 +and this is a demo. + +12:58.800 --> 13:04.560 +It's very modular, so it's easy to replace parts of it or to toggle flags, + +13:04.560 --> 13:05.760 +and this is an example. + +13:05.760 --> 13:10.320 +If I try to print the table of methods for a certain class, + +13:11.760 --> 13:13.040 +I'll need a smaller font, + +13:14.320 --> 13:16.480 +it prints the table like this, + +13:16.480 --> 13:18.480 +with the names of the methods, + +13:18.560 --> 13:21.520 +and then links to the source code of the functions. + +13:22.080 --> 13:25.360 +These links only make sense in Emacs and in Eevee, + +13:26.000 --> 13:28.560 +and when we run a link like this one, + +13:29.840 --> 13:33.920 +it shows the source code in the window at the right. + +13:33.920 --> 13:37.760 +So, for some functions, the source code is three lines, + +13:37.760 --> 13:40.400 +for other ones it's one line, and whatever. + +13:41.920 --> 13:43.200 +Anyway, let me go back. + +13:45.360 --> 13:48.000 +Lua can be used in many different styles. + +13:48.000 --> 13:50.160 +Most people hate other people's styles. + +13:51.920 --> 13:56.080 +When I started using it in the year 2000, + +13:56.080 --> 13:59.040 +I learned most of the basic language in a single day. + +13:59.040 --> 14:02.000 +It was very similar to things that I was already using, + +14:02.800 --> 14:08.560 +and then I rewrote the mini language that I was using + +14:08.560 --> 14:14.800 +to generate the HTML for my pages in Lua. + +14:14.800 --> 14:16.960 +Actually, I had to rewrite it many times, + +14:16.960 --> 14:20.880 +but the first version I certainly did in my first weeks + +14:20.880 --> 14:22.480 +or first months using Lua. + +14:25.520 --> 14:29.040 +In the beginning, I was just using it for writing programs + +14:29.040 --> 14:32.160 +that either didn't take in any input at all, + +14:32.160 --> 14:35.280 +because the input was already in the source file, + +14:35.280 --> 14:39.200 +or that worked as a Unix function, + +14:39.200 --> 14:42.960 +the Unix programs that would read files + +14:42.960 --> 14:46.160 +and process these files in some way and output something. + +14:47.840 --> 14:52.480 +I mentioned the basic language here. + +14:52.480 --> 14:55.520 +I only learned how to use closures, metatables, + +14:55.520 --> 14:58.240 +and coroutines many years later. + +15:00.320 --> 15:02.240 +In the beginning, when I started using Lua, + +15:02.240 --> 15:04.080 +it didn't have a package manager. + +15:04.080 --> 15:05.200 +It appeared later. + +15:05.200 --> 15:06.560 +It is called Lua-rocks. + +15:07.680 --> 15:11.600 +It has had this package manager for several years. + +15:12.720 --> 15:15.520 +Most of the rocks for Lua-rocks are poorly documented, + +15:16.480 --> 15:19.040 +documented, and hacker-unfriendly, + +15:19.040 --> 15:21.600 +so you can't rely just on the documentation, + +15:22.160 --> 15:24.320 +and you can't rely just on the source code, + +15:24.320 --> 15:28.320 +because, I mean, if you're a genius, of course you can, + +15:28.320 --> 15:32.320 +but for people who are either lazy or dumb or whatever, + +15:32.320 --> 15:33.920 +like me, or unfocused, + +15:35.120 --> 15:37.120 +the source code is hard to understand + +15:37.120 --> 15:38.800 +and hard to tinker with. + +15:41.440 --> 15:43.040 +Some rocks are excellent. + +15:43.040 --> 15:46.640 +The best rocks are well documented, + +15:46.640 --> 15:48.640 +but they are hacker-unfriendly, + +15:48.640 --> 15:51.840 +in the sense that I hope that I'll be able to explain soon. + +15:53.440 --> 15:58.560 +The best rocks use local variables and metatables a lot, + +15:59.280 --> 16:03.120 +so if you are a beginner learning Lua, + +16:03.120 --> 16:06.080 +you're not going to understand what their source codes do. + +16:06.080 --> 16:08.320 +They use lots of dirty tricks. + +16:08.640 --> 16:12.240 +Let me talk a bit about object orientation in Lua. + +16:12.240 --> 16:13.840 +It can be done in many ways. + +16:15.360 --> 16:18.000 +The main book about Lua, called Programming in Lua, + +16:18.000 --> 16:20.240 +by one of the authors of the language, + +16:21.120 --> 16:22.320 +Roberto Ierzalimschi, + +16:23.200 --> 16:26.960 +presents several ways of doing object orientation in Lua. + +16:26.960 --> 16:29.040 +I hated all of these ways, + +16:29.840 --> 16:32.720 +and also the ways that I tried from the rocks. + +16:34.400 --> 16:36.880 +And then I wrote my own way, + +16:37.360 --> 16:40.160 +wrote my own way of doing object orientation in Lua. + +16:40.160 --> 16:41.440 +It's very minimalistic. + +16:42.000 --> 16:45.120 +It's in this file here, eoo.lua. + +16:47.040 --> 16:49.440 +The main code is just these five lines here, + +16:50.640 --> 16:53.120 +and here's an example of how it works. + +16:56.080 --> 17:02.640 +Here we define a class vector with some metamethods. + +17:02.640 --> 17:07.920 +This metamethod here will tell Lua what to do + +17:07.920 --> 17:11.040 +when the user asks to add two vectors. + +17:11.600 --> 17:14.320 +This one here tells Lua what to do + +17:14.320 --> 17:18.320 +when the user asks Lua to convert a vector to a string, + +17:18.880 --> 17:20.640 +and whatever. + +17:20.640 --> 17:23.920 +This one is something that I'm going to explain in a second. + +17:25.360 --> 17:28.240 +So here we create a vector with these coordinates, + +17:28.240 --> 17:29.360 +three and four. + +17:29.360 --> 17:31.360 +Here we create another vector. + +17:31.360 --> 17:35.200 +If we print here, then Lua uses the function here + +17:35.200 --> 17:36.160 +in the toString. + +17:37.200 --> 17:40.880 +If we add two vectors, it uses the function here + +17:40.880 --> 17:42.400 +in the add metamethod. + +17:42.960 --> 17:45.280 +And if we run the method norm, + +17:45.280 --> 17:49.760 +it is defined here in the table index. + +17:51.280 --> 17:51.780 +Anyway. + +17:52.740 --> 17:54.740 +Even this thing being so small, + +17:54.740 --> 17:58.500 +I used to forget how it's in odds worked all the time. + +17:58.500 --> 18:01.700 +Actually, I always forget how things work, + +18:01.700 --> 18:03.940 +and I have to remember them somehow. + +18:03.940 --> 18:07.940 +And I have to have tricks for remembering + +18:07.940 --> 18:11.940 +and tricks for summarizing things and diagrams and so on. + +18:13.540 --> 18:15.940 +And every time that I forgot how to do it, + +18:15.940 --> 18:17.220 +I just go back to the code. + +18:17.220 --> 18:18.420 +I just go back to the code. + +18:18.500 --> 18:23.140 +And every time that I forgot how this thing worked, + +18:23.140 --> 18:25.540 +I went back to the source code, + +18:25.540 --> 18:27.380 +and then I looked at the diagrams. + +18:27.380 --> 18:29.940 +Or, of course, in the first time, + +18:29.940 --> 18:31.460 +I had to draw the diagrams. + +18:32.260 --> 18:34.420 +And I run the examples. + +18:34.420 --> 18:35.780 +And, of course, in the beginning, + +18:35.780 --> 18:37.460 +I thought that the code was clear. + +18:37.460 --> 18:38.900 +My examples were very brief. + +18:38.900 --> 18:42.820 +And so I had to rewrite the examples many times + +18:42.820 --> 18:46.420 +until they became, let's say, perfect. + +18:48.660 --> 18:53.220 +And I was saying that Lua can be used in many ways. + +18:53.780 --> 18:57.700 +And in my way of using Lua, my favorite way, + +18:57.700 --> 19:02.020 +everything can be inspected and modified from REPLs, + +19:03.140 --> 19:07.140 +like we can do in MX and in Smalltalk, or sort of. + +19:07.860 --> 19:09.940 +So in my favorite way of using Lua, + +19:09.940 --> 19:11.860 +there's no security at all. + +19:11.860 --> 19:14.740 +Everything can be changed at all times. + +19:15.140 --> 19:16.980 +Of course, most people hate that. + +19:17.780 --> 19:20.180 +My init file has lots of classes. + +19:20.820 --> 19:24.100 +And, by the way, instead of keeping many small files + +19:24.100 --> 19:25.140 +with many things, + +19:25.140 --> 19:29.300 +I put lots of stuff in just one big init file. + +19:31.060 --> 19:33.860 +My init file has lots of classes + +19:33.860 --> 19:36.660 +and lots of global functions and lots of crafts. + +19:36.660 --> 19:40.020 +So people hate that, of course. + +19:41.380 --> 19:42.580 +This is an example. + +19:43.460 --> 19:44.500 +This is an example. + +19:44.500 --> 19:48.260 +This is the index at the top of my init file. + +19:50.580 --> 19:52.820 +The classes start here. + +19:55.060 --> 19:57.220 +And then we have some functions. + +19:57.780 --> 20:02.500 +And then we have functions that load certain packages. + +20:02.500 --> 20:03.540 +And then we have craft. + +20:04.260 --> 20:04.760 +Whatever. + +20:06.180 --> 20:08.580 +Most people think that my style of using Lua + +20:08.580 --> 20:10.340 +is dirty and dangerous. + +20:10.340 --> 20:11.860 +And they wouldn't touch my Lua code. + +20:12.100 --> 20:13.140 +With a 10-feet pole. + +20:14.740 --> 20:18.100 +But most of the things that I'm going to present here + +20:18.100 --> 20:20.740 +in this presentation are ideas that should be easy + +20:20.740 --> 20:24.180 +to port to other environments and other languages. + +20:24.180 --> 20:25.780 +Especially the diagrams. + +20:25.780 --> 20:27.620 +So the code is not so important. + +20:30.740 --> 20:33.140 +Now let me talk a bit about LaTeX. + +20:33.140 --> 20:33.860 +Lua LaTeX. + +20:33.860 --> 20:38.820 +That is LaTeX with the Lua interpreter embedded inside. + +20:39.380 --> 20:43.300 +And two ways of generating pictures in LaTeX. + +20:43.300 --> 20:44.820 +Text that is very famous. + +20:44.820 --> 20:47.300 +And picture that is not very famous. + +20:47.300 --> 20:48.900 +And that is very low level. + +20:48.900 --> 20:51.700 +And I think that not many people use. + +20:53.700 --> 20:57.300 +I said before that when I learned Lua, + +20:57.300 --> 21:01.540 +I realized that it was very good for writing little languages. + +21:03.220 --> 21:05.220 +I was doing my PhD at the time. + +21:05.300 --> 21:11.220 +And typesetting the diagrams for my PhD thesis was very boring. + +21:11.220 --> 21:14.100 +So one of the things that I did was that I created + +21:14.980 --> 21:17.540 +a little language for typesetting the diagrams for me. + +21:19.140 --> 21:20.900 +It was called DEDNOT. + +21:20.900 --> 21:25.300 +Because initially it only generated diagrams + +21:25.300 --> 21:26.820 +for natural deduction. + +21:26.820 --> 21:28.580 +And then it had several versions. + +21:30.180 --> 21:34.420 +These are the slides for my presentation about DEDNOT6. + +21:34.420 --> 21:39.140 +And DEDNOT6 is an extensible semi-preprocessor for Lua LaTeX + +21:39.140 --> 21:41.780 +that understands diagrams in ASCII art. + +21:42.340 --> 21:48.740 +In the sense that when I have a LaTeX file that has this. + +21:49.380 --> 21:51.860 +And when DEDNOT6 is loaded. + +21:53.220 --> 21:58.820 +When I give the right commands, DEDNOT6 interprets this block here + +21:58.820 --> 22:00.980 +as something that defines this diagram. + +22:02.260 --> 22:03.540 +Oops, sorry. + +22:05.380 --> 22:07.540 +It interprets this diagram here. + +22:08.340 --> 22:10.900 +This diagram in the comments here + +22:11.460 --> 22:14.260 +as something that defines a diagram called foo, + +22:14.260 --> 22:16.100 +a deduction called foo. + +22:16.100 --> 22:19.140 +And it generates this code here. + +22:19.940 --> 22:27.300 +So that we can just invoke the definition of the deduction + +22:27.300 --> 22:30.260 +by typing backslash DED foo. + +22:31.220 --> 22:37.140 +And DEDNOT6 also supports another language + +22:37.140 --> 22:39.860 +for typesetting bi-dimensional diagrams + +22:39.860 --> 22:42.660 +with arrows and stuff for category theory and blah blah. + +22:44.180 --> 22:46.900 +The specifications of these diagrams look like this. + +22:49.540 --> 22:53.540 +Here is a very good example. + +22:55.540 --> 22:56.980 +This is a huge diagram. + +22:58.420 --> 22:59.460 +Sorry, one second. + +23:00.740 --> 23:03.620 +So the source code that generates this diagram here + +23:03.620 --> 23:05.140 +is just this thing at the left. + +23:07.220 --> 23:08.340 +So it's very visual. + +23:09.220 --> 23:12.340 +We can typeset the diagram in ASCII art here. + +23:12.340 --> 23:13.780 +And then in this part here, + +23:13.780 --> 23:16.660 +we tell how the nodes are to be joined, + +23:17.620 --> 23:20.660 +which arrows have to have annotations and so on. + +23:22.420 --> 23:25.460 +And this language is extensible in the sense that... + +23:25.780 --> 23:26.580 +Where is that? + +23:32.020 --> 23:32.520 +Here. + +23:34.420 --> 23:36.660 +Comments that start with percent colon + +23:37.940 --> 23:41.620 +are interpreted as definitions for three diagrams. + +23:43.620 --> 23:47.220 +Lines that start with percent uppercase D + +23:47.220 --> 23:52.260 +define 2D diagrams with arrows and stuff. + +23:52.740 --> 23:58.020 +And lines that start with comment uppercase L + +23:58.660 --> 24:00.580 +contain blocks of Lua code + +24:00.580 --> 24:04.180 +that we can use to extend the interpreter on the flag. + +24:06.020 --> 24:09.860 +Anyway, here are some recent examples of diagrams + +24:09.860 --> 24:14.580 +that I used DEDNOT6 to typeset. + +24:15.780 --> 24:17.220 +This diagram here + +24:17.780 --> 24:20.580 +was generated by this specification here. + +24:23.460 --> 24:26.980 +And this diagram here with the curved arrows + +24:27.780 --> 24:30.580 +was generated by this specification here. + +24:34.180 --> 24:37.220 +So DEDNOT6 was very easy to extend. + +24:37.220 --> 24:38.260 +At that some point, + +24:38.260 --> 24:42.820 +I started to use it to generate diagrams using Peaked Chewy, + +24:42.820 --> 24:45.540 +mainly for the classes that I gave you. + +24:45.540 --> 24:48.420 +For the classes that I give at the university. + +24:48.420 --> 24:50.900 +I teach mathematics and whatever. + +24:51.860 --> 24:52.900 +In a bad place. + +24:52.900 --> 24:53.400 +Whatever. + +24:56.260 --> 24:58.500 +Let me show an animation. + +24:59.060 --> 25:02.420 +Here is a diagram that I generated with DEDNOT6. + +25:02.980 --> 25:05.300 +And it is a flipbook animation. + +25:05.300 --> 25:08.740 +Like we type page up and page down + +25:08.740 --> 25:10.660 +and we go to the next page of the book + +25:10.660 --> 25:12.340 +and to the previous page of the book. + +25:13.060 --> 25:15.940 +And here is the source code that generates that. + +25:16.980 --> 25:19.140 +This source code is not very visual. + +25:19.140 --> 25:22.740 +So it's quite clumsy to edit a diagram + +25:22.740 --> 25:25.460 +directly in the tag file like that. + +25:28.020 --> 25:31.860 +These diagrams were inspired by something called Manim. + +25:32.500 --> 25:33.000 +That's... + +25:34.580 --> 25:36.340 +Oh, I've forgotten the name of the guy. + +25:36.340 --> 25:39.300 +But it's a guy that makes many videos about mathematics. + +25:39.300 --> 25:41.780 +And he created this library called Manim + +25:41.780 --> 25:43.460 +for generating his animations. + +25:45.540 --> 25:51.460 +Other people adapted his library to make it more accessible. + +25:52.580 --> 25:53.700 +I tried to learn it. + +25:53.700 --> 25:55.700 +But each animation, + +25:57.060 --> 25:59.220 +even an animation with very few frames, + +25:59.220 --> 26:01.300 +each animation took ages to render. + +26:01.300 --> 26:03.060 +So it wasn't fun. + +26:03.700 --> 26:07.060 +And animations and PDFs can be rendered in seconds. + +26:07.940 --> 26:11.060 +So these things were fun for me + +26:11.060 --> 26:15.300 +because my laptop is very slow and Manim was not fun. + +26:20.100 --> 26:26.020 +Anyway, writing code like this inside the attack file + +26:26.020 --> 26:29.620 +was not very fun because it was hard to debug. + +26:30.500 --> 26:35.860 +So in 2022, I started to play with ways + +26:35.860 --> 26:40.340 +of generating these diagrams from REPLs. + +26:40.980 --> 26:44.500 +And I found a way for PIC2E and a way for TIX. + +26:45.380 --> 26:47.780 +Each one of these ways became a video. + +26:48.740 --> 26:53.780 +If you go to the list of first class videos of EEV, + +26:53.780 --> 26:57.540 +you're going to see that there's a video about PIC2E here + +26:57.540 --> 26:58.740 +and a video about TIX. + +27:00.340 --> 27:07.540 +Here you have some information like length and explanation, etc. + +27:08.180 --> 27:10.420 +And here are the pages for these videos. + +27:12.420 --> 27:15.860 +My page about the video, about PIC2E is like this. + +27:15.860 --> 27:18.740 +It has some diagrams, whatever. + +27:18.740 --> 27:20.980 +And this one is much nicer. + +27:20.980 --> 27:26.100 +And a lot of people watched that video. + +27:26.100 --> 27:30.260 +I mean, I thought that 250 people watched it. + +27:30.900 --> 27:33.940 +For me, that's a million of people. + +27:36.660 --> 27:42.900 +And this video is about how to extract diagrams from the TIX manual + +27:42.900 --> 27:47.700 +and how to run those examples in a REPL + +27:47.700 --> 27:49.780 +and modify them bit by bit. + +27:49.780 --> 27:51.380 +This is a screenshot. + +27:52.100 --> 27:53.620 +But let me go back. + +27:56.420 --> 27:59.700 +At that point, these things were just prototypes. + +27:59.700 --> 28:01.460 +The code was not very nice. + +28:02.020 --> 28:10.500 +And in this year, I was able to unify those two ways of generating PDFs, + +28:11.380 --> 28:14.100 +the one for TIX and the one for PIC2E. + +28:14.100 --> 28:18.740 +And I unified them with many other things that generated diagrams. + +28:19.220 --> 28:27.460 +The basis of these things is something called showchew.lua. + +28:27.460 --> 28:31.380 +I'm not going to show its details now. + +28:32.100 --> 28:40.420 +But its extension that generates TIX code is just this. + +28:40.420 --> 28:45.540 +So we can specify a diagram with just a block like this. + +28:46.260 --> 28:54.180 +And then if we run show00 and it returns a string + +28:54.180 --> 28:58.580 +that is just the inner body of the TIX file. + +28:59.220 --> 29:02.020 +If we run this, we see the whole TIX file. + +29:02.020 --> 29:04.340 +And if we run this, we save the TIX file + +29:04.340 --> 29:08.420 +and we compile the TIX file to generate a PDF. + +29:08.420 --> 29:14.180 +And if we run this, we show the PDF in the lower right window. + +29:16.020 --> 29:20.740 +And that's the same thing for all my recent programs that generate PDFs. + +29:21.620 --> 29:22.980 +They are all integrated. + +29:24.500 --> 29:30.660 +Here is the one that the basis for all my modules that generate diagrams with PIC2E. + +29:33.060 --> 29:34.980 +Its demos are not very interesting. + +29:34.980 --> 29:39.700 +So let me show some demos of extensions that do interesting things. + +29:40.660 --> 29:47.220 +So this is a diagram that I created by editing it in a REPL. + +29:48.820 --> 29:51.220 +I create several PIC2E objects here. + +29:52.500 --> 29:58.660 +And if I execute this, it compiles the object, generates a PDF. + +29:58.660 --> 30:03.860 +And if I tap this, here is the PDF. + +30:04.740 --> 30:10.020 +And if I just ask Lua to display what is books here, + +30:11.700 --> 30:17.220 +it shows the source code in PIC2E of the diagram. + +30:17.940 --> 30:20.340 +And the nice thing is that it is indented. + +30:20.340 --> 30:23.460 +So it's easy to debug the PIC2E code. + +30:24.340 --> 30:30.180 +If anyone is interested, the module that does the tricks for indentation + +30:30.180 --> 30:31.460 +is very easy to understand. + +30:31.540 --> 30:34.500 +It has lots of tests and test blocks. + +30:34.500 --> 30:39.060 +And I think that its data structures are easy to understand. + +30:42.500 --> 30:44.820 +Anyway, here is another example. + +30:50.260 --> 30:51.620 +The show is here. + +30:53.460 --> 30:56.340 +It generates a 3D diagram. + +31:02.420 --> 31:08.500 +Now let me talk about parsers and REPLs in a very strange place. + +31:09.140 --> 31:18.340 +I mean, using REPLs to build parsers step by step and replacing parts by more complex parts. + +31:21.060 --> 31:23.780 +So I said that Lua is very minimalistic. + +31:25.780 --> 31:30.660 +And everybody knows that implementations of regular expressions + +31:30.740 --> 31:31.780 +are big and complex. + +31:33.060 --> 31:39.220 +So instead of coming with full regular expressions, Lua comes with something called patterns + +31:39.220 --> 31:43.620 +and a library function called string.match. + +31:45.380 --> 31:54.420 +Here is a copy of the part of the manual that explains a part of the syntax of patterns. + +31:56.260 --> 32:00.100 +Here's how string.match is described in the manual. + +32:01.220 --> 32:02.500 +It's just this. + +32:02.500 --> 32:06.260 +Looks for the first match of pattern in the string as blah, blah, blah. + +32:07.220 --> 32:11.300 +And then we have to go to the other section of the manual that explains patterns. + +32:17.780 --> 32:26.500 +Lua patterns are so simple, so limited, that they don't even have the alternation operator. + +32:27.220 --> 32:31.460 +Here is how it is described in the ELLISP manual. + +32:33.300 --> 32:37.780 +Backslash pipe specifies an alternative, blah, blah, blah. + +32:40.820 --> 32:47.940 +When we want to build more complex regular expressions, patterns, grammars, etc., + +32:47.940 --> 32:51.780 +we have to use an external library for that. + +32:52.740 --> 32:59.140 +No, sorry, a library that is external, but that was written by one of the authors of Lua itself. + +33:00.340 --> 33:05.380 +This library is called LPEG, and its manual says, + +33:06.180 --> 33:11.940 +LPEG is a new pattern matching library for Lua based on parsing expression grammars, PEGs. + +33:14.900 --> 33:17.460 +The manual is very terse. + +33:18.180 --> 33:20.500 +I found it incredibly hard to read. + +33:20.900 --> 33:22.740 +It doesn't have any diagrams. + +33:22.740 --> 33:24.420 +It has some examples, though. + +33:26.180 --> 33:34.420 +And the Lua wiki has a big page called LPEG tutorial with lots of examples. + +33:35.700 --> 33:42.020 +But it also doesn't have diagrams, and I found some things incredibly hard to understand. + +33:42.020 --> 33:48.500 +For example, this is something that is in the manual of LPEG that I saw and I thought, + +33:49.460 --> 33:53.060 +wow, great, this makes all sense, and this is going to be very useful. + +33:53.060 --> 33:57.300 +It's a way to build grammars that can be recursive, + +33:57.300 --> 34:02.020 +and they sort of can encode BNF grammars. + +34:02.020 --> 34:06.980 +We just have to translate the BNF a bit to get rid of some recursions + +34:06.980 --> 34:08.820 +and to translate them to something else. + +34:10.580 --> 34:13.620 +And the manual also has some things that I thought, + +34:13.620 --> 34:16.580 +oh, no, I don't have any idea of what this thing does. + +34:17.380 --> 34:22.020 +And in fact, I saw these things for the first time more than 10 years ago, + +34:22.020 --> 34:26.020 +and they only started to make sense one year ago. + +34:28.420 --> 34:30.340 +One example is group captures. + +34:33.380 --> 34:37.620 +LPEG also comes with a module called the RE module. + +34:37.620 --> 34:40.180 +Let me pronounce it in Portuguese, the RE module. + +34:40.660 --> 34:45.540 +And its manual says, the RE module provided by the file repo.luen + +34:45.540 --> 34:48.100 +in the distribution supports a somewhat conventional + +34:48.980 --> 34:54.420 +regular expression syntax for pattern usage within LPEG. + +34:55.140 --> 34:57.460 +And this is a quick reference. + +35:00.500 --> 35:02.820 +And this thing is very brief. + +35:02.820 --> 35:06.420 +It has some nice examples, but it's hard to understand in a way. + +35:06.660 --> 35:12.500 +And here are some comments about my attempts to learn HerdotLua. + +35:14.020 --> 35:15.220 +This is a class. + +35:15.220 --> 35:17.060 +In this case, it's a very small class. + +35:18.180 --> 35:21.940 +And this file implements a PM method. + +35:23.540 --> 35:26.980 +I'm going to show examples of other PM methods very soon. + +35:28.020 --> 35:33.380 +So this is a PM method for HerdotLua that lets us compare the syntax + +35:33.620 --> 35:39.460 +of Lua patterns, LPEG, and HerdotLua. + +35:41.620 --> 35:43.060 +See this example here. + +35:43.700 --> 35:49.940 +So if we run this, it loads my version of LPEG. + +35:50.660 --> 35:52.740 +No, sorry, my version of LPEG-REX. + +35:54.820 --> 36:00.180 +And it shows that when we apply the PM method to this Lua pattern, + +36:00.740 --> 36:07.380 +this LPEG pattern, and this RE pattern, they all get the same results. + +36:07.380 --> 36:12.820 +So we can use this thing, this kind of thing here to show how to translate from + +36:14.420 --> 36:20.180 +Lua patterns that are familiar because they are similar to regular expressions, only weaker, + +36:22.260 --> 36:27.620 +to LPEG that is super weird, and to RE that is not so weird. + +36:28.580 --> 36:37.940 +Anyway, the comment says that in 2012, I had a project that needed a precedence parser that + +36:37.940 --> 36:42.900 +could parse arithmetical expressions with the right precedences. + +36:44.580 --> 36:50.180 +And at that point, I was still struggling with pure LPEG, and I couldn't do much with it. + +36:50.180 --> 36:55.300 +So I tried to learn HerdotLua instead, and I wrote this old class here. + +36:56.260 --> 36:59.620 +That allowed me to use preprocessor on patterns for Lua. + +36:59.620 --> 37:04.980 +And the thing is that with this preprocessor, I could specify precedence grammars using this + +37:04.980 --> 37:12.900 +thing here that worked, but it was super clumsy, and I gave up after a few attempts. + +37:14.900 --> 37:21.300 +And in 2022, I heard about something called LPEG-REX that was an + +37:22.260 --> 37:28.740 +a kind of extension of Re, and it was much more powerful than HerdotLua, but after a while, + +37:28.740 --> 37:32.660 +I realized that it had the same defects as HerdotLua. + +37:32.660 --> 37:41.620 +And let me explain that because it has all to do with the things about black boxes and magic + +37:41.620 --> 37:42.980 +that I told in the beginning. + +37:43.380 --> 37:50.100 +Both, I mean, sorry, neither HerdotLua or LPEG-REX had some features that I needed. + +37:51.620 --> 37:57.940 +They didn't let us explain, sorry, they received a pattern that was specified as a string, + +37:57.940 --> 38:04.740 +and it converted that into an LPEG pattern, but it didn't let us explore the LPEG patterns + +38:04.740 --> 38:05.540 +that it was using. + +38:05.540 --> 38:12.900 +So I had to use the LPEG-REX, and it didn't let me explore the LPEG patterns that it was + +38:12.900 --> 38:13.540 +generated. + +38:16.420 --> 38:21.060 +Their code was written in a way that was REPL unfriendly. + +38:21.060 --> 38:28.580 +I couldn't modify parts of the code bit by bit in a REPL and try to change the code + +38:29.700 --> 38:31.620 +without changing the original file, say. + +38:33.300 --> 38:37.220 +The code was very hard to explore, to hack, and to extend, in my opinion. + +38:37.780 --> 38:39.700 +The documentation was not very clear. + +38:40.580 --> 38:50.660 +And I sent one or two messages to the developer of LPEG-REX, and he was too busy to help me. + +38:50.660 --> 38:55.460 +He answered very briefly, and to be honest, I felt rejected. + +38:55.460 --> 38:58.580 +I felt that I wasn't doing anything interesting. + +38:58.580 --> 38:59.380 +Whatever, whatever. + +39:00.340 --> 39:11.940 +So, in 2022, I was trying to learn LPEG-REX, because I was thinking that it would solve + +39:11.940 --> 39:13.780 +my problems, but it didn't. + +39:14.500 --> 39:20.900 +It didn't have the features that I needed, and it was hard to extend, and hard to explore, + +39:20.900 --> 39:22.100 +and hard to debug. + +39:22.980 --> 39:32.420 +I decided to rewrite it in a more hacker-friendly way, in the sense that it was modular, and + +39:32.420 --> 39:35.460 +I could replace any part of the module from a REPL. + +39:37.300 --> 39:48.340 +My version of it was called lpeg1.lua, and I decided that in my version, I wouldn't have + +39:49.060 --> 39:55.460 +the part that receives a grammar specified as a string and converts that to LPEG. + +39:55.460 --> 40:03.780 +I would just have the backend part that are the functions in LPEG that let us specify + +40:04.340 --> 40:05.380 +powerful grammars. + +40:10.340 --> 40:11.540 +So, let me go back. + +40:12.260 --> 40:14.260 +Let me explain a bit about LPEG. + +40:14.900 --> 40:16.180 +Lua has coercions. + +40:18.660 --> 40:22.740 +The plus expects to receive two numbers. + +40:22.740 --> 40:29.860 +If one of its arguments, or both of them, are strings, it converts the strings to numbers. + +40:29.860 --> 40:39.220 +So, in this case here, 2 plus string 3 returns the number 5, and this is the concatenation + +40:39.220 --> 40:39.860 +operator. + +40:40.820 --> 40:42.820 +It expects to receive strings. + +40:43.780 --> 40:49.860 +So, in this case, it will convert the number 2 to the string 2, and the concatenation of + +40:49.860 --> 40:51.700 +these two things will be 23. + +40:52.420 --> 40:54.420 +Sorry, 23 as a string. + +40:56.260 --> 40:58.420 +LPEG also has some coercions. + +41:00.260 --> 41:09.620 +I usually set these globals to let me write my grammars in a very compact way. + +41:10.260 --> 41:17.460 +So, instead of lpeg.p, lpeg.c, etc., I use these globals like uppercase B, uppercase + +41:17.460 --> 41:18.420 +C, and so on. + +41:19.540 --> 41:27.540 +And with these globals, I can write things like this, c1 times string underscore. + +41:28.420 --> 41:40.820 +And LPEG knows that lpeg.c, sorry, it sort of expands these to lpeg.c, but lpeg.c expects + +41:40.820 --> 41:47.380 +to receive an LPEG pattern, and one is not yet an LPEG pattern, so it is coerced into + +41:47.380 --> 41:50.900 +an LPEG pattern by calling lpeg.p. + +41:51.220 --> 42:02.740 +So, this shorting here becomes equivalent to lpeg.c, lpeg.p1, and the multiplication, + +42:02.740 --> 42:09.940 +when at least one of its arguments is an LPEG pattern, it expects to receive two LPEG patterns, + +42:09.940 --> 42:15.700 +and in this case, the one at the right is just a string, so it is coerced to an LPEG + +42:15.700 --> 42:17.620 +pattern by using lpeg.p. + +42:18.180 --> 42:22.180 +With this idea, we can sort of understand this comparison here. + +42:22.900 --> 42:24.980 +I mean, let me run it again. + +42:24.980 --> 42:33.620 +This first part is very similar to a regular expression here at the left, and when we apply + +42:33.620 --> 42:45.940 +this LPEG, sorry, this Lua pattern to this subject here, the result is that the Lua pattern + +42:46.740 --> 42:53.540 +the result is this thing here, this thing, this thing, and this thing. + +42:53.540 --> 43:00.580 +I'm going to call each one of these results captures, so each of these things between + +43:00.580 --> 43:08.020 +parentheses captures a substring of the original string, and these captured substrings are + +43:08.020 --> 43:09.460 +returned in a certain order. + +43:10.420 --> 43:12.740 +Here is how to express the same thing in LPEG. + +43:13.540 --> 43:22.660 +It's very cryptic, but it's a good way to understand some basic operators of LPEG. + +43:22.660 --> 43:34.900 +I mean, we can look at the menu and understand what C, S, and R do, and also exponentiation. + +43:35.860 --> 43:43.060 +And this strange thing here receives this string here, runs a function that I have defined + +43:43.060 --> 43:50.260 +that converts it to an object of a certain class, and that class represents He patterns. + +43:50.260 --> 43:56.900 +So this thing is treated as a pattern for He.Lua, and it is matched against the string, + +43:56.900 --> 43:59.300 +and it returns the same thing as the other one. + +43:59.860 --> 44:06.260 +Also, this thing here also has a comparison with LPEG-REGS, but these patterns are very + +44:06.260 --> 44:06.820 +trivial. + +44:06.820 --> 44:10.180 +They don't do anything very strange. + +44:10.180 --> 44:14.260 +So let's go back and see what kinds of very strange things there are. + +44:16.660 --> 44:23.620 +Here is the page of LPEG-REGS at GitHub. + +44:24.260 --> 44:25.860 +Here's the documentation. + +44:27.940 --> 44:29.380 +It's relatively brief. + +44:29.380 --> 44:35.300 +It explains LPEG-REGS as being an extension of He.Lua. + +44:36.500 --> 44:40.100 +So it explains mainly the additional features. + +44:40.100 --> 44:44.180 +Here is a quick reference that explains only the additional features. + +44:46.820 --> 44:53.380 +Some of these things I was able to understand by using the LPEG-REGS. + +44:54.340 --> 45:02.820 +I was struggling a lot, and some I wasn't able to, even by spending several evenings + +45:02.820 --> 45:04.420 +trying to build examples. + +45:08.260 --> 45:10.900 +And this is something very nice. + +45:12.100 --> 45:19.700 +LPEG-REGS comes with some example parsers, and here is a parser that parses the Lua grammar. + +45:19.700 --> 45:28.100 +I mean, this is the grammar for Lua 5.4 at the end of the reference manual. + +45:28.660 --> 45:35.860 +It's just this, and this is a kind of the BNF, and this is the BNF translated to + +45:37.300 --> 45:39.860 +the language of LPEG-REGS. + +45:40.420 --> 45:47.300 +So this thing uses many constructions that are in He.Lua and some extra constructions + +45:47.300 --> 45:49.460 +that are described here. + +45:50.260 --> 45:58.980 +And with these examples, I was able to understand some of these things here that are described + +45:58.980 --> 46:02.660 +here in the quick reference, but not all. + +46:06.660 --> 46:15.300 +So I wasn't able to use LPEG-REGS by itself because some things didn't make much sense, + +46:15.300 --> 46:18.900 +and I decided to reimplement it in my own style. + +46:20.900 --> 46:27.860 +Because that would be a way to map, at the very least, map what I understood and what + +46:27.860 --> 46:32.980 +I didn't, and learn one feature at a time, do comparisons, and so on. + +46:35.380 --> 46:38.900 +Here, I pointed to two features of LPEG. + +46:38.900 --> 46:44.820 +One, I said, oh great, this thing can be used to define grammars, even recursive + +46:44.900 --> 46:46.020 +grammars, and so on. + +46:46.740 --> 46:52.820 +And this is an oh-no feature, one thing that didn't make any sense at all, group captures. + +46:55.380 --> 47:01.620 +One thing that I did to understand group captures was to represent them as diagrams. + +47:01.620 --> 47:08.020 +Of course, in the beginning, I was drawing these diagrams by hand, but then I realized + +47:08.020 --> 47:15.220 +that I could use the bits of LPEG that I already knew to build a grammar that would + +47:15.220 --> 47:19.380 +parse a little language and generate these diagrams in LaTeX. + +47:20.340 --> 47:21.940 +And I was able to make this. + +47:24.180 --> 47:34.500 +In this diagram here, this thing above the arrow is a piece of Lua code that specifies + +47:34.500 --> 47:36.100 +an LPEG pattern. + +47:36.820 --> 47:43.540 +This thing here at the top is the string that is being matched, and the things below the + +47:43.540 --> 47:51.220 +under braces are the captures that each thing, sorry, each thing captures. + +47:54.580 --> 48:02.100 +And for example, this under brace here corresponds to this pattern here that parses a single + +48:02.100 --> 48:05.060 +character but doesn't return any captures. + +48:05.140 --> 48:10.420 +This thing here parses a single B and doesn't return any captures. + +48:10.420 --> 48:16.740 +This thing here parses a single character and captures it, and this thing here parses + +48:17.620 --> 48:19.700 +the character D and captures it. + +48:20.340 --> 48:29.220 +And this other thing here that transforms this pattern into another pattern returns + +48:29.780 --> 48:36.500 +first a capture with all the string that was parsed by this pattern here, and then all + +48:36.500 --> 48:41.380 +the captures returned by this thing here before the column. + +48:43.380 --> 48:50.900 +So this was a way to build concrete examples for things that the LPEG manual was explaining + +48:50.900 --> 48:53.940 +in a very terse way, and it worked for me. + +48:54.740 --> 49:01.540 +Some things that were very mysterious started to make sense, and I started to have intelligent + +49:01.540 --> 49:03.300 +questions to ask in the mailing list. + +49:07.700 --> 49:16.980 +And with that, I was able to understand what are group captures and group captures that + +49:16.980 --> 49:17.860 +receive a name. + +49:20.900 --> 49:23.140 +Well, let me explain what this does. + +49:23.220 --> 49:30.180 +This thing here captures, sorry, parses the empty string and returns this as a constant. + +49:30.180 --> 49:35.860 +So this is something that doesn't exist in regular expressions. + +49:36.980 --> 49:40.180 +It parses nothing and returns this as a capture. + +49:40.820 --> 49:48.580 +Then this thing here returns these two constants here and parses the empty string, and this + +49:48.660 --> 49:56.500 +thing here, d, converts the results of this thing here into a group capture and stores + +49:56.500 --> 49:58.020 +it in the label d. + +50:01.380 --> 50:03.860 +And then here's another constant capture. + +50:03.860 --> 50:11.460 +And I realized that these things here were similar to how Lua specifies building lists. + +50:11.780 --> 50:16.580 +When we build, sorry, a table, when we build a table and we say that the first element + +50:16.580 --> 50:20.420 +of the table is here, this element is put at the end of the table. + +50:21.140 --> 50:30.820 +When after that we say d equals to, say, 42, we are putting the 42 in the slot whose key + +50:30.820 --> 50:31.380 +is d. + +50:33.380 --> 50:37.860 +This was happening with LPEG captures, but there was something very strange. + +50:37.860 --> 50:46.260 +These group captures could hold more than one capture, more than one value. + +50:46.260 --> 50:50.340 +So there was something between lists and tables. + +50:51.300 --> 51:00.740 +I started to use this notation to explain in my notation what they were doing. + +51:02.340 --> 51:04.900 +Many things start, things stop. + +51:05.220 --> 51:08.900 +Many things start, things started to make sense. + +51:08.900 --> 51:14.420 +Many mysterious sentences in the manual started to make sense, but some didn't. + +51:16.100 --> 51:25.060 +But at least I was able to send some intelligent questions to the mailing list, and the author + +51:25.060 --> 51:27.540 +of Lua and LPEG answered some of them. + +51:29.700 --> 51:31.860 +He was not very happy about my questions. + +51:32.020 --> 51:40.100 +He told me that those diagrams were a waste of time, that the manual was perfectly clear, + +51:40.100 --> 51:42.900 +and so on, whatever. + +51:42.900 --> 51:51.620 +But I was able to, so it was weird, but I was able to understand lots of things from + +51:51.620 --> 51:52.340 +his answers. + +51:53.940 --> 51:57.380 +So this is a copy of one of my messages. + +51:57.380 --> 51:59.300 +Then there's another one, another one. + +51:59.300 --> 52:04.580 +Some were the diagrams, then he complained about these diagrams. + +52:04.580 --> 52:12.980 +He said that these things here that look like table constructions do not exist, whatever. + +52:15.380 --> 52:23.220 +Anyway, once I understood group captures, many features were very, very easy to understand, + +52:23.780 --> 52:28.260 +and I started to be able to use LPEG to build some very interesting things. + +52:30.260 --> 52:36.340 +I was able to reproduce some of the features that I saw in LPEG-REX. + +52:38.420 --> 52:40.980 +Remember that this, where is that? + +52:42.420 --> 52:44.020 +This is a syntax of Lua. + +52:45.860 --> 52:54.820 +Here, I was able to understand how these things here were translated to LPEG code, to LPEG + +52:54.820 --> 52:58.340 +patterns by using group captures in a certain way. + +52:59.460 --> 53:05.060 +And I was able to implement them in lpeg1.lua. + +53:06.340 --> 53:14.180 +And after some time, I was able to use lpeg1.lua to build grammars that were able to parse + +53:17.300 --> 53:19.860 +arithmetical expressions with the right precedence. + +53:19.860 --> 53:25.700 +And here's an example in which I built the grammar step by step, and I test the current + +53:25.700 --> 53:29.300 +grammar, and I replace a bit, and then I test the new grammar, and so on. + +53:31.780 --> 53:38.580 +And you can see that the result is always a tree that is drawn in a nice two-dimensional way. + +53:42.580 --> 53:52.340 +At this point, these powers here are returned as a list, as an operation + +53:52.900 --> 53:56.260 +pow with several arguments here. + +53:56.820 --> 54:04.820 +And then I apply a kind of parsing combinator here that transforms these trees into other trees. + +54:05.540 --> 54:14.180 +And with these combinators here, I can specify that the power is associative in a certain + +54:14.180 --> 54:20.340 +direction, the division is associative in another direction, the minus is associative, + +54:20.420 --> 54:24.420 +so it uses the same direction as a division, and so on. + +54:24.420 --> 54:26.020 +And they have the right precedences. + +54:28.660 --> 54:30.020 +So here are the tests. + +54:33.700 --> 54:37.140 +So here's my file lpeg1.lua. + +54:37.140 --> 54:38.340 +It has several classes. + +54:38.980 --> 54:42.980 +Each class has tests after it. + +54:43.940 --> 54:51.780 +I was able to implement something that lpeg-reqs has that's called keywords. + +54:51.780 --> 54:56.980 +That's very useful for parsing programs in programming languages. + +54:56.980 --> 55:06.580 +I was able to implement something similar to the debugger, to the PEG debugger that lpeg uses, + +55:07.220 --> 55:12.740 +but I was frustrated by some limitations of that debugger, + +55:12.740 --> 55:16.180 +and I implemented my own that is much better. + +55:17.700 --> 55:23.860 +And let me show something else. + +55:23.860 --> 55:32.900 +I was able to translate a good part of the Lua parser here to lpeg1.lua. + +55:33.700 --> 55:39.940 +I haven't finished yet, but I have most of the translation here. + +55:43.140 --> 55:50.420 +And after having all that, I was able to build other grammars very quickly. + +55:51.380 --> 55:59.220 +Writing new parsers finally became fun, and here's one example that I showed in the beginning. + +56:03.620 --> 56:11.220 +If I remember correctly, I took a figure from the Wikipedia. + +56:11.220 --> 56:17.220 +I don't have its link now, but I specified a grammar that parses + +56:18.500 --> 56:21.140 +exactly the example that appears in the Wikipedia. + +56:21.140 --> 56:28.100 +So with my grammar, considering that the top-level entry is statement, + +56:28.100 --> 56:36.900 +when I parse this string here, the result is this tree. + +56:38.100 --> 56:41.780 +And I can do some operations on that. + +56:41.780 --> 56:45.780 +I can define how this thing is to be converted into LaTeX. + +56:47.460 --> 56:51.700 +I can define other operations that convert trees into other trees. + +56:52.580 --> 56:54.980 +And here are some tests of these operations. + +56:58.340 --> 57:00.660 +This is what I showed in the beginning. + +57:00.660 --> 57:04.100 +I'm not going to explain all the details of this thing now. + +57:07.460 --> 57:14.260 +This show converts this thing into LaTeX in the way specified by these instructions here. + +57:15.380 --> 57:16.340 +Let's say that... + +57:19.540 --> 57:20.260 +Well, whatever. + +57:22.740 --> 57:23.540 +Really, whatever. + +57:24.020 --> 57:28.500 +And here's the result, the LaTeX result. + +57:34.900 --> 57:46.260 +And these diagrams here are generated by this file here that defines a simple grammar that parses + +57:46.260 --> 57:53.460 +this thing here, and then LaTeX it in a certain way, and then also tests this thing. + +57:54.100 --> 58:00.900 +To check if this code here, that is Lua code that generates an LPEG grammar, + +58:02.900 --> 58:08.740 +parses this subject here and returns the expected result. + +58:12.740 --> 58:15.140 +So this is the code that I wanted to show. + +58:15.700 --> 58:21.700 +I wanted to show many more things, but I wasn't able to prepare them before the conference. + +58:22.580 --> 58:29.060 +And I hope that soon, for some value of soon, I'll be able to create + +58:29.620 --> 58:37.060 +REPL-based tutorials for LPEG here and lpeg1.lua, where LPEG is something very famous. + +58:37.060 --> 58:39.460 +Here is a module of LPEG. + +58:40.500 --> 58:48.100 +I could also do something like this for LPEG-REX, and lpeg1.lua is the thing that I wrote, + +58:48.820 --> 58:58.900 +the one that has tests in comments, and the tests usually generate trees, + +58:58.900 --> 59:00.900 +and sometimes they generate tag code. + +59:03.300 --> 59:04.420 +Yeah, so that's it. + +59:04.420 --> 59:07.860 +I wanted to present much more, but I wasn't able to prepare it. + +59:07.860 --> 59:08.660 +So sorry. + +59:08.660 --> 59:09.140 +Thanks. + +59:09.140 --> 59:10.260 +Bye. + |