WEBVTT captioned by eduardo Kind: captions Language: en-GB NOTE Intro 00:00:00.000 --> 00:00:03.079 Hi! My name is Eduardo Ochs 00:00:03.080 --> 00:00:05.519 and the title of this talk is: REPLs 00:00:05.520 --> 00:00:09.399 in strange places - Lua, LateX, LPeg, LPegRex, 00:00:09.400 --> 00:00:12.119 and TikZ. I'm the author of an Emacs 00:00:12.120 --> 00:00:14.879 package called eev, and this is a talk 00:00:14.880 --> 00:00:18.479 at the EmacsConf 2023, that is happening in 00:00:18.480 --> 00:00:22.519 December 2023, at the internets. NOTE Diagrams 00:00:22.520 --> 00:00:23.079 This is one of the 00:00:23.080 --> 00:00:24.919 examples of diagrams that we are 00:00:24.920 --> 00:00:27.639 going to see - let me show how I generate 00:00:27.640 --> 00:00:30.599 it... one second, 00:00:30.600 --> 00:00:35.799 I have to use a smaller font here... 00:00:35.800 --> 00:00:39.159 this is a file called ParseTree2.lua... 00:00:39.160 --> 00:00:42.919 let me go back to this block of tests again... 00:00:42.920 --> 00:00:43.879 and now if I run 00:00:43.880 --> 00:00:47.759 this... 00:00:47.760 --> 00:00:50.159 we get these outputs here at the 00:00:50.160 --> 00:00:53.959 right, and then in this line here it 00:00:53.960 --> 00:00:58.119 generates a PDF, and if I type f8 here it 00:00:58.120 --> 00:01:03.919 shows the PDF in the lower right window. NOTE eev 00:01:03.920 --> 00:01:06.079 Let me start by explaining 00:01:06.080 --> 00:01:09.839 briefly what is eev. 00:01:09.840 --> 00:01:12.239 First: it is something that 00:01:12.240 --> 00:01:14.759 appeared by accident in the mid-90s - I 00:01:14.760 --> 00:01:16.799 explained this story in my 00:01:16.800 --> 00:01:20.159 presentation at the EmacsConf 2019... 00:01:20.160 --> 00:01:23.159 it's a package... it's an Emacs 00:01:23.160 --> 00:01:26.519 package that is part of ELPA... it has at 00:01:26.520 --> 00:01:29.679 least 10 users - those are the ones 00:01:29.680 --> 00:01:32.559 that I know by name... 00:01:32.560 --> 00:01:38.439 eev means `emacs-execute-verbosely'... 00:01:38.440 --> 00:01:40.959 eev is something that treats eval-last-sexp 00:01:40.960 --> 00:01:43.999 as the central feature of Emacs... 00:01:44.000 --> 00:01:45.599 eev blurs the distinction between 00:01:45.600 --> 00:01:48.919 programmers and users, and it replaces 00:01:48.920 --> 00:01:51.679 the slogan "users should not be forced to 00:01:51.680 --> 00:01:53.679 see Lisp", that is something that Richard 00:01:53.680 --> 00:01:57.559 Stallman told me once, by "users should see 00:01:57.560 --> 00:02:00.479 Lisp instead of buttons" and "new users 00:02:00.480 --> 00:02:03.999 should see Lisp in the first 5 minutes"... 00:02:04.000 --> 00:02:05.279 I'm going to show 00:02:05.280 --> 00:02:08.959 some examples of that soon. 00:02:08.960 --> 00:02:11.959 Eev uses code in comments a lot, 00:02:11.960 --> 00:02:15.119 and also tests in comments... 00:02:15.120 --> 00:02:17.679 I changed my way of presenting it 00:02:17.680 --> 00:02:19.679 and it became very REPL-centric 00:02:19.680 --> 00:02:22.319 in the last few years, in the 00:02:22.320 --> 00:02:24.399 sense that I start by explaining its 00:02:24.400 --> 00:02:28.679 main features by its support for REPLs... 00:02:28.680 --> 00:02:31.079 eev supposes 00:02:31.080 --> 00:02:32.639 that we want to keep 00:02:32.640 --> 00:02:35.199 executable notes of everything - I'm also 00:02:35.200 --> 00:02:37.159 going to show examples of this in a 00:02:37.160 --> 00:02:40.479 second... eev has lots of "videos for 00:02:40.480 --> 00:02:43.559 people who hate videos", and it tries to 00:02:43.560 --> 00:02:46.159 do everything with very little magic and 00:02:46.160 --> 00:02:48.839 without black boxes - I'm going to explain 00:02:48.840 --> 00:02:50.319 many of these things very soon. NOTE Another figure 00:02:50.320 --> 00:02:52.799 This is a figure that that I'm going 00:02:52.800 --> 00:02:57.119 to show in details soon, that is 00:02:57.120 --> 00:02:59.959 about something important about Lua... 00:02:59.960 --> 00:03:03.959 the font is very bad now, so let me 00:03:03.960 --> 00:03:07.559 change the font... the figure is this one... 00:03:07.560 --> 00:03:08.239 and... 00:03:08.240 --> 00:03:12.519 what most people do when they 00:03:12.520 --> 00:03:14.479 visit a file with something 00:03:14.480 --> 00:03:16.479 interesting on it is that they just go 00:03:16.480 --> 00:03:18.679 there and they set a bookmark there, or 00:03:18.680 --> 00:03:21.959 they put the position in a register... 00:03:21.960 --> 00:03:26.959 but I prefer to keep 00:03:26.960 --> 00:03:29.199 links to everything that is interesting 00:03:29.200 --> 00:03:32.119 as elisp hyperlinks. So, for example, this is 00:03:32.120 --> 00:03:35.079 an elisp hyperlink to a file, that goes 00:03:35.080 --> 00:03:37.599 to this anchor here, and to this string 00:03:37.600 --> 00:03:41.439 after this anchor... this is a variant 00:03:41.440 --> 00:03:44.239 that opens that file in the window 00:03:44.240 --> 00:03:45.479 at the right - 00:03:45.480 --> 00:03:48.879 here... and this is 00:03:48.880 --> 00:03:53.279 a sexp that changes the font. I 00:03:53.280 --> 00:03:56.679 have a command with a very short name 00:03:56.680 --> 00:03:59.439 that does that, but I 00:03:59.440 --> 00:04:02.959 prefer to keep that as a one-liner. 00:04:02.960 --> 00:04:06.919 About the videos... we can see 00:04:06.920 --> 00:04:10.079 the list of first-class videos of eev 00:04:10.080 --> 00:04:14.759 by executing this, M-x find-1stclassvideos, 00:04:14.760 --> 00:04:18.639 or by running this alias here, M-x 1c... 00:04:18.640 --> 00:04:20.679 and then what we see is this... 00:04:20.680 --> 00:04:24.239 the first sexp here 00:04:24.240 --> 00:04:26.959 regenerates this buffer - so we can make a 00:04:26.960 --> 00:04:29.399 mess here and then run this and the 00:04:29.400 --> 00:04:33.519 original buffer is regenerated again in 00:04:33.520 --> 00:04:34.959 a clean way... 00:04:34.960 --> 00:04:36.919 each of these things here 00:04:36.920 --> 00:04:40.879 opens a buffer with information about 00:04:40.880 --> 00:04:43.999 a video... let me take a specific 00:04:44.000 --> 00:04:49.159 example here... this video here is about 00:04:49.160 --> 00:04:51.039 one of the ancestors of this talk, that 00:04:51.040 --> 00:04:53.479 is a library that I wrote 00:04:53.480 --> 00:04:58.479 for creating diagrams in LaTeX using 00:04:58.480 --> 00:05:03.519 a package called Pict2e using REPLs... 00:05:03.520 --> 00:05:03.959 anyway... 00:05:03.960 --> 00:05:06.599 the thing is that if we 00:05:06.600 --> 00:05:11.599 run a sexp like this one and we don't 00:05:11.600 --> 00:05:14.679 have a local copy of the video eev 00:05:14.680 --> 00:05:17.599 will try to download to the local copy - 00:05:17.600 --> 00:05:20.119 and instead of doing that by asking 00:05:20.120 --> 00:05:21.719 something like "do you want me 00:05:21.720 --> 00:05:23.999 to download the local copy? Blah 00:05:24.000 --> 00:05:27.999 blah blah blah blah..." it simply opens a 00:05:28.000 --> 00:05:30.719 buffer like this, I mean, if we don't 00:05:30.720 --> 00:05:33.359 have a local copy yet it will open a 00:05:33.360 --> 00:05:36.359 buffer like this one, in which these 00:05:36.360 --> 00:05:39.199 things here in comments are links to the 00:05:39.200 --> 00:05:43.839 documentation... I mean, this thing here 00:05:43.840 --> 00:05:46.159 explains the idea of local copies 00:05:46.160 --> 00:05:48.759 of files from the internet... 00:05:48.760 --> 00:05:54.759 there are more details here, and here... 00:05:54.760 --> 00:05:57.159 and this is a script that we 00:05:57.160 --> 00:06:00.519 can execute line by line, so instead of 00:06:00.520 --> 00:06:02.359 this script being hidden behind the 00:06:02.360 --> 00:06:06.119 button that we just press after a 00:06:06.120 --> 00:06:08.399 question like "Do you want me to do 00:06:08.400 --> 00:06:10.239 something blah blah blah? Yes or no?" 00:06:10.240 --> 00:06:13.639 the script is visible here and we can 00:06:13.640 --> 00:06:17.599 execute it step by step... it creates a 00:06:17.600 --> 00:06:20.799 terminal with a shell here in the 00:06:20.800 --> 00:06:24.799 right window, and when we type f8 in 00:06:24.800 --> 00:06:26.839 one of these lines here the lines are 00:06:26.840 --> 00:06:29.119 sent... (...) so this is going 00:06:29.120 --> 00:06:31.759 to download a copy of the video... the 00:06:31.760 --> 00:06:36.639 wget says that I already have a copy of 00:06:36.640 --> 00:06:39.479 the video and its subtitles... and so on. 00:06:39.480 --> 00:06:43.879 And after getting a copy of the video 00:06:43.880 --> 00:06:48.159 we can run this sexp here and it displays 00:06:48.160 --> 00:06:49.919 the video. 00:06:49.920 --> 00:06:55.399 I said that eev has lots of 00:06:55.400 --> 00:06:58.399 "videos for people who hate videos", and 00:06:58.400 --> 00:06:59.679 the idea is that very few 00:06:59.680 --> 00:07:02.039 people are going to watch the videos in 00:07:02.040 --> 00:07:06.559 real time... and most of the people that 00:07:06.560 --> 00:07:08.279 I know - or: most of the people that 00:07:08.280 --> 00:07:10.639 are interested in eev in some 00:07:10.640 --> 00:07:13.479 way... they are going to watch just 00:07:13.480 --> 00:07:16.319 small sections of the video, and most of 00:07:16.320 --> 00:07:17.559 the time they're just going to read the 00:07:17.560 --> 00:07:20.799 subtitles of the video. So, for each 00:07:20.800 --> 00:07:23.479 one of the videos we have a page 00:07:23.480 --> 00:07:27.039 about the video... let me see if I 00:07:27.040 --> 00:07:29.999 have internet here... yes. This is a 00:07:30.000 --> 00:07:32.879 page... 00:07:32.880 --> 00:07:38.999 and usually these pages have a link 00:07:39.000 --> 00:07:40.759 to another page that 00:07:40.760 --> 00:07:43.239 has all the subtitles of the 00:07:43.240 --> 00:07:46.639 video... uh, wherever... in this one 00:07:46.640 --> 00:07:48.319 it's not so visible... 00:07:48.320 --> 00:07:50.599 but anyway, there are several 00:07:50.600 --> 00:07:52.479 ways of accessing the subtitles of the 00:07:52.480 --> 00:07:56.239 video, and one of the ways is by running 00:07:56.240 --> 00:07:57.519 this sexp here, 00:07:57.520 --> 00:08:01.559 that opens a file in Lua that is 00:08:01.560 --> 00:08:03.319 what I use to generate the 00:08:03.320 --> 00:08:04.479 subtitles. 00:08:04.480 --> 00:08:08.519 Anyway... by the way, these things... each 00:08:08.520 --> 00:08:12.039 one of these things here is a hyperlink 00:08:12.040 --> 00:08:15.239 to a position of the video, so if I type 00:08:15.240 --> 00:08:18.879 this the right way it goes to that 00:08:18.880 --> 00:08:24.119 position. Anyway, let me go back... 00:08:24.120 --> 00:08:27.279 also, the tutorials of eev... the 00:08:27.280 --> 00:08:31.359 "intros" of eev, that start with "find-" and 00:08:31.360 --> 00:08:34.279 end with "-intro", they have lots of blocks 00:08:34.280 --> 00:08:39.039 that say "[Video links:]", like this one, and 00:08:39.040 --> 00:08:41.359 these blocks have links to positions 00:08:41.360 --> 00:08:43.879 in videos, and if we don't have a local 00:08:43.880 --> 00:08:47.919 copy of the video yet the thing shows 00:08:47.920 --> 00:08:49.799 us a script that lets us download the 00:08:49.800 --> 00:08:50.599 local copy. 00:08:50.600 --> 00:08:54.399 Anyway, I said that I was going 00:08:54.400 --> 00:08:58.759 to explain what I mean by "magic" and 00:08:58.760 --> 00:08:59.639 "black boxes". 00:08:59.640 --> 00:09:03.119 this is something that I've been 00:09:03.120 --> 00:09:05.119 trying to explain for a long time, and I 00:09:05.120 --> 00:09:07.319 think that I got a very good explanation 00:09:07.320 --> 00:09:09.839 about that in a video that I made 00:09:09.840 --> 00:09:13.199 about something called eev-wconfig, that 00:09:13.200 --> 00:09:14.999 is a tool for configuring eev on 00:09:15.000 --> 00:09:19.199 Windows without "magic" - without buttons 00:09:19.200 --> 00:09:22.479 that do things without explaining what 00:09:22.480 --> 00:09:22.879 they're doing. 00:09:22.880 --> 00:09:25.799 This is a part of the subtitles 00:09:25.800 --> 00:09:28.039 of the video, let me read that... 00:09:28.040 --> 00:09:32.319 eev-wconfig is an attempt to solve the 00:09:32.320 --> 00:09:35.039 problem of how to install these things 00:09:35.040 --> 00:09:37.279 on Windows both without magic and with 00:09:37.280 --> 00:09:37.879 very little 00:09:37.880 --> 00:09:41.679 magic. Remember this slogan: "any 00:09:41.680 --> 00:09:44.359 sufficiently advanced technology is 00:09:44.360 --> 00:09:45.399 indistinguishable from 00:09:45.400 --> 00:09:49.679 magic". Here in this video I'm going to 00:09:49.680 --> 00:09:52.199 use the term magic as a shorthand 00:09:52.200 --> 00:09:55.239 for sufficiently advanced technology, 00:09:55.240 --> 00:09:57.799 that is something that is complex and 00:09:57.800 --> 00:10:00.199 non-obvious and that is 00:10:00.200 --> 00:10:02.119 indistinguishable from magic in the 00:10:02.120 --> 00:10:04.119 sense of being almost impossible to 00:10:04.120 --> 00:10:07.439 understand. And I'm also going to use a 00:10:07.440 --> 00:10:10.279 the term "black box" as a near-synonym for 00:10:10.280 --> 00:10:13.079 magic, and sometimes the term 00:10:13.080 --> 00:10:16.039 "black box" is more convenient even though 00:10:16.040 --> 00:10:17.919 it's a bit longer - it has more 00:10:17.920 --> 00:10:20.159 letters - because when I use the term 00:10:20.160 --> 00:10:22.599 black box it invites us to use 00:10:22.600 --> 00:10:25.479 expressions like "opening the black box", 00:10:25.480 --> 00:10:26.639 and I'm going to use that 00:10:26.640 --> 00:10:28.039 expression a lot. 00:10:28.040 --> 00:10:37.399 Now let me try to explain what is... 00:10:37.400 --> 00:10:41.039 sorry, let me change the font... 00:10:41.040 --> 00:10:45.479 what is Lua. Lua is a minimalistic 00:10:45.480 --> 00:10:49.999 language, in the sense of 00:10:50.000 --> 00:10:53.679 "batteries not included"... it uses 00:10:53.680 --> 00:10:55.999 associative tables for most of its data 00:10:56.000 --> 00:10:56.799 structures... 00:10:56.800 --> 00:11:00.239 and it is so minimalistic 00:11:00.240 --> 00:11:03.999 that its default print function, when 00:11:04.000 --> 00:11:06.679 we tell... when we create an associative 00:11:06.680 --> 00:11:09.559 table and we ask it to print... 00:11:09.560 --> 00:11:13.319 when we ask "print" to print an 00:11:13.320 --> 00:11:15.719 associative table it just prints the 00:11:15.720 --> 00:11:17.879 address of the table. Here are some 00:11:17.880 --> 00:11:21.599 examples... here is a table, and when we 00:11:21.600 --> 00:11:24.679 ask "print" to print it it just says 00:11:24.680 --> 00:11:26.839 that it's the table at this address here. 00:11:26.840 --> 00:11:30.119 So, one of things that that most 00:11:30.120 --> 00:11:32.919 people do when they start using Lua is 00:11:32.920 --> 00:11:35.239 that either they download a package with 00:11:35.240 --> 00:11:37.079 a pretty-printing function or they write 00:11:37.080 --> 00:11:39.359 their own pretty-printing functions. My 00:11:39.360 --> 00:11:41.519 own pretty-printing function is called 00:11:41.520 --> 00:11:46.159 PP, with upper case letters, and it works 00:11:46.160 --> 00:11:47.679 like this... 00:11:47.680 --> 00:11:50.279 and it prints associative tables 00:11:50.280 --> 00:11:53.839 in a way like this. It says that for 00:11:53.840 --> 00:11:57.479 the key 1 the the value associated to 00:11:57.480 --> 00:12:00.039 it is 2, for the key 2 the value is 00:12:00.040 --> 00:12:04.959 3, and for the key 3 the value is 5. 00:12:04.960 --> 00:12:11.079 When I started using Lua one of my 00:12:11.080 --> 00:12:13.839 favorite languages was also a language 00:12:13.840 --> 00:12:15.639 that used associative tables a lot - 00:12:15.640 --> 00:12:16.839 it was called Icon... 00:12:16.840 --> 00:12:21.039 and I had to write my own 00:12:21.040 --> 00:12:23.839 pretty-printing functions for Icon, so 00:12:23.840 --> 00:12:27.319 I just had to port my pretty-printing 00:12:27.320 --> 00:12:29.719 functions to Lua... and my first 00:12:29.720 --> 00:12:32.239 version looked at something like this... it 00:12:32.240 --> 00:12:35.999 just had some some global functions... lots 00:12:36.000 --> 00:12:39.639 of them, actually... 00:12:39.640 --> 00:12:42.279 and after a while I rewrote it, and I 00:12:42.280 --> 00:12:44.599 rewrote it again, and again, and again, and 00:12:44.600 --> 00:12:47.279 this is one of the versions of that, 00:12:47.280 --> 00:12:49.879 is not even the default at this 00:12:49.880 --> 00:12:51.559 point... 00:12:51.560 --> 00:12:54.119 "Tos" is for "to string"... 00:12:54.120 --> 00:12:58.279 and this is a demo... 00:12:58.280 --> 00:13:01.919 it's very modular, so it's easy to replace 00:13:01.920 --> 00:13:05.639 parts of it, or to toggle flags... and this 00:13:05.640 --> 00:13:08.119 is an example. If I try to print the 00:13:08.120 --> 00:13:09.999 table of methods for a certain 00:13:10.000 --> 00:13:14.359 class... I will need a smaller font... 00:13:14.360 --> 00:13:16.799 it prints the table like this, with the 00:13:16.800 --> 00:13:20.039 names of the methods and then links to 00:13:20.040 --> 00:13:21.719 the source code of the functions... 00:13:21.720 --> 00:13:25.399 these links only make sense in Emacs and 00:13:25.400 --> 00:13:25.959 in eev... 00:13:25.960 --> 00:13:30.359 and when we run a link like this one... 00:13:30.360 --> 00:13:32.039 it shows the source code in the 00:13:32.040 --> 00:13:35.079 window at the right. So, for some 00:13:35.080 --> 00:13:37.879 functions the source code is three lines, 00:13:37.880 --> 00:13:39.999 for other ones it's one line... and 00:13:40.000 --> 00:13:44.879 whatever. Anyway, let me go 00:13:44.880 --> 00:13:47.039 back... Lua can be used in many different 00:13:47.040 --> 00:13:50.359 styles... most people hate other people's 00:13:50.360 --> 00:13:53.839 styles... when I started using it in the 00:13:53.840 --> 00:13:57.679 year 2000 I learned most of the basic 00:13:57.680 --> 00:13:59.839 language in a single day - it was very 00:13:59.840 --> 00:14:02.119 similar to things that I was already 00:14:02.120 --> 00:14:05.639 using... and then I rewrote the the mini- 00:14:05.640 --> 00:14:10.279 language that I was using to 00:14:10.280 --> 00:14:13.559 generate the HTML for my pages 00:14:13.560 --> 00:14:16.199 in Lua... actually I had to rewrite it 00:14:16.200 --> 00:14:18.079 many times, but the first version I 00:14:18.080 --> 00:14:21.519 certainly did in my first weeks or first 00:14:21.520 --> 00:14:22.519 months using Lua... 00:14:22.520 --> 00:14:27.279 In the beginning I was just using 00:14:27.280 --> 00:14:30.159 it for writing programs that either 00:14:30.160 --> 00:14:32.759 didn't take any input at all - because 00:14:32.760 --> 00:14:35.079 the input was already in the source file - 00:14:35.080 --> 00:14:40.599 or that worked as Unix programs, 00:14:40.600 --> 00:14:43.199 that would read files 00:14:43.200 --> 00:14:45.279 and process these files in some way 00:14:45.280 --> 00:14:48.999 and output something. 00:14:49.000 --> 00:14:52.319 I mentioned the "basic language" here... 00:14:52.320 --> 00:14:54.719 I only learned how to use closures, 00:14:54.720 --> 00:14:58.479 metatables, and coroutines many years later... 00:14:58.480 --> 00:15:02.399 in the beginning, when I started using Lua, 00:15:02.400 --> 00:15:04.199 it didn't have a package manager... 00:15:04.200 --> 00:15:06.799 it appeared later, it is called 00:15:06.800 --> 00:15:10.119 Luarocks... it has had this package 00:15:10.120 --> 00:15:13.279 manager for several years, most 00:15:13.280 --> 00:15:15.719 of the rocks for Luarocks are poorly 00:15:15.720 --> 00:15:18.959 documented and hacker-unfriendly, 00:15:18.960 --> 00:15:21.199 so you can't rely just on the 00:15:21.200 --> 00:15:23.679 documentation and you can't rely just on the 00:15:23.680 --> 00:15:26.719 source code, because, I mean... if you are 00:15:26.720 --> 00:15:29.159 a genius of course you can, but for 00:15:29.160 --> 00:15:31.479 people who are either lazy, or dumb, or 00:15:31.480 --> 00:15:34.279 whatever, like me, or unfocused... 00:15:34.280 --> 00:15:36.759 the source code is hard to 00:15:36.760 --> 00:15:38.959 understand and hard to tinker with. 00:15:38.960 --> 00:15:43.319 Some rocks are excellent. The 00:15:43.320 --> 00:15:46.599 best rocks are well documented 00:15:46.600 --> 00:15:48.719 but they are hacker-unfriendly 00:15:48.720 --> 00:15:50.999 in a sense that I hope that 00:15:51.000 --> 00:15:52.879 I'll be able to explain soon. 00:15:52.880 --> 00:15:54.999 The best rocks use local 00:15:55.000 --> 00:15:58.799 variables and metatables a lot - 00:15:58.800 --> 00:16:02.519 so if you are beginner 00:16:02.520 --> 00:16:03.799 learning Lua you're not going to 00:16:03.800 --> 00:16:06.159 understand what their source code do... 00:16:06.160 --> 00:16:08.519 they use lots of dirty tricks. NOTE Object orientation in Lua 00:16:08.520 --> 00:16:12.479 Let me talk a bit about object 00:16:12.480 --> 00:16:14.959 orientation in Lua. It can be done in 00:16:14.960 --> 00:16:15.879 many ways... 00:16:15.880 --> 00:16:18.999 the main book about Lua, called 00:16:19.000 --> 00:16:21.439 "Programming in Lua", by one of the authors 00:16:21.440 --> 00:16:23.959 of the language, Roberto Ierusalimschy, 00:16:23.960 --> 00:16:26.679 presents several ways of doing 00:16:26.680 --> 00:16:29.279 object orientation in Lua... I hated all 00:16:29.280 --> 00:16:33.199 of these ways - and also the ways that I 00:16:33.200 --> 00:16:34.519 tried from the rocks. 00:16:34.520 --> 00:16:38.559 And then I wrote my own way 00:16:38.560 --> 00:16:40.639 of doing object orientation in Lua... it's 00:16:40.640 --> 00:16:43.559 very minimalistic, it's in this file here, 00:16:43.560 --> 00:16:48.679 eoo.lua... the main code is just this five 00:16:48.680 --> 00:16:49.639 lines here... 00:16:49.640 --> 00:16:53.439 and here's an example of how it works. 00:16:53.440 --> 00:16:58.439 Here we define the class Vector, 00:16:58.440 --> 00:17:02.719 with some metamethods... 00:17:02.720 --> 00:17:05.959 this metamethod here will tell Lua 00:17:05.960 --> 00:17:08.319 what to do when the 00:17:08.320 --> 00:17:12.639 user asks to add two vectors, this one 00:17:12.640 --> 00:17:15.919 here tells Lua what to do when the user 00:17:15.920 --> 00:17:18.479 asks Lua to convert a vector to a string, 00:17:18.480 --> 00:17:21.439 and... whatever, this one is 00:17:21.440 --> 00:17:24.039 something that I'm going to explain in a 00:17:24.040 --> 00:17:27.479 second. So, here we create a vector with 00:17:27.480 --> 00:17:30.279 these coordinates, 3 and 4... here we create 00:17:30.280 --> 00:17:33.319 another Vector... if we "print" here then Lua 00:17:33.320 --> 00:17:36.439 uses this function here, in the __tostring... 00:17:36.440 --> 00:17:39.759 if we add the two vectors it uses this 00:17:39.760 --> 00:17:43.479 function here, in the __add metamethod, and 00:17:43.480 --> 00:17:45.359 if we run the method :norm... 00:17:45.360 --> 00:17:49.959 it is defined here, in the table __index. 00:17:49.960 --> 00:17:57.999 Anyway... 00:17:58.000 --> 00:18:02.439 Even this thing being so small I used 00:18:02.440 --> 00:18:04.719 to forget how its innards worked all 00:18:04.720 --> 00:18:08.119 the time. Actually I always forget how 00:18:08.120 --> 00:18:09.759 things work and I have to remember them 00:18:09.760 --> 00:18:12.479 somehow... and I have to have 00:18:12.480 --> 00:18:15.959 tricks for remembering, and tricks for 00:18:15.960 --> 00:18:18.719 summarizing things, and diagrams, and so 00:18:18.720 --> 00:18:22.199 on. And every time that I forgot how this 00:18:22.200 --> 00:18:24.799 thing worked I went back to the 00:18:24.800 --> 00:18:26.879 source code, and then I looked at the 00:18:26.880 --> 00:18:29.039 diagrams... or, of course, in the 00:18:29.040 --> 00:18:31.719 first times I had to draw the diagrams... 00:18:31.720 --> 00:18:35.239 and I run the examples, and of course in 00:18:35.240 --> 00:18:36.479 in the beginning I thought that the code 00:18:36.480 --> 00:18:39.119 was clear and my examples were very 00:18:39.120 --> 00:18:41.559 brief, and so I had to rewrite the 00:18:41.560 --> 00:18:44.719 examples many times until they became, 00:18:44.720 --> 00:18:45.639 let's say... 00:18:45.640 --> 00:18:47.759 perfect. 00:18:47.760 --> 00:18:52.599 I was saying that Lua can be used in 00:18:52.600 --> 00:18:56.359 many ways, and in my way of using Lua - in 00:18:56.360 --> 00:18:59.439 my favorite way - everything can be 00:18:59.440 --> 00:19:02.159 inspected and modified from REPLs, 00:19:02.160 --> 00:19:06.319 like we can do in Emacs and in SmallTalk, 00:19:06.320 --> 00:19:08.519 or sort of. So, in my 00:19:08.520 --> 00:19:10.239 favorite way of using Lua there's no 00:19:10.240 --> 00:19:12.679 security at all, everything can be 00:19:12.680 --> 00:19:14.919 changed at all times. 00:19:14.920 --> 00:19:19.119 Of course most people hate that... NOTE My init file 00:19:19.120 --> 00:19:22.599 My init file has lots of classes... by the 00:19:22.600 --> 00:19:26.079 way, instead of keeping many small files 00:19:26.080 --> 00:19:29.639 with many things I put lots of stuff 00:19:29.640 --> 00:19:31.279 in just one big init file. 00:19:31.280 --> 00:19:34.599 My init file has lots of classes, 00:19:34.600 --> 00:19:37.959 and lots of global functions, and 00:19:37.960 --> 00:19:41.799 lots of cruft - and people hate that, 00:19:41.800 --> 00:19:44.559 of course. This is an example... 00:19:44.560 --> 00:19:46.439 this is the index at the top 00:19:46.440 --> 00:19:48.359 of my init file, 00:19:48.360 --> 00:19:53.119 the classes start here, and then 00:19:53.120 --> 00:19:59.119 we have some functions, and 00:19:59.120 --> 00:20:01.199 then we have functions that load 00:20:01.200 --> 00:20:03.839 certain packages, and then we have... cruft. 00:20:03.840 --> 00:20:04.919 Whatever. 00:20:04.920 --> 00:20:08.119 Most people think that my style 00:20:08.120 --> 00:20:10.279 of using Lua is dirty, and dangerous... 00:20:10.280 --> 00:20:12.959 and they wouldn't touch my Lua code 00:20:12.960 --> 00:20:15.479 with a 10 feet pole... but most of the 00:20:15.480 --> 00:20:18.599 things that I'm going to present here in 00:20:18.600 --> 00:20:23.199 this presentation are ideas that should 00:20:23.200 --> 00:20:28.159 be easy to port to other environments 00:20:28.160 --> 00:20:32.279 and other languages, especially the 00:20:32.280 --> 00:20:35.279 diagrams... so the code is not so important. NOTE LaTeX and LuaLaTeX 00:20:35.280 --> 00:20:39.039 Now let me talk a bit about LuaLaTeX, 00:20:39.040 --> 00:20:41.359 that is LaTeX with a Lua interpreter 00:20:41.360 --> 00:20:44.559 embedded inside, and two ways 00:20:44.560 --> 00:20:48.839 of generating pictures in LaTeX: TikZ, 00:20:48.840 --> 00:20:54.439 that is very famous, and Pict2e, that is not 00:20:54.440 --> 00:20:57.359 very famous and that is very low level... 00:20:57.360 --> 00:21:02.359 and I think that not many people use it. 00:21:02.360 --> 00:21:04.119 I said before that when I 00:21:04.120 --> 00:21:06.919 learned Lua I realized that it was 00:21:06.920 --> 00:21:09.199 very good for writing little 00:21:09.200 --> 00:21:14.919 languages. I was doing my PhD at the 00:21:14.920 --> 00:21:19.839 time and typesetting the diagrams for 00:21:19.840 --> 00:21:24.039 my PhD thesis was very boring, so 00:21:24.040 --> 00:21:29.879 one of the things that I did was that I 00:21:29.880 --> 00:21:34.439 created a little language for typesetting 00:21:34.440 --> 00:21:36.359 the diagrams for me. it was 00:21:36.360 --> 00:21:38.879 called Dednat because initially 00:21:38.880 --> 00:21:41.039 it only generated diagrams for 00:21:41.040 --> 00:21:43.759 Natural Deduction, and then it had 00:21:43.760 --> 00:21:45.119 several versions... 00:21:45.120 --> 00:21:46.679 these are the slides for my 00:21:46.680 --> 00:21:52.159 presentation about Dednat6... "Dednat6 is 00:21:52.160 --> 00:21:56.159 an extensible semi-preprocessor for 00:21:56.160 --> 00:22:01.159 LuaLaTeX that understands diagrams in 00:22:01.160 --> 00:22:05.359 ASCII art"... in the sense that when I have 00:22:05.360 --> 00:22:11.399 a .tex file that has this, and when 00:22:11.400 --> 00:22:13.279 Dednat6 is loaded, 00:22:13.280 --> 00:22:15.039 when I give the right commands 00:22:15.040 --> 00:22:19.559 Dednat6 interprets this block here as 00:22:19.560 --> 00:22:22.559 something that defines this 00:22:22.560 --> 00:22:28.239 diagram... oops, sorry, it interprets this 00:22:28.240 --> 00:22:30.599 diagram here, this diagram in 00:22:30.600 --> 00:22:34.879 comments here, as something that defines 00:22:34.880 --> 00:22:39.759 a diagram called foo... a deduction called 00:22:39.760 --> 00:22:41.439 foo, and it generates this code here... 00:22:41.440 --> 00:22:44.239 so that we can just invoke 00:22:44.240 --> 00:22:45.519 the definition of the 00:22:45.520 --> 00:22:47.719 deduction by typing \ded{foo}. 00:22:47.720 --> 00:22:50.759 And Dednat6 also 00:22:50.760 --> 00:22:57.159 supports another language for typesetting 00:22:57.160 --> 00:22:59.639 bidimensional diagrams with 00:22:59.640 --> 00:23:05.399 arrows and stuff for category Theory and 00:23:05.400 --> 00:23:08.519 blah blah blah... the specifications of 00:23:08.520 --> 00:23:12.039 these diagrams look like this... 00:23:12.040 --> 00:23:14.559 here is a... sorry, here is a very good 00:23:14.560 --> 00:23:16.719 example, this is a huge diagram... 00:23:16.720 --> 00:23:18.599 sorry, one second... 00:23:18.600 --> 00:23:20.399 so, the source code that generates 00:23:20.400 --> 00:23:25.119 this diagram here is just this thing at 00:23:25.120 --> 00:23:32.039 the left, so it's very visual... we can 00:23:32.040 --> 00:23:35.679 typeset the diagram in ASCII art here and 00:23:35.680 --> 00:23:38.759 then in this part here we tell how 00:23:38.760 --> 00:23:41.279 the nodes are to be joined, which 00:23:41.280 --> 00:23:43.799 arrows have to to have annotations, and 00:23:43.800 --> 00:23:45.039 so on... 00:23:45.040 --> 00:23:46.799 and this language is extensible in 00:23:46.800 --> 00:23:48.679 the sense that... uh, where's that... 00:23:48.680 --> 00:23:52.559 here: comments that start with "%:" 00:23:52.560 --> 00:23:54.119 are interpreted as 00:23:54.120 --> 00:23:56.079 definitions for tree diagrams, 00:23:56.080 --> 00:23:58.559 lines that start with "%D" 00:23:58.560 --> 00:24:00.639 define 2D diagrams with arrows and 00:24:00.640 --> 00:24:04.279 stuff, and lines that start with "%L" 00:24:04.280 --> 00:24:06.759 contain blocks of Lua code 00:24:06.760 --> 00:24:09.479 that we can use to extend the interpreter 00:24:09.480 --> 00:24:10.439 on-the-fly... 00:24:10.440 --> 00:24:12.679 anyway, here are some recent 00:24:12.680 --> 00:24:15.319 examples of diagrams that I used 00:24:15.320 --> 00:24:19.839 Dednat6 to typeset... this diagram 00:24:19.840 --> 00:24:21.919 here was generated by this 00:24:21.920 --> 00:24:22.879 specification here... 00:24:22.880 --> 00:24:27.239 and this diagram here with the 00:24:27.240 --> 00:24:30.719 curved arrows was generated by this 00:24:30.720 --> 00:24:32.719 specification here. 00:24:32.720 --> 00:24:39.079 So, Dednat6 was very easy to extend, 00:24:39.080 --> 00:24:41.879 and at some point I started to use it 00:24:41.880 --> 00:24:44.679 to generate diagrams using Pict2e - 00:24:44.680 --> 00:24:47.359 mainly for the classes that I give 00:24:47.360 --> 00:24:50.079 at the University... I teach mathematics and 00:24:50.080 --> 00:24:57.239 whatever... in a bad place. Whatever... 00:24:57.240 --> 00:25:00.039 Let me show an animation... here is a 00:25:00.040 --> 00:25:02.479 diagram that I generated with Dednat6, 00:25:02.480 --> 00:25:06.319 and it is a flip book animation, like... we 00:25:06.320 --> 00:25:09.279 type PgUp and PgDn and we go 00:25:09.280 --> 00:25:11.119 to the next page of the book and to the 00:25:11.120 --> 00:25:12.439 previous page of the book... 00:25:12.440 --> 00:25:16.279 and here is the source code that generates 00:25:16.280 --> 00:25:19.159 that. This source code is not very visual, 00:25:19.160 --> 00:25:22.559 so it's quite clumsy to edit that 00:25:22.560 --> 00:25:27.519 diagram directly in the .tex file like 00:25:27.520 --> 00:25:28.079 that... NOTE Manim 00:25:28.080 --> 00:25:30.199 These diagrams were inspired 00:25:30.200 --> 00:25:33.039 by something called my Manim, that... 00:25:33.040 --> 00:25:37.559 I forgot the name of the guy, but 00:25:37.560 --> 00:25:41.479 it's a guy that makes many videos about 00:25:41.480 --> 00:25:44.839 Mathematics, and he created this library 00:25:44.840 --> 00:25:48.599 called Manim for generating his 00:25:48.600 --> 00:25:51.839 animations, and other people adapted 00:25:51.840 --> 00:25:55.919 his library to make it more accessible... 00:25:55.920 --> 00:25:59.359 I tried to learn it, but 00:25:59.360 --> 00:26:01.199 each animation, even an animation 00:26:01.200 --> 00:26:03.679 that has very few frames... each 00:26:03.680 --> 00:26:07.319 animation took ages to render, so it 00:26:07.320 --> 00:26:11.159 wasn't fun... and animations in PDFs can 00:26:11.160 --> 00:26:13.639 be rendered in seconds. So these 00:26:13.640 --> 00:26:18.679 things were fun for me, because my laptop 00:26:18.680 --> 00:26:24.359 is very very slow, and my Manim was not fun. NOTE Generating diagrams from REPLs 00:26:24.360 --> 00:26:27.359 Anyway, writing code like this 00:26:27.360 --> 00:26:32.719 inside a .tex file was not very 00:26:32.720 --> 00:26:35.519 fun because it was hard to 00:26:35.520 --> 00:26:38.719 debug... so in 2022 I started to play 00:26:38.720 --> 00:26:41.319 with ways of generating these 00:26:41.320 --> 00:26:43.839 diagrams from REPLs, and I found a 00:26:43.840 --> 00:26:47.319 way for Pict2e and a way for TikZ... 00:26:47.320 --> 00:26:50.159 each one of these ways became a video... 00:26:50.160 --> 00:26:53.679 if you go to the list of first-class 00:26:53.680 --> 00:26:57.719 videos of eev you're going to see 00:26:57.720 --> 00:26:59.919 that there's a video about Pict2e here 00:26:59.920 --> 00:27:03.399 here and a video about TikZ... 00:27:03.400 --> 00:27:05.759 here you have some some information 00:27:05.760 --> 00:27:09.839 like length, an explanation, etc... 00:27:09.840 --> 00:27:11.719 and here are the pages for these videos. 00:27:11.720 --> 00:27:15.999 My page about the video about Pict2e 00:27:16.000 --> 00:27:20.079 looks like this, it has some diagrams... 00:27:20.080 --> 00:27:23.919 whatever... and this one is much 00:27:23.920 --> 00:27:26.679 nicer, and a lot of people 00:27:26.680 --> 00:27:30.599 watched that video... I mean, I think 00:27:30.600 --> 00:27:33.719 that 250 people watched it - for me that's 00:27:33.720 --> 00:27:35.599 a million of people... 00:27:35.600 --> 00:27:39.159 and this video is about how to 00:27:39.160 --> 00:27:44.079 extract diagrams from the manual... from 00:27:44.080 --> 00:27:46.599 the TikZ manual and how to run those 00:27:46.600 --> 00:27:49.759 examples in a REPL and modify 00:27:49.760 --> 00:27:53.159 them bit by bit... this is a a 00:27:53.160 --> 00:27:57.439 screenshot... but let me go back. 00:27:57.440 --> 00:28:00.959 At that point these things were just 00:28:00.960 --> 00:28:03.239 prototypes, the code was not very nice... 00:28:03.240 --> 00:28:07.519 and in this year I wrote... I was able 00:28:07.520 --> 00:28:12.399 to unify those two ways of generating PDFs, 00:28:12.400 --> 00:28:16.039 the one for TikZ and the one for Pict2e, 00:28:16.040 --> 00:28:18.719 and I unified them with many other 00:28:18.720 --> 00:28:20.879 things that generated diagrams. 00:28:20.880 --> 00:28:24.279 The basis of these things is 00:28:24.280 --> 00:28:29.319 something called Show2.lua... I'm not going 00:28:29.320 --> 00:28:35.759 to show its details now, but its 00:28:35.760 --> 00:28:39.079 extension that generates TikZ code 00:28:39.080 --> 00:28:43.039 is just this, so we can specify a 00:28:43.040 --> 00:28:45.799 diagram with just a block like this, 00:28:45.800 --> 00:28:49.079 and then uh if we 00:28:49.080 --> 00:28:54.239 run :show00() it returns a string 00:28:54.240 --> 00:28:56.199 that is just the body... the inner 00:28:56.200 --> 00:29:00.279 body of the .tex file, if we run this we 00:29:00.280 --> 00:29:02.999 see the whole .tex file, and if we run 00:29:03.000 --> 00:29:05.119 this we save the .tex file and we 00:29:05.120 --> 00:29:08.119 compile the .tex file to generate a PDF... 00:29:08.120 --> 00:29:10.959 and if we run this we show the PDF in 00:29:10.960 --> 00:29:14.239 the lower right window. 00:29:14.240 --> 00:29:17.759 And that's the same thing for all 00:29:17.760 --> 00:29:20.199 my recent programs that generate 00:29:20.200 --> 00:29:22.439 PDFs - they are all 00:29:22.440 --> 00:29:26.199 integrated... here is the one that... 00:29:26.200 --> 00:29:29.359 the basis for all my modules that generate 00:29:29.360 --> 00:29:30.719 diagrams with Pict2e... 00:29:30.720 --> 00:29:34.879 its demos are not very interesting, 00:29:34.880 --> 00:29:36.799 so let me show some demos of 00:29:36.800 --> 00:29:39.759 extensions that do interesting things... 00:29:39.760 --> 00:29:45.319 so, this is a diagram that I created 00:29:45.320 --> 00:29:47.479 by editing it in a REPL... 00:29:47.480 --> 00:29:51.279 I create several Pict objects here... 00:29:51.280 --> 00:29:54.479 and if I execute this it 00:29:54.480 --> 00:29:59.959 compiles an object, generates a PDF, and 00:29:59.960 --> 00:30:04.759 if I tap this... here is the PDF. 00:30:04.760 --> 00:30:07.599 And if I just ask Lua to 00:30:07.600 --> 00:30:10.079 display what is "pux", here, 00:30:10.080 --> 00:30:15.719 it shows the source code in Pict2e 00:30:15.720 --> 00:30:17.999 of the diagram... and the 00:30:18.000 --> 00:30:20.959 nice thing is that it is indented, so 00:30:20.960 --> 00:30:23.599 it's easy to debug the Pict2e code. 00:30:23.600 --> 00:30:25.919 If anyone is interested the 00:30:25.920 --> 00:30:28.639 module that does the tricks for 00:30:28.640 --> 00:30:31.879 indentation is very easy to understand... 00:30:31.880 --> 00:30:35.959 it has lots of tests and test blocks, 00:30:35.960 --> 00:30:38.599 and I think that its data 00:30:38.600 --> 00:30:42.079 structures are easy to understand. 00:30:42.080 --> 00:30:44.359 Anyway... here is another 00:30:44.360 --> 00:30:51.359 example. The :show() is 00:30:51.360 --> 00:30:56.439 here... it generates a 3D diagram. NOTE Parsers 00:30:56.440 --> 00:31:06.279 Now let me talk about parsers and 00:31:06.280 --> 00:31:09.559 REPLs in VERY strange places... I mean, 00:31:09.560 --> 00:31:13.359 using REPLs to build parsers step by step 00:31:13.360 --> 00:31:17.959 and" replacing parts by more complex 00:31:17.960 --> 00:31:23.039 parts. So, I said that Lua is very 00:31:23.040 --> 00:31:28.279 minimalistic, and everybody knows that 00:31:28.280 --> 00:31:30.759 implementations of regular expressions 00:31:30.760 --> 00:31:32.479 are big and complex.. 00:31:32.480 --> 00:31:34.679 so, instead of coming with 00:31:34.680 --> 00:31:37.439 full regular expressions Lua comes with 00:31:37.440 --> 00:31:39.879 something called "patterns" and a 00:31:39.880 --> 00:31:43.839 library function called "string.match". 00:31:43.840 --> 00:31:44.599 Here is 00:31:44.600 --> 00:31:50.319 a copy of the part of the manual that 00:31:50.320 --> 00:31:53.399 explains the syntax... a part of the 00:31:53.400 --> 00:31:57.159 syntax of of patterns... here's how 00:31:57.160 --> 00:31:59.279 string.match is described in the 00:31:59.280 --> 00:32:03.199 manual - it's just this... "looks for 00:32:03.200 --> 00:32:05.359 the first match of pattern in the string 00:32:05.360 --> 00:32:08.039 as blah blah blah"... and then we have to 00:32:08.040 --> 00:32:10.159 go to the other section of the menual 00:32:10.160 --> 00:32:11.479 that explains patterns. 00:32:11.480 --> 00:32:20.079 Lua patterns are so simple, 00:32:20.080 --> 00:32:23.159 so limited, that they don't even 00:32:23.160 --> 00:32:26.519 have the the alternation operator... 00:32:26.520 --> 00:32:29.759 here is how it is described in the 00:32:29.760 --> 00:32:31.599 elisp manual - 00:32:31.600 --> 00:32:36.039 backslash-pipe specifies 00:32:36.040 --> 00:32:40.359 an alternative, blah blah blah. 00:32:40.360 --> 00:32:42.879 When we want to to build more 00:32:42.880 --> 00:32:45.319 complex... regular expressions, 00:32:45.320 --> 00:32:49.199 patterns, grammars, etc... we have to use 00:32:49.200 --> 00:32:52.679 an external library for that... no, 00:32:52.680 --> 00:32:56.279 sorry, a library that is external 00:32:56.280 --> 00:32:58.239 but that was written by one of the 00:32:58.240 --> 00:33:00.879 authors of Lua itself. This library 00:33:00.880 --> 00:33:05.879 is called Lpeg, and its manual says... 00:33:05.880 --> 00:33:09.599 "Lpeg is a new pattern matching library for 00:33:09.600 --> 00:33:12.039 Lua based on Parsing Expression Grammars 00:33:12.040 --> 00:33:18.759 (PEGs)". The manual is very terse, I 00:33:18.760 --> 00:33:21.559 found it incredibly hard to read... it 00:33:21.560 --> 00:33:25.439 doesn't have any diagrams - it has some 00:33:25.440 --> 00:33:29.759 examples, though... and the Lua Wiki 00:33:29.760 --> 00:33:33.879 has a big page called Lpeg Tutorial 00:33:33.880 --> 00:33:35.359 with lots of examples... 00:33:35.360 --> 00:33:38.879 but it it also doesn't have 00:33:38.880 --> 00:33:41.199 diagrams and I found some things 00:33:41.200 --> 00:33:42.719 incredibly hard to understand. 00:33:42.720 --> 00:33:45.879 For example, this is something that is in 00:33:45.880 --> 00:33:48.879 the the manual of Lpeg that I saw and I 00:33:48.880 --> 00:33:51.639 thought: "Wow, great! This makes all sense 00:33:51.640 --> 00:33:53.159 and is going to be very useful!"... 00:33:53.160 --> 00:33:54.199 it's a way to to build 00:33:54.200 --> 00:33:57.199 grammars that can be recursive, 00:33:57.200 --> 00:34:01.359 and they sort of can encode BNF 00:34:01.360 --> 00:34:03.439 grammars... we just have to translate the 00:34:03.440 --> 00:34:06.479 BNF a bit to get rid of some 00:34:06.480 --> 00:34:08.079 recursions and to translate them to 00:34:08.080 --> 00:34:08.999 something else. 00:34:09.000 --> 00:34:11.919 And the manual also has some things 00:34:11.920 --> 00:34:15.159 that I thought: "Oh, no! I don't have any 00:34:15.160 --> 00:34:18.359 idea of what this thing does"... and in fact 00:34:18.360 --> 00:34:20.399 I saw these things for the first 00:34:20.400 --> 00:34:22.359 time more than 10 years ago and they 00:34:22.360 --> 00:34:26.079 only started to make sense one year ago. 00:34:26.080 --> 00:34:30.519 One example is group captures. 00:34:30.520 --> 00:34:36.359 Lpeg also comes with a 00:34:36.360 --> 00:34:38.719 module called the Re module... let me 00:34:38.720 --> 00:34:41.719 pronounce as it in Portuguese - the Re 00:34:41.720 --> 00:34:45.759 module... its manual says: "The Re 00:34:45.760 --> 00:34:48.199 module (provided by the file re.lua in the 00:34:48.200 --> 00:34:51.159 distribution) supports a somewhat conventional 00:34:51.160 --> 00:34:56.239 regular expression syntax for pattern usage 00:34:56.240 --> 00:34:58.679 within lpeg"... and 00:34:58.680 --> 00:35:03.519 this is a quick reference... this 00:35:03.520 --> 00:35:06.319 thing is very brief, it has some nice 00:35:06.320 --> 00:35:08.919 examples but it's hard to understand anyway... 00:35:08.920 --> 00:35:13.199 and here are some comments about 00:35:13.200 --> 00:35:17.279 my attempts to learn Re.lua. This is 00:35:17.280 --> 00:35:20.639 a class... in this case it's a very small 00:35:20.640 --> 00:35:24.839 class... this file implements a :pm() 00:35:24.840 --> 00:35:28.679 method - I'm going to show examples of 00:35:28.680 --> 00:35:32.239 other :pm() methods very soon - so, this is 00:35:32.240 --> 00:35:35.799 a :pm() method for Re.lua that lets us 00:35:35.800 --> 00:35:38.719 compare the syntax of Lua patterns, Lpeg, 00:35:38.720 --> 00:35:43.999 and Re... let's see this example here... so, 00:35:44.000 --> 00:35:47.319 if we run this it loads my version of 00:35:47.320 --> 00:35:52.799 lpeg... no, sorry, my version of lpegrex... 00:35:52.800 --> 00:35:57.119 and it shows that when we apply 00:35:57.120 --> 00:36:01.199 the :pm() method to this Lua pattern, this 00:36:01.200 --> 00:36:04.879 lpeg pattern, and this Re pattern 00:36:04.880 --> 00:36:07.999 they all give the same results. So we can 00:36:08.000 --> 00:36:10.799 use this thing... this kind of thing here 00:36:10.800 --> 00:36:14.119 to show how to translate from Lua 00:36:14.120 --> 00:36:16.519 patterns, that are familiar because 00:36:16.520 --> 00:36:18.519 they're similar to regular expressions, 00:36:18.520 --> 00:36:20.199 only weaker... 00:36:20.200 --> 00:36:24.799 to lpeg, that is super weird 00:36:24.800 --> 00:36:27.759 and to Re, that is not so weird. 00:36:27.760 --> 00:36:35.159 Anyway, the comment says that in 2012 00:36:35.160 --> 00:36:37.519 I had a project that needed a 00:36:37.520 --> 00:36:40.479 precedence passer that could parse 00:36:40.480 --> 00:36:43.239 arithmetical expressions with the right 00:36:43.240 --> 00:36:46.639 precedences... and at that point I was 00:36:46.640 --> 00:36:49.919 still struggling with pure lpeg, and I 00:36:49.920 --> 00:36:52.359 couldn't do much with it, so I tried to 00:36:52.360 --> 00:36:55.519 learn Re.lua instead, and I wrote this old 00:36:55.520 --> 00:36:56.319 class here... 00:36:56.320 --> 00:37:01.039 that allowed me to use a preprocessor 00:37:01.040 --> 00:37:03.279 on patterns for Lua. And the thing is that 00:37:03.280 --> 00:37:04.879 with this preprocessor I could 00:37:04.880 --> 00:37:07.839 specify precedence grammars using this 00:37:07.840 --> 00:37:11.879 thing here, that worked, but was super 00:37:11.880 --> 00:37:15.999 clumsy... and I gave up after a few attempts. 00:37:16.000 --> 00:37:21.879 and in 2022 I heard about something 00:37:21.880 --> 00:37:23.239 called lpegrex, 00:37:23.240 --> 00:37:29.799 that was a... a kind of extension or Re, 00:37:29.800 --> 00:37:32.879 and it was much more powerful than re.lua, 00:37:32.880 --> 00:37:34.919 but after a while I realized that it 00:37:34.920 --> 00:37:37.639 had the same defects as re.lua... 00:37:37.640 --> 00:37:40.839 and let me explain that, because 00:37:40.840 --> 00:37:44.439 it has all to do with the things about 00:37:44.440 --> 00:37:48.039 black boxes and magic that I told in the 00:37:48.040 --> 00:37:52.919 beginning. Both... I mean, sorry, neither 00:37:52.920 --> 00:37:57.199 re.lua or lpegrex had some features that 00:37:57.200 --> 00:38:00.799 I needed... they didn't let us explore... 00:38:00.800 --> 00:38:03.679 sorry, they received a pattern that was 00:38:03.680 --> 00:38:06.839 specified as a string, and it converted 00:38:06.840 --> 00:38:09.679 that into an lpeg pattern, but it didn't 00:38:09.680 --> 00:38:12.559 let us explore the the lpeg patterns 00:38:12.560 --> 00:38:15.159 that it generated... 00:38:15.160 --> 00:38:18.759 their code was written in a way 00:38:18.760 --> 00:38:21.319 that was REPL-unfriendly - I 00:38:21.320 --> 00:38:24.279 couldn't modify parts of the code 00:38:24.280 --> 00:38:28.399 bit by bit in a REPL and try to change 00:38:28.400 --> 00:38:31.719 the code without changing the 00:38:31.720 --> 00:38:34.199 original file... the code was very 00:38:34.200 --> 00:38:36.839 hard to explore, to hack, and to extend - 00:38:36.840 --> 00:38:39.159 in my opinion... the documentation was not 00:38:39.160 --> 00:38:43.319 very clear... and I sent one or two messages 00:38:43.320 --> 00:38:47.159 to the the developer of lpegrex and... 00:38:47.160 --> 00:38:50.759 he was too busy to help me. He 00:38:50.760 --> 00:38:53.959 answered it very briefly, and, uh, to be 00:38:53.960 --> 00:38:56.599 honest I felt... rejected. I felt that I 00:38:56.600 --> 00:38:58.679 wasn't doing anything interesting... 00:38:58.680 --> 00:39:03.399 whatever, whatever... 00:39:03.400 --> 00:39:09.239 So, in 2022 I was trying to learn lpegrex 00:39:09.240 --> 00:39:11.559 because I was thinking that it would 00:39:11.560 --> 00:39:13.719 solve my problems - but it didn't... 00:39:13.720 --> 00:39:16.479 it didn't have the features that I needed, 00:39:16.480 --> 00:39:20.919 it was hard to extend, hard to explore, 00:39:20.920 --> 00:39:23.279 and hard to debug, and I 00:39:23.280 --> 00:39:25.039 decided to rewrite it in a more 00:39:25.040 --> 00:39:30.639 hacker-friendly way - in the sense that... 00:39:30.640 --> 00:39:33.759 was modular, and I could replace any 00:39:33.760 --> 00:39:35.399 part of the module from a REPL... NOTE ELpeg1.lua 00:39:35.400 --> 00:39:43.679 My version of it was called ELpeg1.lua... 00:39:43.680 --> 00:39:47.679 and I decided that in my version I 00:39:47.680 --> 00:39:49.639 wouldn't have the part that 00:39:49.640 --> 00:39:54.879 receives a grammar specified as a string 00:39:54.880 --> 00:39:57.519 and converts that to lpeg... I would 00:39:57.520 --> 00:40:00.959 just have the backend part, that are the 00:40:00.960 --> 00:40:03.999 functions in lpeg that let us specify 00:40:04.000 --> 00:40:05.479 powerful grammars. 00:40:05.480 --> 00:40:11.759 Let me go back. Let me explain a 00:40:11.760 --> 00:40:15.519 bit about lpeg... Lua has 00:40:15.520 --> 00:40:21.599 coercions: the + expects to receive 00:40:21.600 --> 00:40:23.999 true numbers, and if one of its arguments, 00:40:24.000 --> 00:40:26.999 or both of them, are strings, it converts 00:40:27.000 --> 00:40:29.839 the string... the strings to numbers so in 00:40:29.840 --> 00:40:33.519 this case here, 2+"3", 00:40:33.520 --> 00:40:36.159 it returns the number 5, 00:40:36.160 --> 00:40:39.359 and this is the concatenation 00:40:39.360 --> 00:40:42.119 operator... it expects to receive 00:40:42.120 --> 00:40:44.999 strings, so in this case it will 00:40:45.000 --> 00:40:47.359 convert the number 2 to the string "2", 00:40:47.360 --> 00:40:50.279 and the concatenation of thes two 00:40:50.280 --> 00:40:54.479 things will be 23... oops, sorry, "23" 00:40:54.480 --> 00:40:56.279 as a string. 00:40:56.280 --> 00:40:58.519 Lpeg also has some coercions. 00:40:58.520 --> 00:41:01.759 I usually set these 00:41:01.760 --> 00:41:05.799 globals to let me write my grammars 00:41:05.800 --> 00:41:09.719 in a very compact way, so instead 00:41:09.720 --> 00:41:14.759 of lpeg.B, lpeg.C, etc I use these globals, 00:41:14.760 --> 00:41:18.359 like uppercase B, uppercase C, and so on... 00:41:18.360 --> 00:41:21.679 and with these globals I can write 00:41:21.680 --> 00:41:26.759 things like this: C(1)*"_"... 00:41:26.760 --> 00:41:33.199 and lpeg knows that lpeg.C... 00:41:33.200 --> 00:41:38.879 it sort of expands this to lpeg.C, 00:41:38.880 --> 00:41:42.039 but lpeg.C expects to receive 00:41:42.040 --> 00:41:44.839 an lpeg pattern, and 1 is not yet an 00:41:44.840 --> 00:41:47.879 lpeg pattern, so it is coerced into an 00:41:47.880 --> 00:41:51.799 lpeg pattern by calling lpeg.P, 00:41:51.800 --> 00:41:55.679 so this short thing here becomes 00:41:55.680 --> 00:42:03.399 equivalent to lpeg.C(lpeg.P(1)), and the 00:42:03.400 --> 00:42:07.399 multiplication, when at least one of its 00:42:07.400 --> 00:42:10.759 arguments is an lpeg pattern... it expects 00:42:10.760 --> 00:42:13.199 to receive two lpeg patterns, and in 00:42:13.200 --> 00:42:15.239 this case the one at the right is 00:42:15.240 --> 00:42:18.319 just a string, so it is coerced to an lpeg 00:42:18.320 --> 00:42:20.079 pattern by using lpeg.P. 00:42:20.080 --> 00:42:25.599 With this idea we can sort of 00:42:25.600 --> 00:42:28.439 understand the comparison here. I mean, 00:42:28.440 --> 00:42:31.719 let me run it again... this first part is 00:42:31.720 --> 00:42:34.679 very similar to a regular expression 00:42:34.680 --> 00:42:35.359 here at the left... 00:42:35.360 --> 00:42:39.759 and when we apply this... Lua pattern 00:42:39.760 --> 00:42:43.639 to this subject here the result 00:42:43.640 --> 00:42:47.799 is this thing here, this thing, this 00:42:47.800 --> 00:42:54.319 thing and this thing... I'm going to 00:42:54.320 --> 00:42:56.119 call each one of these results 00:42:56.120 --> 00:42:59.519 "captures", so each of these things 00:42:59.520 --> 00:43:03.319 between parentheses "captures" a substring 00:43:03.320 --> 00:43:06.039 of the original string and these 00:43:06.040 --> 00:43:08.559 captured substrings are returned in a 00:43:08.560 --> 00:43:11.839 certain order. Here is how to express the 00:43:11.840 --> 00:43:12.759 same thing in lpeg... 00:43:12.760 --> 00:43:15.919 it's very cryptic but it's a 00:43:15.920 --> 00:43:20.719 good way to understand the some basic 00:43:20.720 --> 00:43:23.879 operators of lpeg, I mean we can look at 00:43:23.880 --> 00:43:26.479 the manual and understand and 00:43:26.480 --> 00:43:30.519 what C, S and R do, and also 00:43:30.520 --> 00:43:37.959 exponentiation... and this strange thing 00:43:37.960 --> 00:43:41.319 here receives this string here, runs 00:43:41.320 --> 00:43:43.279 a function that I have defined, that 00:43:43.280 --> 00:43:46.039 converts it to an object of a certain 00:43:46.040 --> 00:43:47.759 class, and that class 00:43:47.760 --> 00:43:52.399 represents Re patterns, so this thing 00:43:52.400 --> 00:43:54.479 is treated as a pattern for re.lua, 00:43:54.480 --> 00:43:56.479 and it is matched against the string, 00:43:56.480 --> 00:43:59.439 and it returns the same thing as the 00:43:59.440 --> 00:44:02.559 other one. 00:44:02.560 --> 00:44:05.519 Also, this thing here also has a 00:44:05.520 --> 00:44:08.479 comparison with lpegrex, but these 00:44:08.480 --> 00:44:11.559 patterns are very trivial, they 00:44:11.560 --> 00:44:13.359 don't do anything very strange... 00:44:13.360 --> 00:44:15.759 so let's go back and see what 00:44:15.760 --> 00:44:18.239 kinds of very strange things there are. 00:44:18.240 --> 00:44:26.559 Here is the page of lpegrex at github, 00:44:26.560 --> 00:44:29.719 here's the documentation... 00:44:29.720 --> 00:44:32.439 it's relatively brief, 00:44:32.440 --> 00:44:35.239 it explains lpegrex as being an 00:44:35.240 --> 00:44:39.719 extension of Re.lua, so it explains 00:44:39.720 --> 00:44:42.879 mainly the additional features... here is a 00:44:42.880 --> 00:44:45.119 quick reference that explains only the 00:44:45.120 --> 00:44:46.359 additional features... 00:44:46.360 --> 00:44:49.639 some of the these things 00:44:49.640 --> 00:44:50.919 I was able to understand 00:44:50.920 --> 00:44:57.559 by struggling a lot, and some I wasn't 00:44:57.560 --> 00:45:02.439 able to even by spending several evenings 00:45:02.440 --> 00:45:04.319 try to to build examples... 00:45:04.320 --> 00:45:12.879 and this is something very nice. Lpegrex 00:45:12.880 --> 00:45:15.879 comes with some example parsers... and 00:45:15.880 --> 00:45:18.679 here is a parser that parses the Lua 00:45:18.680 --> 00:45:22.479 grammar - I mean, this is the the grammar 00:45:22.480 --> 00:45:25.959 for Lua 5.4 at the end of the 00:45:25.960 --> 00:45:31.199 reference manual... it's just this... this 00:45:31.200 --> 00:45:34.799 is in a kind of BNF, and this is the BNF 00:45:34.800 --> 00:45:35.599 translated 00:45:35.600 --> 00:45:39.919 to the language of lpegrex, so this 00:45:39.920 --> 00:45:43.039 thing uses many constructions that are 00:45:43.040 --> 00:45:47.999 in re.lua and some extra constructions that 00:45:48.000 --> 00:45:50.959 are described here... and with these 00:45:50.960 --> 00:45:54.239 examples I was able to to understand 00:45:54.240 --> 00:45:55.159 some of the... 00:45:55.160 --> 00:45:58.079 of these things here that are 00:45:58.080 --> 00:46:00.239 described here in the quick 00:46:00.240 --> 00:46:04.719 reference - but not all. 00:46:04.720 --> 00:46:11.279 So, I wasn't able to use lpegrex 00:46:11.280 --> 00:46:14.279 by itself, because some things didn't 00:46:14.280 --> 00:46:16.199 make much sense, and I decided to 00:46:16.200 --> 00:46:18.759 reimplement it in my own style, 00:46:18.760 --> 00:46:23.679 because that would be a way to map... 00:46:23.680 --> 00:46:26.839 to at the very least map what I had 00:46:26.840 --> 00:46:29.559 understood and what I didn't, learn 00:46:29.560 --> 00:46:32.999 one feature at a time, do comparisons, and 00:46:33.000 --> 00:46:35.319 so on. 00:46:35.320 --> 00:46:38.799 Here I pointed to two features of lpeg... 00:46:38.800 --> 00:46:41.679 in one I said "Oh, great! This thing can 00:46:41.680 --> 00:46:44.319 be used to to define grammars, even 00:46:44.320 --> 00:46:45.959 recursive grammars", and so on... 00:46:45.960 --> 00:46:49.759 and this is an "Oh, no!" feature - one 00:46:49.760 --> 00:46:51.759 thing that didn't make any sense at all... 00:46:51.760 --> 00:46:56.439 group captures. One thing that I did to 00:46:56.440 --> 00:46:59.039 understand group captures was to 00:46:59.040 --> 00:47:02.319 represent them as diagrams. Of course in 00:47:02.320 --> 00:47:05.359 the beginning I was drawing these 00:47:05.360 --> 00:47:08.919 diagrams by hand, but then I realized 00:47:08.920 --> 00:47:11.559 that I could use the bits of lpeg 00:47:11.560 --> 00:47:14.759 that I already knew to build a grammar 00:47:14.760 --> 00:47:17.479 that would parse a little language and 00:47:17.480 --> 00:47:20.999 generate these diagrams in LaTeX, and I was 00:47:21.000 --> 00:47:21.919 able to make this. 00:47:21.920 --> 00:47:25.279 In this diagram here 00:47:25.280 --> 00:47:30.719 this thing above the arrow is Lua code... 00:47:30.720 --> 00:47:33.759 a piece of Lua code that 00:47:33.760 --> 00:47:37.119 specifies an lpeg pattern... this 00:47:37.120 --> 00:47:39.559 thing here at the top is the string that 00:47:39.560 --> 00:47:43.039 is being matched, and the things below 00:47:43.040 --> 00:47:46.599 the underbraces are the captures that 00:47:46.600 --> 00:47:50.639 each thing... sorry, that each thing 00:47:50.640 --> 00:47:51.319 captures. 00:47:51.320 --> 00:47:58.479 For example, this underbrace here 00:47:58.480 --> 00:48:00.279 corresponds to this pattern here, 00:48:00.280 --> 00:48:02.879 that parses a single character but 00:48:02.880 --> 00:48:05.559 doesn't return any captures, this thing 00:48:05.560 --> 00:48:08.119 here parses a single "b" and doesn't 00:48:08.120 --> 00:48:11.239 return any captures, this thing here 00:48:11.240 --> 00:48:14.399 parses a single character and captures 00:48:14.400 --> 00:48:16.879 it, and this thing here parses the 00:48:16.880 --> 00:48:21.319 character "d" and captures it... and this 00:48:21.320 --> 00:48:24.439 other thing here transforms this 00:48:24.440 --> 00:48:27.279 pattern into another pattern... 00:48:27.280 --> 00:48:33.119 returns first a capture with all 00:48:33.120 --> 00:48:35.079 the string that was parsed by this 00:48:35.080 --> 00:48:37.399 pattern here, and then all the captures 00:48:37.400 --> 00:48:41.079 returned by this thing here before 00:48:41.080 --> 00:48:42.959 the ":". 00:48:42.960 --> 00:48:45.479 So, this was a way to build 00:48:45.480 --> 00:48:48.599 concrete examples for things that the 00:48:48.600 --> 00:48:52.159 lpag manual was explaining in a very terse 00:48:52.160 --> 00:48:55.799 way, and it worked for me - some things 00:48:55.800 --> 00:48:56.999 that were very 00:48:57.000 --> 00:48:59.839 mysterious started to make sense, and I 00:48:59.840 --> 00:49:03.199 started to have intelligent questions 00:49:03.200 --> 00:49:06.079 to ask in the mailing list. 00:49:06.080 --> 00:49:10.959 And with that I was able to 00:49:10.960 --> 00:49:12.959 understand what are group captures, 00:49:12.960 --> 00:49:17.879 and group captures that receive a name... 00:49:17.880 --> 00:49:22.719 Well, let me explain what this does. 00:49:22.720 --> 00:49:27.119 This thing here captures... sorry, parses 00:49:27.120 --> 00:49:29.359 the empty string and returns this as a 00:49:29.360 --> 00:49:32.959 constant... so, this is something that 00:49:32.960 --> 00:49:35.799 doesn't exist in regular expressions... 00:49:35.800 --> 00:49:38.639 it parses nothing and 00:49:38.640 --> 00:49:41.839 returns this as a capture... then this 00:49:41.840 --> 00:49:44.599 thing here returns these two 00:49:44.600 --> 00:49:47.159 constants here, and parses the empty 00:49:47.160 --> 00:49:51.279 string, and this thing here converts 00:49:51.280 --> 00:49:54.159 the results of this thing here into a 00:49:54.160 --> 00:49:57.639 group capture, and stores it in the label 00:49:57.640 --> 00:50:03.279 "d"... and then here's another constant 00:50:03.280 --> 00:50:03.719 capture. NOTE Building lists 00:50:03.720 --> 00:50:05.679 And I realized that these things 00:50:05.680 --> 00:50:08.599 here were similar to how Lua 00:50:08.600 --> 00:50:09.839 specifies building lists... 00:50:09.840 --> 00:50:16.239 when we build... sorry, tables. When 00:50:16.240 --> 00:50:18.759 we build a table, and we say that the 00:50:18.760 --> 00:50:21.879 first element of the table is here, this 00:50:21.880 --> 00:50:23.559 element is put at the end of the table... 00:50:23.560 --> 00:50:29.399 when after the that would say d=42... 00:50:29.400 --> 00:50:31.199 we are putting the 42 00:50:31.200 --> 00:50:34.559 in the the slot whose key is "d". 00:50:34.560 --> 00:50:38.999 This was happening with lpeg captures, 00:50:39.000 --> 00:50:43.359 but there was something very strange... 00:50:43.360 --> 00:50:46.199 these group captures could hold 00:50:46.200 --> 00:50:49.199 more than one capture - more than one 00:50:49.200 --> 00:50:51.759 value... so there was something between 00:50:51.760 --> 00:50:58.039 lists and tables. I started to use this 00:50:58.040 --> 00:51:00.479 notation to... 00:51:00.480 --> 00:51:04.959 explain in my notation what they 00:51:04.960 --> 00:51:08.159 were doing... many things started 00:51:08.160 --> 00:51:10.239 to make sense, many mysterious 00:51:10.240 --> 00:51:12.879 sentences in the manual started to 00:51:12.880 --> 00:51:14.439 make sense... but some didn't... 00:51:14.440 --> 00:51:19.679 but at least I was able to send 00:51:19.680 --> 00:51:22.319 some intelligent questions to the 00:51:22.320 --> 00:51:25.199 mailing lis,t and the author of Lua and 00:51:25.200 --> 00:51:27.359 lpeg answered some of them... 00:51:27.360 --> 00:51:31.519 he was not very happy about my 00:51:31.520 --> 00:51:34.959 questions - he... told me that those 00:51:34.960 --> 00:51:37.679 diagrams were a waste of time, the 00:51:37.680 --> 00:51:40.559 manual was perfectly clear, and so on... 00:51:40.560 --> 00:51:44.919 whatever - but I was able to... 00:51:44.920 --> 00:51:48.879 so, it was weird, but I was able to 00:51:48.880 --> 00:51:51.799 understand lots of things from his 00:51:51.800 --> 00:51:56.519 answers. This is a copy of one of 00:51:56.520 --> 00:51:58.239 my messages, then there's another one, 00:51:58.240 --> 00:52:01.239 another one, some of them had diagrams... 00:52:01.240 --> 00:52:04.359 then he complained about these diagrams, 00:52:04.360 --> 00:52:08.439 he said that these things here, that look 00:52:08.440 --> 00:52:11.119 like table constructors, "do not exist"... 00:52:11.120 --> 00:52:17.199 whatever... anyway, once I understood 00:52:17.200 --> 00:52:20.679 group captures many features 00:52:20.680 --> 00:52:23.359 were very easy to understand 00:52:23.360 --> 00:52:26.039 and I started to be able to use lpeg to 00:52:26.040 --> 00:52:28.159 to build some very interesting things... 00:52:28.160 --> 00:52:33.039 I was able to reproduce some 00:52:33.040 --> 00:52:36.359 of the features that I saw in lpegrex - 00:52:36.360 --> 00:52:41.079 remember that this... where is that? 00:52:41.080 --> 00:52:46.119 this is the syntax of Lua... here - 00:52:46.120 --> 00:52:48.959 I was able to understand 00:52:48.960 --> 00:52:52.479 how these things here were translated to 00:52:52.480 --> 00:52:55.359 lpeg code... to lpeg patterns 00:52:55.360 --> 00:52:58.239 by using group captures in a certain 00:52:58.240 --> 00:53:03.039 way... I was able to implement them 00:53:03.040 --> 00:53:04.759 in ELpeg1.lua... 00:53:04.760 --> 00:53:08.719 and after some time I was able to use 00:53:08.720 --> 00:53:12.879 ELpeg1.lua to build grammars that 00:53:12.880 --> 00:53:14.159 were able to parse 00:53:14.160 --> 00:53:18.679 arithmetical expressions with the 00:53:18.680 --> 00:53:20.959 right precedence... and here's an example 00:53:20.960 --> 00:53:23.319 in which I built the grammar step by step... 00:53:23.320 --> 00:53:29.239 and I test the current grammar, and I 00:53:29.240 --> 00:53:35.079 replace a bit, and then I test the new 00:53:35.080 --> 00:53:36.599 grammar and so on... 00:53:36.600 --> 00:53:39.079 and you can see that the result is 00:53:39.080 --> 00:53:43.359 always a tree that is drawn in a 00:53:43.360 --> 00:53:44.239 nice two dimensional way... 00:53:44.240 --> 00:53:48.919 At this point these powers here 00:53:48.920 --> 00:53:50.559 are returned as a list, 00:53:50.560 --> 00:53:53.119 as an operation "pow" 00:53:53.120 --> 00:53:57.559 with several arguments, here... and then 00:53:57.560 --> 00:54:00.519 I apply a kind of parsing combinator, 00:54:00.520 --> 00:54:03.719 here... that transforms these trees into 00:54:03.720 --> 00:54:08.199 other trees and with these combinators 00:54:08.200 --> 00:54:12.199 here I can specify that the "^" is 00:54:12.200 --> 00:54:14.639 associative in a certain direction... 00:54:14.640 --> 00:54:17.519 that the "/" is associative in 00:54:17.520 --> 00:54:20.119 another direction... the "-" uses 00:54:20.120 --> 00:54:23.079 the same direction as a the "/", 00:54:23.080 --> 00:54:26.079 and so on... and they have the 00:54:26.080 --> 00:54:29.679 right precedences. 00:54:29.680 --> 00:54:34.559 So, here are the tests... 00:54:34.560 --> 00:54:38.119 here is my file ELpeg1.lua... it has 00:54:38.120 --> 00:54:41.719 several classes, each class has tests 00:54:41.720 --> 00:54:42.279 after it... 00:54:42.280 --> 00:54:46.239 I was able to implement something 00:54:46.240 --> 00:54:50.519 that lpegrex has, that is called 00:54:50.520 --> 00:54:53.519 "keywords", that is very useful for parsing 00:54:53.520 --> 00:54:56.479 programs in programming languages... 00:54:56.480 --> 00:54:59.439 I was able to implement something 00:54:59.440 --> 00:55:02.639 similar to the debugger... to the 00:55:02.640 --> 00:55:07.999 lpeg debugger lpeg uses... I was 00:55:08.000 --> 00:55:11.399 frustrated by some limitations of 00:55:11.400 --> 00:55:16.839 the lpeg debugger, and I implemented 00:55:16.840 --> 00:55:23.439 my own that is, uh... much better!... 00:55:23.440 --> 00:55:24.759 Let me show something else... I was 00:55:24.760 --> 00:55:27.119 able to translate a good part of the 00:55:27.120 --> 00:55:33.039 Lua parser, here, to ELpeg1.lua... I haven't 00:55:33.040 --> 00:55:38.399 finished yet, but I have most of the 00:55:38.400 --> 00:55:39.719 the translation here... 00:55:39.720 --> 00:55:47.279 and after having all that I was able to 00:55:47.280 --> 00:55:50.319 build other grammars very quickly... 00:55:50.320 --> 00:55:55.239 writing new parsers finally became fun. 00:55:55.240 --> 00:55:58.719 And here's one example that I showed in the 00:55:58.720 --> 00:56:00.639 beginning. 00:56:00.640 --> 00:56:05.799 If I remember correctly... 00:56:05.800 --> 00:56:10.639 I took a figure from the Wikipedia... 00:56:10.640 --> 00:56:12.439 I don't have its link now... 00:56:12.440 --> 00:56:17.079 but I specified a grammar that parses 00:56:17.080 --> 00:56:20.119 exactly the example that appears 00:56:20.120 --> 00:56:20.839 in the Wikipedia... 00:56:20.840 --> 00:56:24.679 so, with my grammar, considering that 00:56:24.680 --> 00:56:28.719 the top level entry is "Stmt", when I 00:56:28.720 --> 00:56:30.679 parse this string here 00:56:30.680 --> 00:56:36.599 the result is this tree... 00:56:36.600 --> 00:56:41.119 and I can do some operations on that, 00:56:41.120 --> 00:56:44.039 I can define how this thing is to be 00:56:44.040 --> 00:56:45.639 converted into LaTeX, 00:56:45.640 --> 00:56:49.399 I can define other operations 00:56:49.400 --> 00:56:52.999 that convert trees into other trees, and 00:56:53.000 --> 00:56:54.879 here are some tests of these operations... 00:56:54.880 --> 00:57:00.359 This is what I showed in the beginning... 00:57:00.360 --> 00:57:02.759 I'm not going to explain all the details 00:57:02.760 --> 00:57:03.999 of this thing now... 00:57:04.000 --> 00:57:09.199 this :show() converts this thing 00:57:09.200 --> 00:57:11.919 into LaTeX in the way specified by these 00:57:11.920 --> 00:57:16.159 instructions here, that says that... 00:57:16.160 --> 00:57:25.239 well, whatever... 00:57:25.240 --> 00:57:32.959 and here's the result - the LaTeXed result... 00:57:32.960 --> 00:57:41.759 and these diagrams here are generated by 00:57:41.760 --> 00:57:46.719 this file here, that defines a simple 00:57:46.720 --> 00:57:48.479 grammar that parses this thing here, 00:57:48.480 --> 00:57:51.999 and then LaTeXes it in a certain way, and 00:57:52.000 --> 00:57:56.399 and also tests to check if this code here... 00:57:56.400 --> 00:58:01.999 this Lua code that generates an lpeg grammar... 00:58:02.000 --> 00:58:05.799 parses this subject here and 00:58:05.800 --> 00:58:08.599 returns the expected result... 00:58:08.600 --> 00:58:12.239 So: this is the code that I 00:58:12.240 --> 00:58:16.719 wanted to show. I wanted to show many 00:58:16.720 --> 00:58:19.919 more things but I wasn't able to prepare 00:58:19.920 --> 00:58:23.919 them before the conference... and I hope 00:58:23.920 --> 00:58:27.519 that soon - for some value of "soon" - 00:58:27.520 --> 00:58:30.399 I'll be able to create REPL-based 00:58:30.400 --> 00:58:33.919 tutorials for lpeg, Re, and ELpeg1.lua... 00:58:33.920 --> 00:58:36.319 where lpeg is something very famous, 00:58:36.320 --> 00:58:39.199 Re is a module of lpeg... 00:58:39.200 --> 00:58:42.399 I could also do something like this 00:58:42.400 --> 00:58:47.799 for lpegrex... and ELpeg1.lua is 00:58:47.800 --> 00:58:51.159 the thing that I wrote, the one that 00:58:51.160 --> 00:58:56.799 has test in comments, and the tests 00:58:56.800 --> 00:58:59.519 usually generate trees, and sometimes 00:58:59.520 --> 00:59:00.879 they generate TeX code. 00:59:00.880 --> 00:59:04.959 Yeah, so that's it! I wanted to 00:59:04.960 --> 00:59:07.159 present much more but I wasn't able to 00:59:07.160 --> 00:59:10.480 prepare it... so: sorry, thanks, bye! =)