WEBVTT
00:00.000 --> 00:04.880
Hi, my name is Eduard Duax, and the title of this talk is
00:04.880 --> 00:09.680
Rapples in Strange Places, Lua, LaTeX, LPEG, LPEG-REX, and TIX.
00:10.720 --> 00:17.200
I'm the author of an MX package called EEV, and this is a talk at the MXConf 2023
00:17.200 --> 00:20.880
that is happening in December 2023 at the Internets.
00:21.680 --> 00:25.920
And this is one of the examples of diagrams that we are going to see.
00:25.920 --> 00:27.680
Let me show how I generate it.
00:28.640 --> 00:32.240
One second. I have to use a smaller font here.
00:35.600 --> 00:41.920
This is a file called parse32.lua. Let me go back to this block of tests again.
00:42.720 --> 00:50.320
And now if I run this, we get these outputs here at the right.
00:52.000 --> 00:54.960
And then in this line here, it generates a PDF.
00:55.520 --> 01:01.440
And if I type f8 here, it shows the PDF in the lower right window.
01:04.240 --> 01:07.600
Let me start by explaining briefly what is EEV.
01:09.920 --> 01:14.560
First, it's something that appeared by accident in the mid-90s.
01:14.560 --> 01:19.920
I explained this story in my presentation at the MXConf 2019.
01:20.720 --> 01:25.520
It's a package. It's an MX package that is part of ELPA.
01:25.520 --> 01:31.440
It has at least 10 users. Those are the ones that I know by name.
01:33.360 --> 01:36.320
EEV means MX Execute Verbosely.
01:37.360 --> 01:42.640
EEV is something that treats eval as the central feature of MX.
01:43.680 --> 01:46.800
EEV blurs the distinction between programmers and users,
01:47.520 --> 01:50.080
and it replaces the slogan,
01:50.080 --> 01:55.360
users should not be forced to see Lisp, that is something that Richard Stallman told me once,
01:55.360 --> 01:59.280
by users should see Lisp instead of buttons,
01:59.280 --> 02:02.400
and new users should see Lisp in the first five minutes.
02:04.240 --> 02:07.120
I'm going to show some examples of that soon.
02:08.800 --> 02:12.880
EEV uses code in comments a lot, and also tests in comments.
02:13.600 --> 02:21.040
I changed my way of presenting it, and it became very REPL-centric in the last few years,
02:21.040 --> 02:28.240
in the sense that I start by explaining its main features by its support for REPLs.
02:30.320 --> 02:34.320
EEV supposes that we want to keep executable notes of everything.
02:34.320 --> 02:37.040
I'm also going to show examples of this in a second.
02:38.080 --> 02:41.600
EEV has lots of videos for people who hate videos,
02:42.240 --> 02:47.520
and it tries to do everything with very little magic and without black boxes.
02:47.520 --> 02:50.240
I'm going to explain many of these things very soon.
02:52.880 --> 02:58.160
This is a figure that I'm going to show in detail soon,
02:58.160 --> 03:01.520
that is about something important about Lua.
03:01.520 --> 03:05.280
Oops, the font is very bad now, so let me change the font.
03:05.280 --> 03:07.840
The figure is this one, and
03:08.560 --> 03:13.440
what most people do when they visit a file with something interesting on it
03:13.440 --> 03:16.640
is that they just go there, and they set a bookmark there,
03:16.640 --> 03:19.760
or they put the position in the register.
03:21.760 --> 03:28.480
But I prefer to keep links to everything that is interesting as a list of hyperlinks.
03:28.480 --> 03:32.640
So, for example, this is an at least hyperlink to the file
03:32.640 --> 03:36.480
that goes to this anchor here, and to this string I've added.
03:36.480 --> 03:38.560
And to this string after this anchor.
03:39.760 --> 03:45.760
This is a variant that opens that file in the window at the right here.
03:47.440 --> 03:53.520
And this is a sexpid that changes the font.
03:53.520 --> 03:57.680
I have a command with a very short name that does that,
03:58.400 --> 04:02.880
but I prefer to keep that as a one-liner.
04:03.840 --> 04:11.440
About the videos, we can see the list of first-class videos of Eevee by executing this,
04:11.440 --> 04:16.960
meta-x, find first-class videos, or by running this alias here,
04:16.960 --> 04:20.320
meta-x1c, and then what we see is this.
04:21.520 --> 04:25.440
The first sexpid here regenerates this buffer,
04:25.440 --> 04:28.960
so we can make a mess here and then run this,
04:29.040 --> 04:33.040
and the original buffer is regenerated again in a clean way.
04:34.880 --> 04:42.000
Each of these things here opens a buffer with information about a video.
04:42.000 --> 04:44.400
Let me take a specific example here.
04:46.080 --> 04:50.720
This video here is about one of the ancestors of this talk,
04:50.720 --> 04:52.960
that is a library that I wrote for
04:53.920 --> 05:01.120
creating diagrams in LaTeX using a package called pic2e, using repls.
05:02.320 --> 05:10.720
Anyway, the thing is that if we run a sexpid like this one,
05:10.720 --> 05:13.440
and we don't have a local copy of the video,
05:14.080 --> 05:16.320
Eevee will try to load the local copy,
05:16.320 --> 05:19.600
and instead of doing that and by asking something like,
05:19.680 --> 05:20.720
asking something like,
05:20.720 --> 05:24.560
do you want me to download the local copy, blah, blah, blah, blah,
05:25.680 --> 05:28.720
it simply opens a buffer like this.
05:28.720 --> 05:32.480
I mean, if we don't have a local copy yet,
05:32.480 --> 05:34.400
it will open a buffer like this one,
05:35.440 --> 05:41.760
in which these things here in comments are links to the documentation.
05:41.760 --> 05:48.240
I mean, this thing here explains the idea of local copies of files from the internet,
05:48.880 --> 05:52.960
uh, there are more details here and here,
05:53.840 --> 05:59.280
and this is a script that we can execute line by line.
05:59.280 --> 06:04.080
So, instead of this script being hidden behind the button that we just
06:04.960 --> 06:07.360
press after a question like,
06:07.360 --> 06:09.920
do you want me to do something, blah, blah, blah, yes or no,
06:11.200 --> 06:15.120
the script is visible here, and we can execute it step by step.
06:15.200 --> 06:22.080
It creates a terminal with a shell here in the right window,
06:22.080 --> 06:28.880
and when we type f8 in these lines here, the lines are sent to this line.
06:30.080 --> 06:33.920
So, this is going to download the copy of the video,
06:33.920 --> 06:39.360
the wget says that I already have a copy of the video in its subtitles, and so on.
06:39.920 --> 06:47.600
And after getting a copy of the video, we can run this exp here, and it displays the video.
06:52.880 --> 06:57.840
I said that Eevee has lots of videos for people who hate videos,
06:57.840 --> 07:02.960
and the idea is that very few people are going to watch the videos in real time,
07:03.920 --> 07:10.480
and most of the people that I know, or most of the people that are interested in Eevee in some way,
07:10.480 --> 07:15.280
they are going to watch just small sections of the video,
07:15.280 --> 07:18.720
and most of the time they're just going to read the subtitles of the video.
07:19.680 --> 07:25.760
So, for each one of the videos, we have a page about the video.
07:25.760 --> 07:28.400
Let me see if I have internet here.
07:28.400 --> 07:29.680
Yes, this is a page,
07:29.920 --> 07:36.080
and usually these pages have a link to another page,
07:36.080 --> 07:42.880
the page that has all the subtitles of the video, wherever.
07:42.880 --> 07:46.960
In this one, it's not so visible, but anyway,
07:46.960 --> 07:50.480
there are several ways of accessing the subtitles of the video,
07:50.480 --> 07:54.720
and one of the ways is by running this exp here,
07:55.040 --> 07:57.280
is by running this exp here,
07:59.440 --> 08:03.680
that opens a file in Lua that is what I use to generate the subtitles.
08:04.880 --> 08:13.760
Anyway, by the way, each one of these things here is hyperlinked to a position of the video,
08:14.400 --> 08:19.280
so if I type this in the right way, it goes to that position.
08:20.160 --> 08:21.280
Anyway, let me go back.
08:21.920 --> 08:26.640
Also, the tutorials of Eevee, the intros of Eevee,
08:26.640 --> 08:30.560
let's start with find, and with intro,
08:30.560 --> 08:35.760
they have lots of blocks that say video links, like this one,
08:36.800 --> 08:40.080
and these blocks have links to the positions in videos,
08:40.080 --> 08:43.600
and if we don't have a local copy of the video yet,
08:44.480 --> 08:48.800
the thing shows us a script that lets us download the local copy.
08:48.880 --> 08:56.720
Anyway, I said that I was going to explain what I mean by magic and black boxes.
08:59.360 --> 09:02.080
This is something that I've been trying to explain for a long time,
09:02.080 --> 09:05.840
and I think that I got a very good explanation about that
09:05.840 --> 09:09.440
in a video that I made about something called Eevee wconfig,
09:09.440 --> 09:15.120
that is a tool for configuring Eevee on Windows without magic,
09:15.840 --> 09:20.320
on Windows without magic, without buttons that do things
09:21.040 --> 09:22.720
without explaining what they are doing.
09:24.080 --> 09:26.720
This is a part of the subtitles of the video.
09:26.720 --> 09:27.680
Let me read that.
09:30.080 --> 09:32.640
Eevee wconfig is an attempt to solve the problem
09:32.640 --> 09:35.040
of how to install these things on Windows,
09:35.040 --> 09:38.240
both without magic and with very little magic.
09:39.520 --> 09:40.640
Remember this slogan,
09:41.200 --> 09:45.760
any sufficiently advanced technology is indistinguishable from magic.
09:47.280 --> 09:51.600
Here in this video, I'm going to use the term magic as a shorthand
09:51.600 --> 09:55.520
for sufficiently advanced technology,
09:55.520 --> 09:59.040
that is something that is complex and non-obvious,
09:59.040 --> 10:01.840
and that is indistinguishable from magic
10:01.840 --> 10:04.640
in the sense of being almost impossible to understand.
10:05.600 --> 10:10.720
And I'm also going to use the term black box as a near synonym for magic,
10:11.360 --> 10:15.360
and sometimes the term black box is more convenient,
10:15.360 --> 10:18.800
even though it's a bit longer, it has more letters,
10:18.800 --> 10:20.880
because when I use the term black box,
10:20.880 --> 10:25.440
it invites us to use expressions like opening the black box,
10:25.440 --> 10:27.840
and I'm going to use that expression a lot.
10:34.640 --> 10:37.360
Now, let me try to explain what is...
10:37.360 --> 10:40.400
Sorry, let me change the font.
10:44.000 --> 10:44.800
What is Lua?
10:46.000 --> 10:51.840
Lua is a minimalistic language in the sense of batteries not included.
10:52.880 --> 10:56.640
It uses associative tables for most of its data structures,
10:58.800 --> 11:02.240
and it's so minimalistic that it's the default print function.
11:03.120 --> 11:09.280
When we create an associative table and we ask it to print...
11:11.760 --> 11:14.560
We ask print to print an associative table,
11:14.560 --> 11:17.280
it just prints the address of the table.
11:17.280 --> 11:18.480
Here are some examples.
11:19.920 --> 11:23.760
Here is a table, and when we ask print to print it,
11:23.760 --> 11:26.720
it just says that it's the table at this address here.
11:27.440 --> 11:32.400
So, one of the things that most people do when they start using Lua
11:32.400 --> 11:36.320
is that either they download a package with a print to printing function,
11:36.320 --> 11:38.720
or they write their own print to printing functions.
11:39.360 --> 11:42.000
My own print to printing function is called pp,
11:42.000 --> 11:45.840
with uppercase letters, and it works like this,
11:47.600 --> 11:50.880
and it prints associative tables in a way like this.
11:50.880 --> 11:54.800
It says that for the key 1,
11:55.200 --> 11:57.280
the value associated to it is 2,
11:57.280 --> 11:59.200
for the key 2, the value is 3,
11:59.200 --> 12:01.760
and for the key 3, the value is 5.
12:08.960 --> 12:12.160
When I started using Lua, one of my favorite languages
12:12.160 --> 12:15.600
was also a language that used associative tables a lot.
12:15.600 --> 12:16.720
It was called Icon,
12:18.400 --> 12:22.640
and I had to write my own print to printing functions for Icon.
12:22.720 --> 12:28.800
So, I just had to port my print to printing functions to Lua,
12:28.800 --> 12:31.600
and my first version looked something like this.
12:32.240 --> 12:36.560
It just had some global functions, lots of them actually,
12:39.680 --> 12:41.840
and after a while I rewrote it,
12:41.840 --> 12:44.320
and I rewrote it again, and again, and again,
12:44.320 --> 12:49.200
and this is one of the versions that is not even the default at this point.
12:50.000 --> 12:53.920
TOS is for toString,
12:56.800 --> 12:57.920
and this is a demo.
12:58.800 --> 13:04.560
It's very modular, so it's easy to replace parts of it or to toggle flags,
13:04.560 --> 13:05.760
and this is an example.
13:05.760 --> 13:10.320
If I try to print the table of methods for a certain class,
13:11.760 --> 13:13.040
I'll need a smaller font,
13:14.320 --> 13:16.480
it prints the table like this,
13:16.480 --> 13:18.480
with the names of the methods,
13:18.560 --> 13:21.520
and then links to the source code of the functions.
13:22.080 --> 13:25.360
These links only make sense in Emacs and in Eevee,
13:26.000 --> 13:28.560
and when we run a link like this one,
13:29.840 --> 13:33.920
it shows the source code in the window at the right.
13:33.920 --> 13:37.760
So, for some functions, the source code is three lines,
13:37.760 --> 13:40.400
for other ones it's one line, and whatever.
13:41.920 --> 13:43.200
Anyway, let me go back.
13:45.360 --> 13:48.000
Lua can be used in many different styles.
13:48.000 --> 13:50.160
Most people hate other people's styles.
13:51.920 --> 13:56.080
When I started using it in the year 2000,
13:56.080 --> 13:59.040
I learned most of the basic language in a single day.
13:59.040 --> 14:02.000
It was very similar to things that I was already using,
14:02.800 --> 14:08.560
and then I rewrote the mini language that I was using
14:08.560 --> 14:14.800
to generate the HTML for my pages in Lua.
14:14.800 --> 14:16.960
Actually, I had to rewrite it many times,
14:16.960 --> 14:20.880
but the first version I certainly did in my first weeks
14:20.880 --> 14:22.480
or first months using Lua.
14:25.520 --> 14:29.040
In the beginning, I was just using it for writing programs
14:29.040 --> 14:32.160
that either didn't take in any input at all,
14:32.160 --> 14:35.280
because the input was already in the source file,
14:35.280 --> 14:39.200
or that worked as a Unix function,
14:39.200 --> 14:42.960
the Unix programs that would read files
14:42.960 --> 14:46.160
and process these files in some way and output something.
14:47.840 --> 14:52.480
I mentioned the basic language here.
14:52.480 --> 14:55.520
I only learned how to use closures, metatables,
14:55.520 --> 14:58.240
and coroutines many years later.
15:00.320 --> 15:02.240
In the beginning, when I started using Lua,
15:02.240 --> 15:04.080
it didn't have a package manager.
15:04.080 --> 15:05.200
It appeared later.
15:05.200 --> 15:06.560
It is called Lua-rocks.
15:07.680 --> 15:11.600
It has had this package manager for several years.
15:12.720 --> 15:15.520
Most of the rocks for Lua-rocks are poorly documented,
15:16.480 --> 15:19.040
documented, and hacker-unfriendly,
15:19.040 --> 15:21.600
so you can't rely just on the documentation,
15:22.160 --> 15:24.320
and you can't rely just on the source code,
15:24.320 --> 15:28.320
because, I mean, if you're a genius, of course you can,
15:28.320 --> 15:32.320
but for people who are either lazy or dumb or whatever,
15:32.320 --> 15:33.920
like me, or unfocused,
15:35.120 --> 15:37.120
the source code is hard to understand
15:37.120 --> 15:38.800
and hard to tinker with.
15:41.440 --> 15:43.040
Some rocks are excellent.
15:43.040 --> 15:46.640
The best rocks are well documented,
15:46.640 --> 15:48.640
but they are hacker-unfriendly,
15:48.640 --> 15:51.840
in the sense that I hope that I'll be able to explain soon.
15:53.440 --> 15:58.560
The best rocks use local variables and metatables a lot,
15:59.280 --> 16:03.120
so if you are a beginner learning Lua,
16:03.120 --> 16:06.080
you're not going to understand what their source codes do.
16:06.080 --> 16:08.320
They use lots of dirty tricks.
16:08.640 --> 16:12.240
Let me talk a bit about object orientation in Lua.
16:12.240 --> 16:13.840
It can be done in many ways.
16:15.360 --> 16:18.000
The main book about Lua, called Programming in Lua,
16:18.000 --> 16:20.240
by one of the authors of the language,
16:21.120 --> 16:22.320
Roberto Ierzalimschi,
16:23.200 --> 16:26.960
presents several ways of doing object orientation in Lua.
16:26.960 --> 16:29.040
I hated all of these ways,
16:29.840 --> 16:32.720
and also the ways that I tried from the rocks.
16:34.400 --> 16:36.880
And then I wrote my own way,
16:37.360 --> 16:40.160
wrote my own way of doing object orientation in Lua.
16:40.160 --> 16:41.440
It's very minimalistic.
16:42.000 --> 16:45.120
It's in this file here, eoo.lua.
16:47.040 --> 16:49.440
The main code is just these five lines here,
16:50.640 --> 16:53.120
and here's an example of how it works.
16:56.080 --> 17:02.640
Here we define a class vector with some metamethods.
17:02.640 --> 17:07.920
This metamethod here will tell Lua what to do
17:07.920 --> 17:11.040
when the user asks to add two vectors.
17:11.600 --> 17:14.320
This one here tells Lua what to do
17:14.320 --> 17:18.320
when the user asks Lua to convert a vector to a string,
17:18.880 --> 17:20.640
and whatever.
17:20.640 --> 17:23.920
This one is something that I'm going to explain in a second.
17:25.360 --> 17:28.240
So here we create a vector with these coordinates,
17:28.240 --> 17:29.360
three and four.
17:29.360 --> 17:31.360
Here we create another vector.
17:31.360 --> 17:35.200
If we print here, then Lua uses the function here
17:35.200 --> 17:36.160
in the toString.
17:37.200 --> 17:40.880
If we add two vectors, it uses the function here
17:40.880 --> 17:42.400
in the add metamethod.
17:42.960 --> 17:45.280
And if we run the method norm,
17:45.280 --> 17:49.760
it is defined here in the table index.
17:51.280 --> 17:51.780
Anyway.
17:52.740 --> 17:54.740
Even this thing being so small,
17:54.740 --> 17:58.500
I used to forget how it's in odds worked all the time.
17:58.500 --> 18:01.700
Actually, I always forget how things work,
18:01.700 --> 18:03.940
and I have to remember them somehow.
18:03.940 --> 18:07.940
And I have to have tricks for remembering
18:07.940 --> 18:11.940
and tricks for summarizing things and diagrams and so on.
18:13.540 --> 18:15.940
And every time that I forgot how to do it,
18:15.940 --> 18:17.220
I just go back to the code.
18:17.220 --> 18:18.420
I just go back to the code.
18:18.500 --> 18:23.140
And every time that I forgot how this thing worked,
18:23.140 --> 18:25.540
I went back to the source code,
18:25.540 --> 18:27.380
and then I looked at the diagrams.
18:27.380 --> 18:29.940
Or, of course, in the first time,
18:29.940 --> 18:31.460
I had to draw the diagrams.
18:32.260 --> 18:34.420
And I run the examples.
18:34.420 --> 18:35.780
And, of course, in the beginning,
18:35.780 --> 18:37.460
I thought that the code was clear.
18:37.460 --> 18:38.900
My examples were very brief.
18:38.900 --> 18:42.820
And so I had to rewrite the examples many times
18:42.820 --> 18:46.420
until they became, let's say, perfect.
18:48.660 --> 18:53.220
And I was saying that Lua can be used in many ways.
18:53.780 --> 18:57.700
And in my way of using Lua, my favorite way,
18:57.700 --> 19:02.020
everything can be inspected and modified from REPLs,
19:03.140 --> 19:07.140
like we can do in MX and in Smalltalk, or sort of.
19:07.860 --> 19:09.940
So in my favorite way of using Lua,
19:09.940 --> 19:11.860
there's no security at all.
19:11.860 --> 19:14.740
Everything can be changed at all times.
19:15.140 --> 19:16.980
Of course, most people hate that.
19:17.780 --> 19:20.180
My init file has lots of classes.
19:20.820 --> 19:24.100
And, by the way, instead of keeping many small files
19:24.100 --> 19:25.140
with many things,
19:25.140 --> 19:29.300
I put lots of stuff in just one big init file.
19:31.060 --> 19:33.860
My init file has lots of classes
19:33.860 --> 19:36.660
and lots of global functions and lots of crafts.
19:36.660 --> 19:40.020
So people hate that, of course.
19:41.380 --> 19:42.580
This is an example.
19:43.460 --> 19:44.500
This is an example.
19:44.500 --> 19:48.260
This is the index at the top of my init file.
19:50.580 --> 19:52.820
The classes start here.
19:55.060 --> 19:57.220
And then we have some functions.
19:57.780 --> 20:02.500
And then we have functions that load certain packages.
20:02.500 --> 20:03.540
And then we have craft.
20:04.260 --> 20:04.760
Whatever.
20:06.180 --> 20:08.580
Most people think that my style of using Lua
20:08.580 --> 20:10.340
is dirty and dangerous.
20:10.340 --> 20:11.860
And they wouldn't touch my Lua code.
20:12.100 --> 20:13.140
With a 10-feet pole.
20:14.740 --> 20:18.100
But most of the things that I'm going to present here
20:18.100 --> 20:20.740
in this presentation are ideas that should be easy
20:20.740 --> 20:24.180
to port to other environments and other languages.
20:24.180 --> 20:25.780
Especially the diagrams.
20:25.780 --> 20:27.620
So the code is not so important.
20:30.740 --> 20:33.140
Now let me talk a bit about LaTeX.
20:33.140 --> 20:33.860
Lua LaTeX.
20:33.860 --> 20:38.820
That is LaTeX with the Lua interpreter embedded inside.
20:39.380 --> 20:43.300
And two ways of generating pictures in LaTeX.
20:43.300 --> 20:44.820
Text that is very famous.
20:44.820 --> 20:47.300
And picture that is not very famous.
20:47.300 --> 20:48.900
And that is very low level.
20:48.900 --> 20:51.700
And I think that not many people use.
20:53.700 --> 20:57.300
I said before that when I learned Lua,
20:57.300 --> 21:01.540
I realized that it was very good for writing little languages.
21:03.220 --> 21:05.220
I was doing my PhD at the time.
21:05.300 --> 21:11.220
And typesetting the diagrams for my PhD thesis was very boring.
21:11.220 --> 21:14.100
So one of the things that I did was that I created
21:14.980 --> 21:17.540
a little language for typesetting the diagrams for me.
21:19.140 --> 21:20.900
It was called DEDNOT.
21:20.900 --> 21:25.300
Because initially it only generated diagrams
21:25.300 --> 21:26.820
for natural deduction.
21:26.820 --> 21:28.580
And then it had several versions.
21:30.180 --> 21:34.420
These are the slides for my presentation about DEDNOT6.
21:34.420 --> 21:39.140
And DEDNOT6 is an extensible semi-preprocessor for Lua LaTeX
21:39.140 --> 21:41.780
that understands diagrams in ASCII art.
21:42.340 --> 21:48.740
In the sense that when I have a LaTeX file that has this.
21:49.380 --> 21:51.860
And when DEDNOT6 is loaded.
21:53.220 --> 21:58.820
When I give the right commands, DEDNOT6 interprets this block here
21:58.820 --> 22:00.980
as something that defines this diagram.
22:02.260 --> 22:03.540
Oops, sorry.
22:05.380 --> 22:07.540
It interprets this diagram here.
22:08.340 --> 22:10.900
This diagram in the comments here
22:11.460 --> 22:14.260
as something that defines a diagram called foo,
22:14.260 --> 22:16.100
a deduction called foo.
22:16.100 --> 22:19.140
And it generates this code here.
22:19.940 --> 22:27.300
So that we can just invoke the definition of the deduction
22:27.300 --> 22:30.260
by typing backslash DED foo.
22:31.220 --> 22:37.140
And DEDNOT6 also supports another language
22:37.140 --> 22:39.860
for typesetting bi-dimensional diagrams
22:39.860 --> 22:42.660
with arrows and stuff for category theory and blah blah.
22:44.180 --> 22:46.900
The specifications of these diagrams look like this.
22:49.540 --> 22:53.540
Here is a very good example.
22:55.540 --> 22:56.980
This is a huge diagram.
22:58.420 --> 22:59.460
Sorry, one second.
23:00.740 --> 23:03.620
So the source code that generates this diagram here
23:03.620 --> 23:05.140
is just this thing at the left.
23:07.220 --> 23:08.340
So it's very visual.
23:09.220 --> 23:12.340
We can typeset the diagram in ASCII art here.
23:12.340 --> 23:13.780
And then in this part here,
23:13.780 --> 23:16.660
we tell how the nodes are to be joined,
23:17.620 --> 23:20.660
which arrows have to have annotations and so on.
23:22.420 --> 23:25.460
And this language is extensible in the sense that...
23:25.780 --> 23:26.580
Where is that?
23:32.020 --> 23:32.520
Here.
23:34.420 --> 23:36.660
Comments that start with percent colon
23:37.940 --> 23:41.620
are interpreted as definitions for three diagrams.
23:43.620 --> 23:47.220
Lines that start with percent uppercase D
23:47.220 --> 23:52.260
define 2D diagrams with arrows and stuff.
23:52.740 --> 23:58.020
And lines that start with comment uppercase L
23:58.660 --> 24:00.580
contain blocks of Lua code
24:00.580 --> 24:04.180
that we can use to extend the interpreter on the flag.
24:06.020 --> 24:09.860
Anyway, here are some recent examples of diagrams
24:09.860 --> 24:14.580
that I used DEDNOT6 to typeset.
24:15.780 --> 24:17.220
This diagram here
24:17.780 --> 24:20.580
was generated by this specification here.
24:23.460 --> 24:26.980
And this diagram here with the curved arrows
24:27.780 --> 24:30.580
was generated by this specification here.
24:34.180 --> 24:37.220
So DEDNOT6 was very easy to extend.
24:37.220 --> 24:38.260
At that some point,
24:38.260 --> 24:42.820
I started to use it to generate diagrams using Peaked Chewy,
24:42.820 --> 24:45.540
mainly for the classes that I gave you.
24:45.540 --> 24:48.420
For the classes that I give at the university.
24:48.420 --> 24:50.900
I teach mathematics and whatever.
24:51.860 --> 24:52.900
In a bad place.
24:52.900 --> 24:53.400
Whatever.
24:56.260 --> 24:58.500
Let me show an animation.
24:59.060 --> 25:02.420
Here is a diagram that I generated with DEDNOT6.
25:02.980 --> 25:05.300
And it is a flipbook animation.
25:05.300 --> 25:08.740
Like we type page up and page down
25:08.740 --> 25:10.660
and we go to the next page of the book
25:10.660 --> 25:12.340
and to the previous page of the book.
25:13.060 --> 25:15.940
And here is the source code that generates that.
25:16.980 --> 25:19.140
This source code is not very visual.
25:19.140 --> 25:22.740
So it's quite clumsy to edit a diagram
25:22.740 --> 25:25.460
directly in the tag file like that.
25:28.020 --> 25:31.860
These diagrams were inspired by something called Manim.
25:32.500 --> 25:33.000
That's...
25:34.580 --> 25:36.340
Oh, I've forgotten the name of the guy.
25:36.340 --> 25:39.300
But it's a guy that makes many videos about mathematics.
25:39.300 --> 25:41.780
And he created this library called Manim
25:41.780 --> 25:43.460
for generating his animations.
25:45.540 --> 25:51.460
Other people adapted his library to make it more accessible.
25:52.580 --> 25:53.700
I tried to learn it.
25:53.700 --> 25:55.700
But each animation,
25:57.060 --> 25:59.220
even an animation with very few frames,
25:59.220 --> 26:01.300
each animation took ages to render.
26:01.300 --> 26:03.060
So it wasn't fun.
26:03.700 --> 26:07.060
And animations and PDFs can be rendered in seconds.
26:07.940 --> 26:11.060
So these things were fun for me
26:11.060 --> 26:15.300
because my laptop is very slow and Manim was not fun.
26:20.100 --> 26:26.020
Anyway, writing code like this inside the attack file
26:26.020 --> 26:29.620
was not very fun because it was hard to debug.
26:30.500 --> 26:35.860
So in 2022, I started to play with ways
26:35.860 --> 26:40.340
of generating these diagrams from REPLs.
26:40.980 --> 26:44.500
And I found a way for PIC2E and a way for TIX.
26:45.380 --> 26:47.780
Each one of these ways became a video.
26:48.740 --> 26:53.780
If you go to the list of first class videos of EEV,
26:53.780 --> 26:57.540
you're going to see that there's a video about PIC2E here
26:57.540 --> 26:58.740
and a video about TIX.
27:00.340 --> 27:07.540
Here you have some information like length and explanation, etc.
27:08.180 --> 27:10.420
And here are the pages for these videos.
27:12.420 --> 27:15.860
My page about the video, about PIC2E is like this.
27:15.860 --> 27:18.740
It has some diagrams, whatever.
27:18.740 --> 27:20.980
And this one is much nicer.
27:20.980 --> 27:26.100
And a lot of people watched that video.
27:26.100 --> 27:30.260
I mean, I thought that 250 people watched it.
27:30.900 --> 27:33.940
For me, that's a million of people.
27:36.660 --> 27:42.900
And this video is about how to extract diagrams from the TIX manual
27:42.900 --> 27:47.700
and how to run those examples in a REPL
27:47.700 --> 27:49.780
and modify them bit by bit.
27:49.780 --> 27:51.380
This is a screenshot.
27:52.100 --> 27:53.620
But let me go back.
27:56.420 --> 27:59.700
At that point, these things were just prototypes.
27:59.700 --> 28:01.460
The code was not very nice.
28:02.020 --> 28:10.500
And in this year, I was able to unify those two ways of generating PDFs,
28:11.380 --> 28:14.100
the one for TIX and the one for PIC2E.
28:14.100 --> 28:18.740
And I unified them with many other things that generated diagrams.
28:19.220 --> 28:27.460
The basis of these things is something called showchew.lua.
28:27.460 --> 28:31.380
I'm not going to show its details now.
28:32.100 --> 28:40.420
But its extension that generates TIX code is just this.
28:40.420 --> 28:45.540
So we can specify a diagram with just a block like this.
28:46.260 --> 28:54.180
And then if we run show00 and it returns a string
28:54.180 --> 28:58.580
that is just the inner body of the TIX file.
28:59.220 --> 29:02.020
If we run this, we see the whole TIX file.
29:02.020 --> 29:04.340
And if we run this, we save the TIX file
29:04.340 --> 29:08.420
and we compile the TIX file to generate a PDF.
29:08.420 --> 29:14.180
And if we run this, we show the PDF in the lower right window.
29:16.020 --> 29:20.740
And that's the same thing for all my recent programs that generate PDFs.
29:21.620 --> 29:22.980
They are all integrated.
29:24.500 --> 29:30.660
Here is the one that the basis for all my modules that generate diagrams with PIC2E.
29:33.060 --> 29:34.980
Its demos are not very interesting.
29:34.980 --> 29:39.700
So let me show some demos of extensions that do interesting things.
29:40.660 --> 29:47.220
So this is a diagram that I created by editing it in a REPL.
29:48.820 --> 29:51.220
I create several PIC2E objects here.
29:52.500 --> 29:58.660
And if I execute this, it compiles the object, generates a PDF.
29:58.660 --> 30:03.860
And if I tap this, here is the PDF.
30:04.740 --> 30:10.020
And if I just ask Lua to display what is books here,
30:11.700 --> 30:17.220
it shows the source code in PIC2E of the diagram.
30:17.940 --> 30:20.340
And the nice thing is that it is indented.
30:20.340 --> 30:23.460
So it's easy to debug the PIC2E code.
30:24.340 --> 30:30.180
If anyone is interested, the module that does the tricks for indentation
30:30.180 --> 30:31.460
is very easy to understand.
30:31.540 --> 30:34.500
It has lots of tests and test blocks.
30:34.500 --> 30:39.060
And I think that its data structures are easy to understand.
30:42.500 --> 30:44.820
Anyway, here is another example.
30:50.260 --> 30:51.620
The show is here.
30:53.460 --> 30:56.340
It generates a 3D diagram.
31:02.420 --> 31:08.500
Now let me talk about parsers and REPLs in a very strange place.
31:09.140 --> 31:18.340
I mean, using REPLs to build parsers step by step and replacing parts by more complex parts.
31:21.060 --> 31:23.780
So I said that Lua is very minimalistic.
31:25.780 --> 31:30.660
And everybody knows that implementations of regular expressions
31:30.740 --> 31:31.780
are big and complex.
31:33.060 --> 31:39.220
So instead of coming with full regular expressions, Lua comes with something called patterns
31:39.220 --> 31:43.620
and a library function called string.match.
31:45.380 --> 31:54.420
Here is a copy of the part of the manual that explains a part of the syntax of patterns.
31:56.260 --> 32:00.100
Here's how string.match is described in the manual.
32:01.220 --> 32:02.500
It's just this.
32:02.500 --> 32:06.260
Looks for the first match of pattern in the string as blah, blah, blah.
32:07.220 --> 32:11.300
And then we have to go to the other section of the manual that explains patterns.
32:17.780 --> 32:26.500
Lua patterns are so simple, so limited, that they don't even have the alternation operator.
32:27.220 --> 32:31.460
Here is how it is described in the ELLISP manual.
32:33.300 --> 32:37.780
Backslash pipe specifies an alternative, blah, blah, blah.
32:40.820 --> 32:47.940
When we want to build more complex regular expressions, patterns, grammars, etc.,
32:47.940 --> 32:51.780
we have to use an external library for that.
32:52.740 --> 32:59.140
No, sorry, a library that is external, but that was written by one of the authors of Lua itself.
33:00.340 --> 33:05.380
This library is called LPEG, and its manual says,
33:06.180 --> 33:11.940
LPEG is a new pattern matching library for Lua based on parsing expression grammars, PEGs.
33:14.900 --> 33:17.460
The manual is very terse.
33:18.180 --> 33:20.500
I found it incredibly hard to read.
33:20.900 --> 33:22.740
It doesn't have any diagrams.
33:22.740 --> 33:24.420
It has some examples, though.
33:26.180 --> 33:34.420
And the Lua wiki has a big page called LPEG tutorial with lots of examples.
33:35.700 --> 33:42.020
But it also doesn't have diagrams, and I found some things incredibly hard to understand.
33:42.020 --> 33:48.500
For example, this is something that is in the manual of LPEG that I saw and I thought,
33:49.460 --> 33:53.060
wow, great, this makes all sense, and this is going to be very useful.
33:53.060 --> 33:57.300
It's a way to build grammars that can be recursive,
33:57.300 --> 34:02.020
and they sort of can encode BNF grammars.
34:02.020 --> 34:06.980
We just have to translate the BNF a bit to get rid of some recursions
34:06.980 --> 34:08.820
and to translate them to something else.
34:10.580 --> 34:13.620
And the manual also has some things that I thought,
34:13.620 --> 34:16.580
oh, no, I don't have any idea of what this thing does.
34:17.380 --> 34:22.020
And in fact, I saw these things for the first time more than 10 years ago,
34:22.020 --> 34:26.020
and they only started to make sense one year ago.
34:28.420 --> 34:30.340
One example is group captures.
34:33.380 --> 34:37.620
LPEG also comes with a module called the RE module.
34:37.620 --> 34:40.180
Let me pronounce it in Portuguese, the RE module.
34:40.660 --> 34:45.540
And its manual says, the RE module provided by the file repo.luen
34:45.540 --> 34:48.100
in the distribution supports a somewhat conventional
34:48.980 --> 34:54.420
regular expression syntax for pattern usage within LPEG.
34:55.140 --> 34:57.460
And this is a quick reference.
35:00.500 --> 35:02.820
And this thing is very brief.
35:02.820 --> 35:06.420
It has some nice examples, but it's hard to understand in a way.
35:06.660 --> 35:12.500
And here are some comments about my attempts to learn HerdotLua.
35:14.020 --> 35:15.220
This is a class.
35:15.220 --> 35:17.060
In this case, it's a very small class.
35:18.180 --> 35:21.940
And this file implements a PM method.
35:23.540 --> 35:26.980
I'm going to show examples of other PM methods very soon.
35:28.020 --> 35:33.380
So this is a PM method for HerdotLua that lets us compare the syntax
35:33.620 --> 35:39.460
of Lua patterns, LPEG, and HerdotLua.
35:41.620 --> 35:43.060
See this example here.
35:43.700 --> 35:49.940
So if we run this, it loads my version of LPEG.
35:50.660 --> 35:52.740
No, sorry, my version of LPEG-REX.
35:54.820 --> 36:00.180
And it shows that when we apply the PM method to this Lua pattern,
36:00.740 --> 36:07.380
this LPEG pattern, and this RE pattern, they all get the same results.
36:07.380 --> 36:12.820
So we can use this thing, this kind of thing here to show how to translate from
36:14.420 --> 36:20.180
Lua patterns that are familiar because they are similar to regular expressions, only weaker,
36:22.260 --> 36:27.620
to LPEG that is super weird, and to RE that is not so weird.
36:28.580 --> 36:37.940
Anyway, the comment says that in 2012, I had a project that needed a precedence parser that
36:37.940 --> 36:42.900
could parse arithmetical expressions with the right precedences.
36:44.580 --> 36:50.180
And at that point, I was still struggling with pure LPEG, and I couldn't do much with it.
36:50.180 --> 36:55.300
So I tried to learn HerdotLua instead, and I wrote this old class here.
36:56.260 --> 36:59.620
That allowed me to use preprocessor on patterns for Lua.
36:59.620 --> 37:04.980
And the thing is that with this preprocessor, I could specify precedence grammars using this
37:04.980 --> 37:12.900
thing here that worked, but it was super clumsy, and I gave up after a few attempts.
37:14.900 --> 37:21.300
And in 2022, I heard about something called LPEG-REX that was an
37:22.260 --> 37:28.740
a kind of extension of Re, and it was much more powerful than HerdotLua, but after a while,
37:28.740 --> 37:32.660
I realized that it had the same defects as HerdotLua.
37:32.660 --> 37:41.620
And let me explain that because it has all to do with the things about black boxes and magic
37:41.620 --> 37:42.980
that I told in the beginning.
37:43.380 --> 37:50.100
Both, I mean, sorry, neither HerdotLua or LPEG-REX had some features that I needed.
37:51.620 --> 37:57.940
They didn't let us explain, sorry, they received a pattern that was specified as a string,
37:57.940 --> 38:04.740
and it converted that into an LPEG pattern, but it didn't let us explore the LPEG patterns
38:04.740 --> 38:05.540
that it was using.
38:05.540 --> 38:12.900
So I had to use the LPEG-REX, and it didn't let me explore the LPEG patterns that it was
38:12.900 --> 38:13.540
generated.
38:16.420 --> 38:21.060
Their code was written in a way that was REPL unfriendly.
38:21.060 --> 38:28.580
I couldn't modify parts of the code bit by bit in a REPL and try to change the code
38:29.700 --> 38:31.620
without changing the original file, say.
38:33.300 --> 38:37.220
The code was very hard to explore, to hack, and to extend, in my opinion.
38:37.780 --> 38:39.700
The documentation was not very clear.
38:40.580 --> 38:50.660
And I sent one or two messages to the developer of LPEG-REX, and he was too busy to help me.
38:50.660 --> 38:55.460
He answered very briefly, and to be honest, I felt rejected.
38:55.460 --> 38:58.580
I felt that I wasn't doing anything interesting.
38:58.580 --> 38:59.380
Whatever, whatever.
39:00.340 --> 39:11.940
So, in 2022, I was trying to learn LPEG-REX, because I was thinking that it would solve
39:11.940 --> 39:13.780
my problems, but it didn't.
39:14.500 --> 39:20.900
It didn't have the features that I needed, and it was hard to extend, and hard to explore,
39:20.900 --> 39:22.100
and hard to debug.
39:22.980 --> 39:32.420
I decided to rewrite it in a more hacker-friendly way, in the sense that it was modular, and
39:32.420 --> 39:35.460
I could replace any part of the module from a REPL.
39:37.300 --> 39:48.340
My version of it was called lpeg1.lua, and I decided that in my version, I wouldn't have
39:49.060 --> 39:55.460
the part that receives a grammar specified as a string and converts that to LPEG.
39:55.460 --> 40:03.780
I would just have the backend part that are the functions in LPEG that let us specify
40:04.340 --> 40:05.380
powerful grammars.
40:10.340 --> 40:11.540
So, let me go back.
40:12.260 --> 40:14.260
Let me explain a bit about LPEG.
40:14.900 --> 40:16.180
Lua has coercions.
40:18.660 --> 40:22.740
The plus expects to receive two numbers.
40:22.740 --> 40:29.860
If one of its arguments, or both of them, are strings, it converts the strings to numbers.
40:29.860 --> 40:39.220
So, in this case here, 2 plus string 3 returns the number 5, and this is the concatenation
40:39.220 --> 40:39.860
operator.
40:40.820 --> 40:42.820
It expects to receive strings.
40:43.780 --> 40:49.860
So, in this case, it will convert the number 2 to the string 2, and the concatenation of
40:49.860 --> 40:51.700
these two things will be 23.
40:52.420 --> 40:54.420
Sorry, 23 as a string.
40:56.260 --> 40:58.420
LPEG also has some coercions.
41:00.260 --> 41:09.620
I usually set these globals to let me write my grammars in a very compact way.
41:10.260 --> 41:17.460
So, instead of lpeg.p, lpeg.c, etc., I use these globals like uppercase B, uppercase
41:17.460 --> 41:18.420
C, and so on.
41:19.540 --> 41:27.540
And with these globals, I can write things like this, c1 times string underscore.
41:28.420 --> 41:40.820
And LPEG knows that lpeg.c, sorry, it sort of expands these to lpeg.c, but lpeg.c expects
41:40.820 --> 41:47.380
to receive an LPEG pattern, and one is not yet an LPEG pattern, so it is coerced into
41:47.380 --> 41:50.900
an LPEG pattern by calling lpeg.p.
41:51.220 --> 42:02.740
So, this shorting here becomes equivalent to lpeg.c, lpeg.p1, and the multiplication,
42:02.740 --> 42:09.940
when at least one of its arguments is an LPEG pattern, it expects to receive two LPEG patterns,
42:09.940 --> 42:15.700
and in this case, the one at the right is just a string, so it is coerced to an LPEG
42:15.700 --> 42:17.620
pattern by using lpeg.p.
42:18.180 --> 42:22.180
With this idea, we can sort of understand this comparison here.
42:22.900 --> 42:24.980
I mean, let me run it again.
42:24.980 --> 42:33.620
This first part is very similar to a regular expression here at the left, and when we apply
42:33.620 --> 42:45.940
this LPEG, sorry, this Lua pattern to this subject here, the result is that the Lua pattern
42:46.740 --> 42:53.540
the result is this thing here, this thing, this thing, and this thing.
42:53.540 --> 43:00.580
I'm going to call each one of these results captures, so each of these things between
43:00.580 --> 43:08.020
parentheses captures a substring of the original string, and these captured substrings are
43:08.020 --> 43:09.460
returned in a certain order.
43:10.420 --> 43:12.740
Here is how to express the same thing in LPEG.
43:13.540 --> 43:22.660
It's very cryptic, but it's a good way to understand some basic operators of LPEG.
43:22.660 --> 43:34.900
I mean, we can look at the menu and understand what C, S, and R do, and also exponentiation.
43:35.860 --> 43:43.060
And this strange thing here receives this string here, runs a function that I have defined
43:43.060 --> 43:50.260
that converts it to an object of a certain class, and that class represents He patterns.
43:50.260 --> 43:56.900
So this thing is treated as a pattern for He.Lua, and it is matched against the string,
43:56.900 --> 43:59.300
and it returns the same thing as the other one.
43:59.860 --> 44:06.260
Also, this thing here also has a comparison with LPEG-REGS, but these patterns are very
44:06.260 --> 44:06.820
trivial.
44:06.820 --> 44:10.180
They don't do anything very strange.
44:10.180 --> 44:14.260
So let's go back and see what kinds of very strange things there are.
44:16.660 --> 44:23.620
Here is the page of LPEG-REGS at GitHub.
44:24.260 --> 44:25.860
Here's the documentation.
44:27.940 --> 44:29.380
It's relatively brief.
44:29.380 --> 44:35.300
It explains LPEG-REGS as being an extension of He.Lua.
44:36.500 --> 44:40.100
So it explains mainly the additional features.
44:40.100 --> 44:44.180
Here is a quick reference that explains only the additional features.
44:46.820 --> 44:53.380
Some of these things I was able to understand by using the LPEG-REGS.
44:54.340 --> 45:02.820
I was struggling a lot, and some I wasn't able to, even by spending several evenings
45:02.820 --> 45:04.420
trying to build examples.
45:08.260 --> 45:10.900
And this is something very nice.
45:12.100 --> 45:19.700
LPEG-REGS comes with some example parsers, and here is a parser that parses the Lua grammar.
45:19.700 --> 45:28.100
I mean, this is the grammar for Lua 5.4 at the end of the reference manual.
45:28.660 --> 45:35.860
It's just this, and this is a kind of the BNF, and this is the BNF translated to
45:37.300 --> 45:39.860
the language of LPEG-REGS.
45:40.420 --> 45:47.300
So this thing uses many constructions that are in He.Lua and some extra constructions
45:47.300 --> 45:49.460
that are described here.
45:50.260 --> 45:58.980
And with these examples, I was able to understand some of these things here that are described
45:58.980 --> 46:02.660
here in the quick reference, but not all.
46:06.660 --> 46:15.300
So I wasn't able to use LPEG-REGS by itself because some things didn't make much sense,
46:15.300 --> 46:18.900
and I decided to reimplement it in my own style.
46:20.900 --> 46:27.860
Because that would be a way to map, at the very least, map what I understood and what
46:27.860 --> 46:32.980
I didn't, and learn one feature at a time, do comparisons, and so on.
46:35.380 --> 46:38.900
Here, I pointed to two features of LPEG.
46:38.900 --> 46:44.820
One, I said, oh great, this thing can be used to define grammars, even recursive
46:44.900 --> 46:46.020
grammars, and so on.
46:46.740 --> 46:52.820
And this is an oh-no feature, one thing that didn't make any sense at all, group captures.
46:55.380 --> 47:01.620
One thing that I did to understand group captures was to represent them as diagrams.
47:01.620 --> 47:08.020
Of course, in the beginning, I was drawing these diagrams by hand, but then I realized
47:08.020 --> 47:15.220
that I could use the bits of LPEG that I already knew to build a grammar that would
47:15.220 --> 47:19.380
parse a little language and generate these diagrams in LaTeX.
47:20.340 --> 47:21.940
And I was able to make this.
47:24.180 --> 47:34.500
In this diagram here, this thing above the arrow is a piece of Lua code that specifies
47:34.500 --> 47:36.100
an LPEG pattern.
47:36.820 --> 47:43.540
This thing here at the top is the string that is being matched, and the things below the
47:43.540 --> 47:51.220
under braces are the captures that each thing, sorry, each thing captures.
47:54.580 --> 48:02.100
And for example, this under brace here corresponds to this pattern here that parses a single
48:02.100 --> 48:05.060
character but doesn't return any captures.
48:05.140 --> 48:10.420
This thing here parses a single B and doesn't return any captures.
48:10.420 --> 48:16.740
This thing here parses a single character and captures it, and this thing here parses
48:17.620 --> 48:19.700
the character D and captures it.
48:20.340 --> 48:29.220
And this other thing here that transforms this pattern into another pattern returns
48:29.780 --> 48:36.500
first a capture with all the string that was parsed by this pattern here, and then all
48:36.500 --> 48:41.380
the captures returned by this thing here before the column.
48:43.380 --> 48:50.900
So this was a way to build concrete examples for things that the LPEG manual was explaining
48:50.900 --> 48:53.940
in a very terse way, and it worked for me.
48:54.740 --> 49:01.540
Some things that were very mysterious started to make sense, and I started to have intelligent
49:01.540 --> 49:03.300
questions to ask in the mailing list.
49:07.700 --> 49:16.980
And with that, I was able to understand what are group captures and group captures that
49:16.980 --> 49:17.860
receive a name.
49:20.900 --> 49:23.140
Well, let me explain what this does.
49:23.220 --> 49:30.180
This thing here captures, sorry, parses the empty string and returns this as a constant.
49:30.180 --> 49:35.860
So this is something that doesn't exist in regular expressions.
49:36.980 --> 49:40.180
It parses nothing and returns this as a capture.
49:40.820 --> 49:48.580
Then this thing here returns these two constants here and parses the empty string, and this
49:48.660 --> 49:56.500
thing here, d, converts the results of this thing here into a group capture and stores
49:56.500 --> 49:58.020
it in the label d.
50:01.380 --> 50:03.860
And then here's another constant capture.
50:03.860 --> 50:11.460
And I realized that these things here were similar to how Lua specifies building lists.
50:11.780 --> 50:16.580
When we build, sorry, a table, when we build a table and we say that the first element
50:16.580 --> 50:20.420
of the table is here, this element is put at the end of the table.
50:21.140 --> 50:30.820
When after that we say d equals to, say, 42, we are putting the 42 in the slot whose key
50:30.820 --> 50:31.380
is d.
50:33.380 --> 50:37.860
This was happening with LPEG captures, but there was something very strange.
50:37.860 --> 50:46.260
These group captures could hold more than one capture, more than one value.
50:46.260 --> 50:50.340
So there was something between lists and tables.
50:51.300 --> 51:00.740
I started to use this notation to explain in my notation what they were doing.
51:02.340 --> 51:04.900
Many things start, things stop.
51:05.220 --> 51:08.900
Many things start, things started to make sense.
51:08.900 --> 51:14.420
Many mysterious sentences in the manual started to make sense, but some didn't.
51:16.100 --> 51:25.060
But at least I was able to send some intelligent questions to the mailing list, and the author
51:25.060 --> 51:27.540
of Lua and LPEG answered some of them.
51:29.700 --> 51:31.860
He was not very happy about my questions.
51:32.020 --> 51:40.100
He told me that those diagrams were a waste of time, that the manual was perfectly clear,
51:40.100 --> 51:42.900
and so on, whatever.
51:42.900 --> 51:51.620
But I was able to, so it was weird, but I was able to understand lots of things from
51:51.620 --> 51:52.340
his answers.
51:53.940 --> 51:57.380
So this is a copy of one of my messages.
51:57.380 --> 51:59.300
Then there's another one, another one.
51:59.300 --> 52:04.580
Some were the diagrams, then he complained about these diagrams.
52:04.580 --> 52:12.980
He said that these things here that look like table constructions do not exist, whatever.
52:15.380 --> 52:23.220
Anyway, once I understood group captures, many features were very, very easy to understand,
52:23.780 --> 52:28.260
and I started to be able to use LPEG to build some very interesting things.
52:30.260 --> 52:36.340
I was able to reproduce some of the features that I saw in LPEG-REX.
52:38.420 --> 52:40.980
Remember that this, where is that?
52:42.420 --> 52:44.020
This is a syntax of Lua.
52:45.860 --> 52:54.820
Here, I was able to understand how these things here were translated to LPEG code, to LPEG
52:54.820 --> 52:58.340
patterns by using group captures in a certain way.
52:59.460 --> 53:05.060
And I was able to implement them in lpeg1.lua.
53:06.340 --> 53:14.180
And after some time, I was able to use lpeg1.lua to build grammars that were able to parse
53:17.300 --> 53:19.860
arithmetical expressions with the right precedence.
53:19.860 --> 53:25.700
And here's an example in which I built the grammar step by step, and I test the current
53:25.700 --> 53:29.300
grammar, and I replace a bit, and then I test the new grammar, and so on.
53:31.780 --> 53:38.580
And you can see that the result is always a tree that is drawn in a nice two-dimensional way.
53:42.580 --> 53:52.340
At this point, these powers here are returned as a list, as an operation
53:52.900 --> 53:56.260
pow with several arguments here.
53:56.820 --> 54:04.820
And then I apply a kind of parsing combinator here that transforms these trees into other trees.
54:05.540 --> 54:14.180
And with these combinators here, I can specify that the power is associative in a certain
54:14.180 --> 54:20.340
direction, the division is associative in another direction, the minus is associative,
54:20.420 --> 54:24.420
so it uses the same direction as a division, and so on.
54:24.420 --> 54:26.020
And they have the right precedences.
54:28.660 --> 54:30.020
So here are the tests.
54:33.700 --> 54:37.140
So here's my file lpeg1.lua.
54:37.140 --> 54:38.340
It has several classes.
54:38.980 --> 54:42.980
Each class has tests after it.
54:43.940 --> 54:51.780
I was able to implement something that lpeg-reqs has that's called keywords.
54:51.780 --> 54:56.980
That's very useful for parsing programs in programming languages.
54:56.980 --> 55:06.580
I was able to implement something similar to the debugger, to the PEG debugger that lpeg uses,
55:07.220 --> 55:12.740
but I was frustrated by some limitations of that debugger,
55:12.740 --> 55:16.180
and I implemented my own that is much better.
55:17.700 --> 55:23.860
And let me show something else.
55:23.860 --> 55:32.900
I was able to translate a good part of the Lua parser here to lpeg1.lua.
55:33.700 --> 55:39.940
I haven't finished yet, but I have most of the translation here.
55:43.140 --> 55:50.420
And after having all that, I was able to build other grammars very quickly.
55:51.380 --> 55:59.220
Writing new parsers finally became fun, and here's one example that I showed in the beginning.
56:03.620 --> 56:11.220
If I remember correctly, I took a figure from the Wikipedia.
56:11.220 --> 56:17.220
I don't have its link now, but I specified a grammar that parses
56:18.500 --> 56:21.140
exactly the example that appears in the Wikipedia.
56:21.140 --> 56:28.100
So with my grammar, considering that the top-level entry is statement,
56:28.100 --> 56:36.900
when I parse this string here, the result is this tree.
56:38.100 --> 56:41.780
And I can do some operations on that.
56:41.780 --> 56:45.780
I can define how this thing is to be converted into LaTeX.
56:47.460 --> 56:51.700
I can define other operations that convert trees into other trees.
56:52.580 --> 56:54.980
And here are some tests of these operations.
56:58.340 --> 57:00.660
This is what I showed in the beginning.
57:00.660 --> 57:04.100
I'm not going to explain all the details of this thing now.
57:07.460 --> 57:14.260
This show converts this thing into LaTeX in the way specified by these instructions here.
57:15.380 --> 57:16.340
Let's say that...
57:19.540 --> 57:20.260
Well, whatever.
57:22.740 --> 57:23.540
Really, whatever.
57:24.020 --> 57:28.500
And here's the result, the LaTeX result.
57:34.900 --> 57:46.260
And these diagrams here are generated by this file here that defines a simple grammar that parses
57:46.260 --> 57:53.460
this thing here, and then LaTeX it in a certain way, and then also tests this thing.
57:54.100 --> 58:00.900
To check if this code here, that is Lua code that generates an LPEG grammar,
58:02.900 --> 58:08.740
parses this subject here and returns the expected result.
58:12.740 --> 58:15.140
So this is the code that I wanted to show.
58:15.700 --> 58:21.700
I wanted to show many more things, but I wasn't able to prepare them before the conference.
58:22.580 --> 58:29.060
And I hope that soon, for some value of soon, I'll be able to create
58:29.620 --> 58:37.060
REPL-based tutorials for LPEG here and lpeg1.lua, where LPEG is something very famous.
58:37.060 --> 58:39.460
Here is a module of LPEG.
58:40.500 --> 58:48.100
I could also do something like this for LPEG-REX, and lpeg1.lua is the thing that I wrote,
58:48.820 --> 58:58.900
the one that has tests in comments, and the tests usually generate trees,
58:58.900 --> 59:00.900
and sometimes they generate tag code.
59:03.300 --> 59:04.420
Yeah, so that's it.
59:04.420 --> 59:07.860
I wanted to present much more, but I wasn't able to prepare it.
59:07.860 --> 59:08.660
So sorry.
59:08.660 --> 59:09.140
Thanks.
59:09.140 --> 59:10.260
Bye.
