path: root/2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt

WEBVTT

00:00.000 --> 00:04.880
Hi, my name is Eduard Duax, and the title of this talk is

00:04.880 --> 00:09.680
Rapples in Strange Places, Lua, LaTeX, LPEG, LPEG-REX, and TIX.

00:10.720 --> 00:17.200
I'm the author of an MX package called EEV, and this is a talk at the MXConf 2023

00:17.200 --> 00:20.880
that is happening in December 2023 at the Internets.

00:21.680 --> 00:25.920
And this is one of the examples of diagrams that we are going to see.

00:25.920 --> 00:27.680
Let me show how I generate it.

00:28.640 --> 00:32.240
One second. I have to use a smaller font here.

00:35.600 --> 00:41.920
This is a file called parse32.lua. Let me go back to this block of tests again.

00:42.720 --> 00:50.320
And now if I run this, we get these outputs here at the right.

00:52.000 --> 00:54.960
And then in this line here, it generates a PDF.

00:55.520 --> 01:01.440
And if I type f8 here, it shows the PDF in the lower right window.

01:04.240 --> 01:07.600
Let me start by explaining briefly what is EEV.

01:09.920 --> 01:14.560
First, it's something that appeared by accident in the mid-90s.

01:14.560 --> 01:19.920
I explained this story in my presentation at the MXConf 2019.

01:20.720 --> 01:25.520
It's a package. It's an MX package that is part of ELPA.

01:25.520 --> 01:31.440
It has at least 10 users. Those are the ones that I know by name.

01:33.360 --> 01:36.320
EEV means MX Execute Verbosely.

01:37.360 --> 01:42.640
EEV is something that treats eval as the central feature of MX.

01:43.680 --> 01:46.800
EEV blurs the distinction between programmers and users,

01:47.520 --> 01:50.080
and it replaces the slogan,

01:50.080 --> 01:55.360
users should not be forced to see Lisp, that is something that Richard Stallman told me once,

01:55.360 --> 01:59.280
by users should see Lisp instead of buttons,

01:59.280 --> 02:02.400
and new users should see Lisp in the first five minutes.

02:04.240 --> 02:07.120
I'm going to show some examples of that soon.

02:08.800 --> 02:12.880
EEV uses code in comments a lot, and also tests in comments.

02:13.600 --> 02:21.040
I changed my way of presenting it, and it became very REPL-centric in the last few years,

02:21.040 --> 02:28.240
in the sense that I start by explaining its main features by its support for REPLs.

02:30.320 --> 02:34.320
EEV supposes that we want to keep executable notes of everything.

02:34.320 --> 02:37.040
I'm also going to show examples of this in a second.

02:38.080 --> 02:41.600
EEV has lots of videos for people who hate videos,

02:42.240 --> 02:47.520
and it tries to do everything with very little magic and without black boxes.

02:47.520 --> 02:50.240
I'm going to explain many of these things very soon.

02:52.880 --> 02:58.160
This is a figure that I'm going to show in detail soon,

02:58.160 --> 03:01.520
that is about something important about Lua.

03:01.520 --> 03:05.280
Oops, the font is very bad now, so let me change the font.

03:05.280 --> 03:07.840
The figure is this one, and

03:08.560 --> 03:13.440
what most people do when they visit a file with something interesting on it

03:13.440 --> 03:16.640
is that they just go there, and they set a bookmark there,

03:16.640 --> 03:19.760
or they put the position in the register.

03:21.760 --> 03:28.480
But I prefer to keep links to everything that is interesting as a list of hyperlinks.

03:28.480 --> 03:32.640
So, for example, this is an at least hyperlink to the file

03:32.640 --> 03:36.480
that goes to this anchor here, and to this string I've added.

03:36.480 --> 03:38.560
And to this string after this anchor.

03:39.760 --> 03:45.760
This is a variant that opens that file in the window at the right here.

03:47.440 --> 03:53.520
And this is a sexpid that changes the font.

03:53.520 --> 03:57.680
I have a command with a very short name that does that,

03:58.400 --> 04:02.880
but I prefer to keep that as a one-liner.

04:03.840 --> 04:11.440
About the videos, we can see the list of first-class videos of Eevee by executing this,

04:11.440 --> 04:16.960
meta-x, find first-class videos, or by running this alias here,

04:16.960 --> 04:20.320
meta-x1c, and then what we see is this.

04:21.520 --> 04:25.440
The first sexpid here regenerates this buffer,

04:25.440 --> 04:28.960
so we can make a mess here and then run this,

04:29.040 --> 04:33.040
and the original buffer is regenerated again in a clean way.

04:34.880 --> 04:42.000
Each of these things here opens a buffer with information about a video.

04:42.000 --> 04:44.400
Let me take a specific example here.

04:46.080 --> 04:50.720
This video here is about one of the ancestors of this talk,

04:50.720 --> 04:52.960
that is a library that I wrote for

04:53.920 --> 05:01.120
creating diagrams in LaTeX using a package called pic2e, using repls.

05:02.320 --> 05:10.720
Anyway, the thing is that if we run a sexpid like this one,

05:10.720 --> 05:13.440
and we don't have a local copy of the video,

05:14.080 --> 05:16.320
Eevee will try to load the local copy,

05:16.320 --> 05:19.600
and instead of doing that and by asking something like,

05:19.680 --> 05:20.720
asking something like,

05:20.720 --> 05:24.560
do you want me to download the local copy, blah, blah, blah, blah,

05:25.680 --> 05:28.720
it simply opens a buffer like this.

05:28.720 --> 05:32.480
I mean, if we don't have a local copy yet,

05:32.480 --> 05:34.400
it will open a buffer like this one,

05:35.440 --> 05:41.760
in which these things here in comments are links to the documentation.

05:41.760 --> 05:48.240
I mean, this thing here explains the idea of local copies of files from the internet,

05:48.880 --> 05:52.960
uh, there are more details here and here,

05:53.840 --> 05:59.280
and this is a script that we can execute line by line.

05:59.280 --> 06:04.080
So, instead of this script being hidden behind the button that we just

06:04.960 --> 06:07.360
press after a question like,

06:07.360 --> 06:09.920
do you want me to do something, blah, blah, blah, yes or no,

06:11.200 --> 06:15.120
the script is visible here, and we can execute it step by step.

06:15.200 --> 06:22.080
It creates a terminal with a shell here in the right window,

06:22.080 --> 06:28.880
and when we type f8 in these lines here, the lines are sent to this line.

06:30.080 --> 06:33.920
So, this is going to download the copy of the video,

06:33.920 --> 06:39.360
the wget says that I already have a copy of the video in its subtitles, and so on.

06:39.920 --> 06:47.600
And after getting a copy of the video, we can run this exp here, and it displays the video.

06:52.880 --> 06:57.840
I said that Eevee has lots of videos for people who hate videos,

06:57.840 --> 07:02.960
and the idea is that very few people are going to watch the videos in real time,

07:03.920 --> 07:10.480
and most of the people that I know, or most of the people that are interested in Eevee in some way,

07:10.480 --> 07:15.280
they are going to watch just small sections of the video,

07:15.280 --> 07:18.720
and most of the time they're just going to read the subtitles of the video.

07:19.680 --> 07:25.760
So, for each one of the videos, we have a page about the video.

07:25.760 --> 07:28.400
Let me see if I have internet here.

07:28.400 --> 07:29.680
Yes, this is a page,

07:29.920 --> 07:36.080
and usually these pages have a link to another page,

07:36.080 --> 07:42.880
the page that has all the subtitles of the video, wherever.

07:42.880 --> 07:46.960
In this one, it's not so visible, but anyway,

07:46.960 --> 07:50.480
there are several ways of accessing the subtitles of the video,

07:50.480 --> 07:54.720
and one of the ways is by running this exp here,

07:55.040 --> 07:57.280
is by running this exp here,

07:59.440 --> 08:03.680
that opens a file in Lua that is what I use to generate the subtitles.

08:04.880 --> 08:13.760
Anyway, by the way, each one of these things here is hyperlinked to a position of the video,

08:14.400 --> 08:19.280
so if I type this in the right way, it goes to that position.

08:20.160 --> 08:21.280
Anyway, let me go back.

08:21.920 --> 08:26.640
Also, the tutorials of Eevee, the intros of Eevee,

08:26.640 --> 08:30.560
let's start with find, and with intro,

08:30.560 --> 08:35.760
they have lots of blocks that say video links, like this one,

08:36.800 --> 08:40.080
and these blocks have links to the positions in videos,

08:40.080 --> 08:43.600
and if we don't have a local copy of the video yet,

08:44.480 --> 08:48.800
the thing shows us a script that lets us download the local copy.

08:48.880 --> 08:56.720
Anyway, I said that I was going to explain what I mean by magic and black boxes.

08:59.360 --> 09:02.080
This is something that I've been trying to explain for a long time,

09:02.080 --> 09:05.840
and I think that I got a very good explanation about that

09:05.840 --> 09:09.440
in a video that I made about something called Eevee wconfig,

09:09.440 --> 09:15.120
that is a tool for configuring Eevee on Windows without magic,

09:15.840 --> 09:20.320
on Windows without magic, without buttons that do things

09:21.040 --> 09:22.720
without explaining what they are doing.

09:24.080 --> 09:26.720
This is a part of the subtitles of the video.

09:26.720 --> 09:27.680
Let me read that.

09:30.080 --> 09:32.640
Eevee wconfig is an attempt to solve the problem

09:32.640 --> 09:35.040
of how to install these things on Windows,

09:35.040 --> 09:38.240
both without magic and with very little magic.

09:39.520 --> 09:40.640
Remember this slogan,

09:41.200 --> 09:45.760
any sufficiently advanced technology is indistinguishable from magic.

09:47.280 --> 09:51.600
Here in this video, I'm going to use the term magic as a shorthand

09:51.600 --> 09:55.520
for sufficiently advanced technology,

09:55.520 --> 09:59.040
that is something that is complex and non-obvious,

09:59.040 --> 10:01.840
and that is indistinguishable from magic

10:01.840 --> 10:04.640
in the sense of being almost impossible to understand.

10:05.600 --> 10:10.720
And I'm also going to use the term black box as a near synonym for magic,

10:11.360 --> 10:15.360
and sometimes the term black box is more convenient,

10:15.360 --> 10:18.800
even though it's a bit longer, it has more letters,

10:18.800 --> 10:20.880
because when I use the term black box,

10:20.880 --> 10:25.440
it invites us to use expressions like opening the black box,

10:25.440 --> 10:27.840
and I'm going to use that expression a lot.

10:34.640 --> 10:37.360
Now, let me try to explain what is...

10:37.360 --> 10:40.400
Sorry, let me change the font.

10:44.000 --> 10:44.800
What is Lua?

10:46.000 --> 10:51.840
Lua is a minimalistic language in the sense of batteries not included.

10:52.880 --> 10:56.640
It uses associative tables for most of its data structures,

10:58.800 --> 11:02.240
and it's so minimalistic that it's the default print function.

11:03.120 --> 11:09.280
When we create an associative table and we ask it to print...

11:11.760 --> 11:14.560
We ask print to print an associative table,

11:14.560 --> 11:17.280
it just prints the address of the table.

11:17.280 --> 11:18.480
Here are some examples.

11:19.920 --> 11:23.760
Here is a table, and when we ask print to print it,

11:23.760 --> 11:26.720
it just says that it's the table at this address here.

11:27.440 --> 11:32.400
So, one of the things that most people do when they start using Lua

11:32.400 --> 11:36.320
is that either they download a package with a print to printing function,

11:36.320 --> 11:38.720
or they write their own print to printing functions.

11:39.360 --> 11:42.000
My own print to printing function is called pp,

11:42.000 --> 11:45.840
with uppercase letters, and it works like this,

11:47.600 --> 11:50.880
and it prints associative tables in a way like this.

11:50.880 --> 11:54.800
It says that for the key 1,

11:55.200 --> 11:57.280
the value associated to it is 2,

11:57.280 --> 11:59.200
for the key 2, the value is 3,

11:59.200 --> 12:01.760
and for the key 3, the value is 5.

12:08.960 --> 12:12.160
When I started using Lua, one of my favorite languages

12:12.160 --> 12:15.600
was also a language that used associative tables a lot.

12:15.600 --> 12:16.720
It was called Icon,

12:18.400 --> 12:22.640
and I had to write my own print to printing functions for Icon.

12:22.720 --> 12:28.800
So, I just had to port my print to printing functions to Lua,

12:28.800 --> 12:31.600
and my first version looked something like this.

12:32.240 --> 12:36.560
It just had some global functions, lots of them actually,

12:39.680 --> 12:41.840
and after a while I rewrote it,

12:41.840 --> 12:44.320
and I rewrote it again, and again, and again,

12:44.320 --> 12:49.200
and this is one of the versions that is not even the default at this point.

12:50.000 --> 12:53.920
TOS is for toString,

12:56.800 --> 12:57.920
and this is a demo.

12:58.800 --> 13:04.560
It's very modular, so it's easy to replace parts of it or to toggle flags,

13:04.560 --> 13:05.760
and this is an example.

13:05.760 --> 13:10.320
If I try to print the table of methods for a certain class,

13:11.760 --> 13:13.040
I'll need a smaller font,

13:14.320 --> 13:16.480
it prints the table like this,

13:16.480 --> 13:18.480
with the names of the methods,

13:18.560 --> 13:21.520
and then links to the source code of the functions.

13:22.080 --> 13:25.360
These links only make sense in Emacs and in Eevee,

13:26.000 --> 13:28.560
and when we run a link like this one,

13:29.840 --> 13:33.920
it shows the source code in the window at the right.

13:33.920 --> 13:37.760
So, for some functions, the source code is three lines,

13:37.760 --> 13:40.400
for other ones it's one line, and whatever.

13:41.920 --> 13:43.200
Anyway, let me go back.

13:45.360 --> 13:48.000
Lua can be used in many different styles.

13:48.000 --> 13:50.160
Most people hate other people's styles.

13:51.920 --> 13:56.080
When I started using it in the year 2000,

13:56.080 --> 13:59.040
I learned most of the basic language in a single day.

13:59.040 --> 14:02.000
It was very similar to things that I was already using,

14:02.800 --> 14:08.560
and then I rewrote the mini language that I was using

14:08.560 --> 14:14.800
to generate the HTML for my pages in Lua.

14:14.800 --> 14:16.960
Actually, I had to rewrite it many times,

14:16.960 --> 14:20.880
but the first version I certainly did in my first weeks

14:20.880 --> 14:22.480
or first months using Lua.

14:25.520 --> 14:29.040
In the beginning, I was just using it for writing programs

14:29.040 --> 14:32.160
that either didn't take in any input at all,

14:32.160 --> 14:35.280
because the input was already in the source file,

14:35.280 --> 14:39.200
or that worked as a Unix function,

14:39.200 --> 14:42.960
the Unix programs that would read files

14:42.960 --> 14:46.160
and process these files in some way and output something.

14:47.840 --> 14:52.480
I mentioned the basic language here.

14:52.480 --> 14:55.520
I only learned how to use closures, metatables,

14:55.520 --> 14:58.240
and coroutines many years later.

15:00.320 --> 15:02.240
In the beginning, when I started using Lua,

15:02.240 --> 15:04.080
it didn't have a package manager.

15:04.080 --> 15:05.200
It appeared later.

15:05.200 --> 15:06.560
It is called Lua-rocks.

15:07.680 --> 15:11.600
It has had this package manager for several years.

15:12.720 --> 15:15.520
Most of the rocks for Lua-rocks are poorly documented,

15:16.480 --> 15:19.040
documented, and hacker-unfriendly,

15:19.040 --> 15:21.600
so you can't rely just on the documentation,

15:22.160 --> 15:24.320
and you can't rely just on the source code,

15:24.320 --> 15:28.320
because, I mean, if you're a genius, of course you can,

15:28.320 --> 15:32.320
but for people who are either lazy or dumb or whatever,

15:32.320 --> 15:33.920
like me, or unfocused,

15:35.120 --> 15:37.120
the source code is hard to understand

15:37.120 --> 15:38.800
and hard to tinker with.

15:41.440 --> 15:43.040
Some rocks are excellent.

15:43.040 --> 15:46.640
The best rocks are well documented,

15:46.640 --> 15:48.640
but they are hacker-unfriendly,

15:48.640 --> 15:51.840
in the sense that I hope that I'll be able to explain soon.

15:53.440 --> 15:58.560
The best rocks use local variables and metatables a lot,

15:59.280 --> 16:03.120
so if you are a beginner learning Lua,

16:03.120 --> 16:06.080
you're not going to understand what their source codes do.

16:06.080 --> 16:08.320
They use lots of dirty tricks.

16:08.640 --> 16:12.240
Let me talk a bit about object orientation in Lua.

16:12.240 --> 16:13.840
It can be done in many ways.

16:15.360 --> 16:18.000
The main book about Lua, called Programming in Lua,

16:18.000 --> 16:20.240
by one of the authors of the language,

16:21.120 --> 16:22.320
Roberto Ierzalimschi,

16:23.200 --> 16:26.960
presents several ways of doing object orientation in Lua.

16:26.960 --> 16:29.040
I hated all of these ways,

16:29.840 --> 16:32.720
and also the ways that I tried from the rocks.

16:34.400 --> 16:36.880
And then I wrote my own way,

16:37.360 --> 16:40.160
wrote my own way of doing object orientation in Lua.

16:40.160 --> 16:41.440
It's very minimalistic.

16:42.000 --> 16:45.120
It's in this file here, eoo.lua.

16:47.040 --> 16:49.440
The main code is just these five lines here,

16:50.640 --> 16:53.120
and here's an example of how it works.

16:56.080 --> 17:02.640
Here we define a class vector with some metamethods.

17:02.640 --> 17:07.920
This metamethod here will tell Lua what to do

17:07.920 --> 17:11.040
when the user asks to add two vectors.

17:11.600 --> 17:14.320
This one here tells Lua what to do

17:14.320 --> 17:18.320
when the user asks Lua to convert a vector to a string,

17:18.880 --> 17:20.640
and whatever.

17:20.640 --> 17:23.920
This one is something that I'm going to explain in a second.

17:25.360 --> 17:28.240
So here we create a vector with these coordinates,

17:28.240 --> 17:29.360
three and four.

17:29.360 --> 17:31.360
Here we create another vector.

17:31.360 --> 17:35.200
If we print here, then Lua uses the function here

17:35.200 --> 17:36.160
in the toString.

17:37.200 --> 17:40.880
If we add two vectors, it uses the function here

17:40.880 --> 17:42.400
in the add metamethod.

17:42.960 --> 17:45.280
And if we run the method norm,

17:45.280 --> 17:49.760
it is defined here in the table index.

17:51.280 --> 17:51.780
Anyway.

17:52.740 --> 17:54.740
Even this thing being so small,

17:54.740 --> 17:58.500
I used to forget how it's in odds worked all the time.

17:58.500 --> 18:01.700
Actually, I always forget how things work,

18:01.700 --> 18:03.940
and I have to remember them somehow.

18:03.940 --> 18:07.940
And I have to have tricks for remembering

18:07.940 --> 18:11.940
and tricks for summarizing things and diagrams and so on.

18:13.540 --> 18:15.940
And every time that I forgot how to do it,

18:15.940 --> 18:17.220
I just go back to the code.

18:17.220 --> 18:18.420
I just go back to the code.

18:18.500 --> 18:23.140
And every time that I forgot how this thing worked,

18:23.140 --> 18:25.540
I went back to the source code,

18:25.540 --> 18:27.380
and then I looked at the diagrams.

18:27.380 --> 18:29.940
Or, of course, in the first time,

18:29.940 --> 18:31.460
I had to draw the diagrams.

18:32.260 --> 18:34.420
And I run the examples.

18:34.420 --> 18:35.780
And, of course, in the beginning,

18:35.780 --> 18:37.460
I thought that the code was clear.

18:37.460 --> 18:38.900
My examples were very brief.

18:38.900 --> 18:42.820
And so I had to rewrite the examples many times

18:42.820 --> 18:46.420
until they became, let's say, perfect.

18:48.660 --> 18:53.220
And I was saying that Lua can be used in many ways.

18:53.780 --> 18:57.700
And in my way of using Lua, my favorite way,

18:57.700 --> 19:02.020
everything can be inspected and modified from REPLs,

19:03.140 --> 19:07.140
like we can do in MX and in Smalltalk, or sort of.

19:07.860 --> 19:09.940
So in my favorite way of using Lua,

19:09.940 --> 19:11.860
there's no security at all.

19:11.860 --> 19:14.740
Everything can be changed at all times.

19:15.140 --> 19:16.980
Of course, most people hate that.

19:17.780 --> 19:20.180
My init file has lots of classes.

19:20.820 --> 19:24.100
And, by the way, instead of keeping many small files

19:24.100 --> 19:25.140
with many things,

19:25.140 --> 19:29.300
I put lots of stuff in just one big init file.

19:31.060 --> 19:33.860
My init file has lots of classes

19:33.860 --> 19:36.660
and lots of global functions and lots of crafts.

19:36.660 --> 19:40.020
So people hate that, of course.

19:41.380 --> 19:42.580
This is an example.

19:43.460 --> 19:44.500
This is an example.

19:44.500 --> 19:48.260
This is the index at the top of my init file.

19:50.580 --> 19:52.820
The classes start here.

19:55.060 --> 19:57.220
And then we have some functions.

19:57.780 --> 20:02.500
And then we have functions that load certain packages.

20:02.500 --> 20:03.540
And then we have craft.

20:04.260 --> 20:04.760
Whatever.

20:06.180 --> 20:08.580
Most people think that my style of using Lua

20:08.580 --> 20:10.340
is dirty and dangerous.

20:10.340 --> 20:11.860
And they wouldn't touch my Lua code.

20:12.100 --> 20:13.140
With a 10-feet pole.

20:14.740 --> 20:18.100
But most of the things that I'm going to present here

20:18.100 --> 20:20.740
in this presentation are ideas that should be easy

20:20.740 --> 20:24.180
to port to other environments and other languages.

20:24.180 --> 20:25.780
Especially the diagrams.

20:25.780 --> 20:27.620
So the code is not so important.

20:30.740 --> 20:33.140
Now let me talk a bit about LaTeX.

20:33.140 --> 20:33.860
Lua LaTeX.

20:33.860 --> 20:38.820
That is LaTeX with the Lua interpreter embedded inside.

20:39.380 --> 20:43.300
And two ways of generating pictures in LaTeX.

20:43.300 --> 20:44.820
Text that is very famous.

20:44.820 --> 20:47.300
And picture that is not very famous.

20:47.300 --> 20:48.900
And that is very low level.

20:48.900 --> 20:51.700
And I think that not many people use.

20:53.700 --> 20:57.300
I said before that when I learned Lua,

20:57.300 --> 21:01.540
I realized that it was very good for writing little languages.

21:03.220 --> 21:05.220
I was doing my PhD at the time.

21:05.300 --> 21:11.220
And typesetting the diagrams for my PhD thesis was very boring.

21:11.220 --> 21:14.100
So one of the things that I did was that I created

21:14.980 --> 21:17.540
a little language for typesetting the diagrams for me.

21:19.140 --> 21:20.900
It was called DEDNOT.

21:20.900 --> 21:25.300
Because initially it only generated diagrams

21:25.300 --> 21:26.820
for natural deduction.

21:26.820 --> 21:28.580
And then it had several versions.

21:30.180 --> 21:34.420
These are the slides for my presentation about DEDNOT6.

21:34.420 --> 21:39.140
And DEDNOT6 is an extensible semi-preprocessor for Lua LaTeX

21:39.140 --> 21:41.780
that understands diagrams in ASCII art.

21:42.340 --> 21:48.740
In the sense that when I have a LaTeX file that has this.

21:49.380 --> 21:51.860
And when DEDNOT6 is loaded.

21:53.220 --> 21:58.820
When I give the right commands, DEDNOT6 interprets this block here

21:58.820 --> 22:00.980
as something that defines this diagram.

22:02.260 --> 22:03.540
Oops, sorry.

22:05.380 --> 22:07.540
It interprets this diagram here.

22:08.340 --> 22:10.900
This diagram in the comments here

22:11.460 --> 22:14.260
as something that defines a diagram called foo,

22:14.260 --> 22:16.100
a deduction called foo.

22:16.100 --> 22:19.140
And it generates this code here.

22:19.940 --> 22:27.300
So that we can just invoke the definition of the deduction

22:27.300 --> 22:30.260
by typing backslash DED foo.

22:31.220 --> 22:37.140
And DEDNOT6 also supports another language

22:37.140 --> 22:39.860
for typesetting bi-dimensional diagrams

22:39.860 --> 22:42.660
with arrows and stuff for category theory and blah blah.

22:44.180 --> 22:46.900
The specifications of these diagrams look like this.

22:49.540 --> 22:53.540
Here is a very good example.

22:55.540 --> 22:56.980
This is a huge diagram.

22:58.420 --> 22:59.460
Sorry, one second.

23:00.740 --> 23:03.620
So the source code that generates this diagram here

23:03.620 --> 23:05.140
is just this thing at the left.

23:07.220 --> 23:08.340
So it's very visual.

23:09.220 --> 23:12.340
We can typeset the diagram in ASCII art here.

23:12.340 --> 23:13.780
And then in this part here,

23:13.780 --> 23:16.660
we tell how the nodes are to be joined,

23:17.620 --> 23:20.660
which arrows have to have annotations and so on.

23:22.420 --> 23:25.460
And this language is extensible in the sense that...

23:25.780 --> 23:26.580
Where is that?

23:32.020 --> 23:32.520
Here.

23:34.420 --> 23:36.660
Comments that start with percent colon

23:37.940 --> 23:41.620
are interpreted as definitions for three diagrams.

23:43.620 --> 23:47.220
Lines that start with percent uppercase D

23:47.220 --> 23:52.260
define 2D diagrams with arrows and stuff.

23:52.740 --> 23:58.020
And lines that start with comment uppercase L

23:58.660 --> 24:00.580
contain blocks of Lua code

24:00.580 --> 24:04.180
that we can use to extend the interpreter on the flag.

24:06.020 --> 24:09.860
Anyway, here are some recent examples of diagrams

24:09.860 --> 24:14.580
that I used DEDNOT6 to typeset.

24:15.780 --> 24:17.220
This diagram here

24:17.780 --> 24:20.580
was generated by this specification here.

24:23.460 --> 24:26.980
And this diagram here with the curved arrows

24:27.780 --> 24:30.580
was generated by this specification here.

24:34.180 --> 24:37.220
So DEDNOT6 was very easy to extend.

24:37.220 --> 24:38.260
At that some point,

24:38.260 --> 24:42.820
I started to use it to generate diagrams using Peaked Chewy,

24:42.820 --> 24:45.540
mainly for the classes that I gave you.

24:45.540 --> 24:48.420
For the classes that I give at the university.

24:48.420 --> 24:50.900
I teach mathematics and whatever.

24:51.860 --> 24:52.900
In a bad place.

24:52.900 --> 24:53.400
Whatever.

24:56.260 --> 24:58.500
Let me show an animation.

24:59.060 --> 25:02.420
Here is a diagram that I generated with DEDNOT6.

25:02.980 --> 25:05.300
And it is a flipbook animation.

25:05.300 --> 25:08.740
Like we type page up and page down

25:08.740 --> 25:10.660
and we go to the next page of the book

25:10.660 --> 25:12.340
and to the previous page of the book.

25:13.060 --> 25:15.940
And here is the source code that generates that.

25:16.980 --> 25:19.140
This source code is not very visual.

25:19.140 --> 25:22.740
So it's quite clumsy to edit a diagram

25:22.740 --> 25:25.460
directly in the tag file like that.

25:28.020 --> 25:31.860
These diagrams were inspired by something called Manim.

25:32.500 --> 25:33.000
That's...

25:34.580 --> 25:36.340
Oh, I've forgotten the name of the guy.

25:36.340 --> 25:39.300
But it's a guy that makes many videos about mathematics.

25:39.300 --> 25:41.780
And he created this library called Manim

25:41.780 --> 25:43.460
for generating his animations.

25:45.540 --> 25:51.460
Other people adapted his library to make it more accessible.

25:52.580 --> 25:53.700
I tried to learn it.

25:53.700 --> 25:55.700
But each animation,

25:57.060 --> 25:59.220
even an animation with very few frames,

25:59.220 --> 26:01.300
each animation took ages to render.

26:01.300 --> 26:03.060
So it wasn't fun.

26:03.700 --> 26:07.060
And animations and PDFs can be rendered in seconds.

26:07.940 --> 26:11.060
So these things were fun for me

26:11.060 --> 26:15.300
because my laptop is very slow and Manim was not fun.

26:20.100 --> 26:26.020
Anyway, writing code like this inside the attack file

26:26.020 --> 26:29.620
was not very fun because it was hard to debug.

26:30.500 --> 26:35.860
So in 2022, I started to play with ways

26:35.860 --> 26:40.340
of generating these diagrams from REPLs.

26:40.980 --> 26:44.500
And I found a way for PIC2E and a way for TIX.

26:45.380 --> 26:47.780
Each one of these ways became a video.

26:48.740 --> 26:53.780
If you go to the list of first class videos of EEV,

26:53.780 --> 26:57.540
you're going to see that there's a video about PIC2E here

26:57.540 --> 26:58.740
and a video about TIX.

27:00.340 --> 27:07.540
Here you have some information like length and explanation, etc.

27:08.180 --> 27:10.420
And here are the pages for these videos.

27:12.420 --> 27:15.860
My page about the video, about PIC2E is like this.

27:15.860 --> 27:18.740
It has some diagrams, whatever.

27:18.740 --> 27:20.980
And this one is much nicer.

27:20.980 --> 27:26.100
And a lot of people watched that video.

27:26.100 --> 27:30.260
I mean, I thought that 250 people watched it.

27:30.900 --> 27:33.940
For me, that's a million of people.

27:36.660 --> 27:42.900
And this video is about how to extract diagrams from the TIX manual

27:42.900 --> 27:47.700
and how to run those examples in a REPL

27:47.700 --> 27:49.780
and modify them bit by bit.

27:49.780 --> 27:51.380
This is a screenshot.

27:52.100 --> 27:53.620
But let me go back.

27:56.420 --> 27:59.700
At that point, these things were just prototypes.

27:59.700 --> 28:01.460
The code was not very nice.

28:02.020 --> 28:10.500
And in this year, I was able to unify those two ways of generating PDFs,

28:11.380 --> 28:14.100
the one for TIX and the one for PIC2E.

28:14.100 --> 28:18.740
And I unified them with many other things that generated diagrams.

28:19.220 --> 28:27.460
The basis of these things is something called showchew.lua.

28:27.460 --> 28:31.380
I'm not going to show its details now.

28:32.100 --> 28:40.420
But its extension that generates TIX code is just this.

28:40.420 --> 28:45.540
So we can specify a diagram with just a block like this.

28:46.260 --> 28:54.180
And then if we run show00 and it returns a string

28:54.180 --> 28:58.580
that is just the inner body of the TIX file.

28:59.220 --> 29:02.020
If we run this, we see the whole TIX file.

29:02.020 --> 29:04.340
And if we run this, we save the TIX file

29:04.340 --> 29:08.420
and we compile the TIX file to generate a PDF.

29:08.420 --> 29:14.180
And if we run this, we show the PDF in the lower right window.

29:16.020 --> 29:20.740
And that's the same thing for all my recent programs that generate PDFs.

29:21.620 --> 29:22.980
They are all integrated.

29:24.500 --> 29:30.660
Here is the one that the basis for all my modules that generate diagrams with PIC2E.

29:33.060 --> 29:34.980
Its demos are not very interesting.

29:34.980 --> 29:39.700
So let me show some demos of extensions that do interesting things.

29:40.660 --> 29:47.220
So this is a diagram that I created by editing it in a REPL.

29:48.820 --> 29:51.220
I create several PIC2E objects here.

29:52.500 --> 29:58.660
And if I execute this, it compiles the object, generates a PDF.

29:58.660 --> 30:03.860
And if I tap this, here is the PDF.

30:04.740 --> 30:10.020
And if I just ask Lua to display what is books here,

30:11.700 --> 30:17.220
it shows the source code in PIC2E of the diagram.

30:17.940 --> 30:20.340
And the nice thing is that it is indented.

30:20.340 --> 30:23.460
So it's easy to debug the PIC2E code.

30:24.340 --> 30:30.180
If anyone is interested, the module that does the tricks for indentation

30:30.180 --> 30:31.460
is very easy to understand.

30:31.540 --> 30:34.500
It has lots of tests and test blocks.

30:34.500 --> 30:39.060
And I think that its data structures are easy to understand.

30:42.500 --> 30:44.820
Anyway, here is another example.

30:50.260 --> 30:51.620
The show is here.

30:53.460 --> 30:56.340
It generates a 3D diagram.

31:02.420 --> 31:08.500
Now let me talk about parsers and REPLs in a very strange place.

31:09.140 --> 31:18.340
I mean, using REPLs to build parsers step by step and replacing parts by more complex parts.

31:21.060 --> 31:23.780
So I said that Lua is very minimalistic.

31:25.780 --> 31:30.660
And everybody knows that implementations of regular expressions

31:30.740 --> 31:31.780
are big and complex.

31:33.060 --> 31:39.220
So instead of coming with full regular expressions, Lua comes with something called patterns

31:39.220 --> 31:43.620
and a library function called string.match.

31:45.380 --> 31:54.420
Here is a copy of the part of the manual that explains a part of the syntax of patterns.

31:56.260 --> 32:00.100
Here's how string.match is described in the manual.

32:01.220 --> 32:02.500
It's just this.

32:02.500 --> 32:06.260
Looks for the first match of pattern in the string as blah, blah, blah.

32:07.220 --> 32:11.300
And then we have to go to the other section of the manual that explains patterns.

32:17.780 --> 32:26.500
Lua patterns are so simple, so limited, that they don't even have the alternation operator.

32:27.220 --> 32:31.460
Here is how it is described in the ELLISP manual.

32:33.300 --> 32:37.780
Backslash pipe specifies an alternative, blah, blah, blah.

32:40.820 --> 32:47.940
When we want to build more complex regular expressions, patterns, grammars, etc.,

32:47.940 --> 32:51.780
we have to use an external library for that.

32:52.740 --> 32:59.140
No, sorry, a library that is external, but that was written by one of the authors of Lua itself.

33:00.340 --> 33:05.380
This library is called LPEG, and its manual says,

33:06.180 --> 33:11.940
LPEG is a new pattern matching library for Lua based on parsing expression grammars, PEGs.

33:14.900 --> 33:17.460
The manual is very terse.

33:18.180 --> 33:20.500
I found it incredibly hard to read.

33:20.900 --> 33:22.740
It doesn't have any diagrams.

33:22.740 --> 33:24.420
It has some examples, though.

33:26.180 --> 33:34.420
And the Lua wiki has a big page called LPEG tutorial with lots of examples.

33:35.700 --> 33:42.020
But it also doesn't have diagrams, and I found some things incredibly hard to understand.

33:42.020 --> 33:48.500
For example, this is something that is in the manual of LPEG that I saw and I thought,

33:49.460 --> 33:53.060
wow, great, this makes all sense, and this is going to be very useful.

33:53.060 --> 33:57.300
It's a way to build grammars that can be recursive,

33:57.300 --> 34:02.020
and they sort of can encode BNF grammars.

34:02.020 --> 34:06.980
We just have to translate the BNF a bit to get rid of some recursions

34:06.980 --> 34:08.820
and to translate them to something else.

34:10.580 --> 34:13.620
And the manual also has some things that I thought,

34:13.620 --> 34:16.580
oh, no, I don't have any idea of what this thing does.

34:17.380 --> 34:22.020
And in fact, I saw these things for the first time more than 10 years ago,

34:22.020 --> 34:26.020
and they only started to make sense one year ago.

34:28.420 --> 34:30.340
One example is group captures.

34:33.380 --> 34:37.620
LPEG also comes with a module called the RE module.

34:37.620 --> 34:40.180
Let me pronounce it in Portuguese, the RE module.

34:40.660 --> 34:45.540
And its manual says, the RE module provided by the file repo.luen

34:45.540 --> 34:48.100
in the distribution supports a somewhat conventional

34:48.980 --> 34:54.420
regular expression syntax for pattern usage within LPEG.

34:55.140 --> 34:57.460
And this is a quick reference.

35:00.500 --> 35:02.820
And this thing is very brief.

35:02.820 --> 35:06.420
It has some nice examples, but it's hard to understand in a way.

35:06.660 --> 35:12.500
And here are some comments about my attempts to learn HerdotLua.

35:14.020 --> 35:15.220
This is a class.

35:15.220 --> 35:17.060
In this case, it's a very small class.

35:18.180 --> 35:21.940
And this file implements a PM method.

35:23.540 --> 35:26.980
I'm going to show examples of other PM methods very soon.

35:28.020 --> 35:33.380
So this is a PM method for HerdotLua that lets us compare the syntax

35:33.620 --> 35:39.460
of Lua patterns, LPEG, and HerdotLua.

35:41.620 --> 35:43.060
See this example here.

35:43.700 --> 35:49.940
So if we run this, it loads my version of LPEG.

35:50.660 --> 35:52.740
No, sorry, my version of LPEG-REX.

35:54.820 --> 36:00.180
And it shows that when we apply the PM method to this Lua pattern,

36:00.740 --> 36:07.380
this LPEG pattern, and this RE pattern, they all get the same results.

36:07.380 --> 36:12.820
So we can use this thing, this kind of thing here to show how to translate from

36:14.420 --> 36:20.180
Lua patterns that are familiar because they are similar to regular expressions, only weaker,

36:22.260 --> 36:27.620
to LPEG that is super weird, and to RE that is not so weird.

36:28.580 --> 36:37.940
Anyway, the comment says that in 2012, I had a project that needed a precedence parser that

36:37.940 --> 36:42.900
could parse arithmetical expressions with the right precedences.

36:44.580 --> 36:50.180
And at that point, I was still struggling with pure LPEG, and I couldn't do much with it.

36:50.180 --> 36:55.300
So I tried to learn HerdotLua instead, and I wrote this old class here.

36:56.260 --> 36:59.620
That allowed me to use preprocessor on patterns for Lua.

36:59.620 --> 37:04.980
And the thing is that with this preprocessor, I could specify precedence grammars using this

37:04.980 --> 37:12.900
thing here that worked, but it was super clumsy, and I gave up after a few attempts.

37:14.900 --> 37:21.300
And in 2022, I heard about something called LPEG-REX that was an

37:22.260 --> 37:28.740
a kind of extension of Re, and it was much more powerful than HerdotLua, but after a while,

37:28.740 --> 37:32.660
I realized that it had the same defects as HerdotLua.

37:32.660 --> 37:41.620
And let me explain that because it has all to do with the things about black boxes and magic

37:41.620 --> 37:42.980
that I told in the beginning.

37:43.380 --> 37:50.100
Both, I mean, sorry, neither HerdotLua or LPEG-REX had some features that I needed.

37:51.620 --> 37:57.940
They didn't let us explain, sorry, they received a pattern that was specified as a string,

37:57.940 --> 38:04.740
and it converted that into an LPEG pattern, but it didn't let us explore the LPEG patterns

38:04.740 --> 38:05.540
that it was using.

38:05.540 --> 38:12.900
So I had to use the LPEG-REX, and it didn't let me explore the LPEG patterns that it was

38:12.900 --> 38:13.540
generated.

38:16.420 --> 38:21.060
Their code was written in a way that was REPL unfriendly.

38:21.060 --> 38:28.580
I couldn't modify parts of the code bit by bit in a REPL and try to change the code

38:29.700 --> 38:31.620
without changing the original file, say.

38:33.300 --> 38:37.220
The code was very hard to explore, to hack, and to extend, in my opinion.

38:37.780 --> 38:39.700
The documentation was not very clear.

38:40.580 --> 38:50.660
And I sent one or two messages to the developer of LPEG-REX, and he was too busy to help me.

38:50.660 --> 38:55.460
He answered very briefly, and to be honest, I felt rejected.

38:55.460 --> 38:58.580
I felt that I wasn't doing anything interesting.

38:58.580 --> 38:59.380
Whatever, whatever.

39:00.340 --> 39:11.940
So, in 2022, I was trying to learn LPEG-REX, because I was thinking that it would solve

39:11.940 --> 39:13.780
my problems, but it didn't.

39:14.500 --> 39:20.900
It didn't have the features that I needed, and it was hard to extend, and hard to explore,

39:20.900 --> 39:22.100
and hard to debug.

39:22.980 --> 39:32.420
I decided to rewrite it in a more hacker-friendly way, in the sense that it was modular, and

39:32.420 --> 39:35.460
I could replace any part of the module from a REPL.

39:37.300 --> 39:48.340
My version of it was called lpeg1.lua, and I decided that in my version, I wouldn't have

39:49.060 --> 39:55.460
the part that receives a grammar specified as a string and converts that to LPEG.

39:55.460 --> 40:03.780
I would just have the backend part that are the functions in LPEG that let us specify

40:04.340 --> 40:05.380
powerful grammars.

40:10.340 --> 40:11.540
So, let me go back.

40:12.260 --> 40:14.260
Let me explain a bit about LPEG.

40:14.900 --> 40:16.180
Lua has coercions.

40:18.660 --> 40:22.740
The plus expects to receive two numbers.

40:22.740 --> 40:29.860
If one of its arguments, or both of them, are strings, it converts the strings to numbers.

40:29.860 --> 40:39.220
So, in this case here, 2 plus string 3 returns the number 5, and this is the concatenation

40:39.220 --> 40:39.860
operator.

40:40.820 --> 40:42.820
It expects to receive strings.

40:43.780 --> 40:49.860
So, in this case, it will convert the number 2 to the string 2, and the concatenation of

40:49.860 --> 40:51.700
these two things will be 23.

40:52.420 --> 40:54.420
Sorry, 23 as a string.

40:56.260 --> 40:58.420
LPEG also has some coercions.

41:00.260 --> 41:09.620
I usually set these globals to let me write my grammars in a very compact way.

41:10.260 --> 41:17.460
So, instead of lpeg.p, lpeg.c, etc., I use these globals like uppercase B, uppercase

41:17.460 --> 41:18.420
C, and so on.

41:19.540 --> 41:27.540
And with these globals, I can write things like this, c1 times string underscore.

41:28.420 --> 41:40.820
And LPEG knows that lpeg.c, sorry, it sort of expands these to lpeg.c, but lpeg.c expects

41:40.820 --> 41:47.380
to receive an LPEG pattern, and one is not yet an LPEG pattern, so it is coerced into

41:47.380 --> 41:50.900
an LPEG pattern by calling lpeg.p.

41:51.220 --> 42:02.740
So, this shorting here becomes equivalent to lpeg.c, lpeg.p1, and the multiplication,

42:02.740 --> 42:09.940
when at least one of its arguments is an LPEG pattern, it expects to receive two LPEG patterns,

42:09.940 --> 42:15.700
and in this case, the one at the right is just a string, so it is coerced to an LPEG

42:15.700 --> 42:17.620
pattern by using lpeg.p.

42:18.180 --> 42:22.180
With this idea, we can sort of understand this comparison here.

42:22.900 --> 42:24.980
I mean, let me run it again.

42:24.980 --> 42:33.620
This first part is very similar to a regular expression here at the left, and when we apply

42:33.620 --> 42:45.940
this LPEG, sorry, this Lua pattern to this subject here, the result is that the Lua pattern

42:46.740 --> 42:53.540
the result is this thing here, this thing, this thing, and this thing.

42:53.540 --> 43:00.580
I'm going to call each one of these results captures, so each of these things between

43:00.580 --> 43:08.020
parentheses captures a substring of the original string, and these captured substrings are

43:08.020 --> 43:09.460
returned in a certain order.

43:10.420 --> 43:12.740
Here is how to express the same thing in LPEG.

43:13.540 --> 43:22.660
It's very cryptic, but it's a good way to understand some basic operators of LPEG.

43:22.660 --> 43:34.900
I mean, we can look at the menu and understand what C, S, and R do, and also exponentiation.

43:35.860 --> 43:43.060
And this strange thing here receives this string here, runs a function that I have defined

43:43.060 --> 43:50.260
that converts it to an object of a certain class, and that class represents He patterns.

43:50.260 --> 43:56.900
So this thing is treated as a pattern for He.Lua, and it is matched against the string,

43:56.900 --> 43:59.300
and it returns the same thing as the other one.

43:59.860 --> 44:06.260
Also, this thing here also has a comparison with LPEG-REGS, but these patterns are very

44:06.260 --> 44:06.820
trivial.

44:06.820 --> 44:10.180
They don't do anything very strange.

44:10.180 --> 44:14.260
So let's go back and see what kinds of very strange things there are.

44:16.660 --> 44:23.620
Here is the page of LPEG-REGS at GitHub.

44:24.260 --> 44:25.860
Here's the documentation.

44:27.940 --> 44:29.380
It's relatively brief.

44:29.380 --> 44:35.300
It explains LPEG-REGS as being an extension of He.Lua.

44:36.500 --> 44:40.100
So it explains mainly the additional features.

44:40.100 --> 44:44.180
Here is a quick reference that explains only the additional features.

44:46.820 --> 44:53.380
Some of these things I was able to understand by using the LPEG-REGS.

44:54.340 --> 45:02.820
I was struggling a lot, and some I wasn't able to, even by spending several evenings

45:02.820 --> 45:04.420
trying to build examples.

45:08.260 --> 45:10.900
And this is something very nice.

45:12.100 --> 45:19.700
LPEG-REGS comes with some example parsers, and here is a parser that parses the Lua grammar.

45:19.700 --> 45:28.100
I mean, this is the grammar for Lua 5.4 at the end of the reference manual.

45:28.660 --> 45:35.860
It's just this, and this is a kind of the BNF, and this is the BNF translated to

45:37.300 --> 45:39.860
the language of LPEG-REGS.

45:40.420 --> 45:47.300
So this thing uses many constructions that are in He.Lua and some extra constructions

45:47.300 --> 45:49.460
that are described here.

45:50.260 --> 45:58.980
And with these examples, I was able to understand some of these things here that are described

45:58.980 --> 46:02.660
here in the quick reference, but not all.

46:06.660 --> 46:15.300
So I wasn't able to use LPEG-REGS by itself because some things didn't make much sense,

46:15.300 --> 46:18.900
and I decided to reimplement it in my own style.

46:20.900 --> 46:27.860
Because that would be a way to map, at the very least, map what I understood and what

46:27.860 --> 46:32.980
I didn't, and learn one feature at a time, do comparisons, and so on.

46:35.380 --> 46:38.900
Here, I pointed to two features of LPEG.

46:38.900 --> 46:44.820
One, I said, oh great, this thing can be used to define grammars, even recursive

46:44.900 --> 46:46.020
grammars, and so on.

46:46.740 --> 46:52.820
And this is an oh-no feature, one thing that didn't make any sense at all, group captures.

46:55.380 --> 47:01.620
One thing that I did to understand group captures was to represent them as diagrams.

47:01.620 --> 47:08.020
Of course, in the beginning, I was drawing these diagrams by hand, but then I realized

47:08.020 --> 47:15.220
that I could use the bits of LPEG that I already knew to build a grammar that would

47:15.220 --> 47:19.380
parse a little language and generate these diagrams in LaTeX.

47:20.340 --> 47:21.940
And I was able to make this.

47:24.180 --> 47:34.500
In this diagram here, this thing above the arrow is a piece of Lua code that specifies

47:34.500 --> 47:36.100
an LPEG pattern.

47:36.820 --> 47:43.540
This thing here at the top is the string that is being matched, and the things below the

47:43.540 --> 47:51.220
under braces are the captures that each thing, sorry, each thing captures.

47:54.580 --> 48:02.100
And for example, this under brace here corresponds to this pattern here that parses a single

48:02.100 --> 48:05.060
character but doesn't return any captures.

48:05.140 --> 48:10.420
This thing here parses a single B and doesn't return any captures.

48:10.420 --> 48:16.740
This thing here parses a single character and captures it, and this thing here parses

48:17.620 --> 48:19.700
the character D and captures it.

48:20.340 --> 48:29.220
And this other thing here that transforms this pattern into another pattern returns

48:29.780 --> 48:36.500
first a capture with all the string that was parsed by this pattern here, and then all

48:36.500 --> 48:41.380
the captures returned by this thing here before the column.

48:43.380 --> 48:50.900
So this was a way to build concrete examples for things that the LPEG manual was explaining

48:50.900 --> 48:53.940
in a very terse way, and it worked for me.

48:54.740 --> 49:01.540
Some things that were very mysterious started to make sense, and I started to have intelligent

49:01.540 --> 49:03.300
questions to ask in the mailing list.

49:07.700 --> 49:16.980
And with that, I was able to understand what are group captures and group captures that

49:16.980 --> 49:17.860
receive a name.

49:20.900 --> 49:23.140
Well, let me explain what this does.

49:23.220 --> 49:30.180
This thing here captures, sorry, parses the empty string and returns this as a constant.

49:30.180 --> 49:35.860
So this is something that doesn't exist in regular expressions.

49:36.980 --> 49:40.180
It parses nothing and returns this as a capture.

49:40.820 --> 49:48.580
Then this thing here returns these two constants here and parses the empty string, and this

49:48.660 --> 49:56.500
thing here, d, converts the results of this thing here into a group capture and stores

49:56.500 --> 49:58.020
it in the label d.

50:01.380 --> 50:03.860
And then here's another constant capture.

50:03.860 --> 50:11.460
And I realized that these things here were similar to how Lua specifies building lists.

50:11.780 --> 50:16.580
When we build, sorry, a table, when we build a table and we say that the first element

50:16.580 --> 50:20.420
of the table is here, this element is put at the end of the table.

50:21.140 --> 50:30.820
When after that we say d equals to, say, 42, we are putting the 42 in the slot whose key

50:30.820 --> 50:31.380
is d.

50:33.380 --> 50:37.860
This was happening with LPEG captures, but there was something very strange.

50:37.860 --> 50:46.260
These group captures could hold more than one capture, more than one value.

50:46.260 --> 50:50.340
So there was something between lists and tables.

50:51.300 --> 51:00.740
I started to use this notation to explain in my notation what they were doing.

51:02.340 --> 51:04.900
Many things start, things stop.

51:05.220 --> 51:08.900
Many things start, things started to make sense.

51:08.900 --> 51:14.420
Many mysterious sentences in the manual started to make sense, but some didn't.

51:16.100 --> 51:25.060
But at least I was able to send some intelligent questions to the mailing list, and the author

51:25.060 --> 51:27.540
of Lua and LPEG answered some of them.

51:29.700 --> 51:31.860
He was not very happy about my questions.

51:32.020 --> 51:40.100
He told me that those diagrams were a waste of time, that the manual was perfectly clear,

51:40.100 --> 51:42.900
and so on, whatever.

51:42.900 --> 51:51.620
But I was able to, so it was weird, but I was able to understand lots of things from

51:51.620 --> 51:52.340
his answers.

51:53.940 --> 51:57.380
So this is a copy of one of my messages.

51:57.380 --> 51:59.300
Then there's another one, another one.

51:59.300 --> 52:04.580
Some were the diagrams, then he complained about these diagrams.

52:04.580 --> 52:12.980
He said that these things here that look like table constructions do not exist, whatever.

52:15.380 --> 52:23.220
Anyway, once I understood group captures, many features were very, very easy to understand,

52:23.780 --> 52:28.260
and I started to be able to use LPEG to build some very interesting things.

52:30.260 --> 52:36.340
I was able to reproduce some of the features that I saw in LPEG-REX.

52:38.420 --> 52:40.980
Remember that this, where is that?

52:42.420 --> 52:44.020
This is a syntax of Lua.

52:45.860 --> 52:54.820
Here, I was able to understand how these things here were translated to LPEG code, to LPEG

52:54.820 --> 52:58.340
patterns by using group captures in a certain way.

52:59.460 --> 53:05.060
And I was able to implement them in lpeg1.lua.

53:06.340 --> 53:14.180
And after some time, I was able to use lpeg1.lua to build grammars that were able to parse

53:17.300 --> 53:19.860
arithmetical expressions with the right precedence.

53:19.860 --> 53:25.700
And here's an example in which I built the grammar step by step, and I test the current

53:25.700 --> 53:29.300
grammar, and I replace a bit, and then I test the new grammar, and so on.

53:31.780 --> 53:38.580
And you can see that the result is always a tree that is drawn in a nice two-dimensional way.

53:42.580 --> 53:52.340
At this point, these powers here are returned as a list, as an operation

53:52.900 --> 53:56.260
pow with several arguments here.

53:56.820 --> 54:04.820
And then I apply a kind of parsing combinator here that transforms these trees into other trees.

54:05.540 --> 54:14.180
And with these combinators here, I can specify that the power is associative in a certain

54:14.180 --> 54:20.340
direction, the division is associative in another direction, the minus is associative,

54:20.420 --> 54:24.420
so it uses the same direction as a division, and so on.

54:24.420 --> 54:26.020
And they have the right precedences.

54:28.660 --> 54:30.020
So here are the tests.

54:33.700 --> 54:37.140
So here's my file lpeg1.lua.

54:37.140 --> 54:38.340
It has several classes.

54:38.980 --> 54:42.980
Each class has tests after it.

54:43.940 --> 54:51.780
I was able to implement something that lpeg-reqs has that's called keywords.

54:51.780 --> 54:56.980
That's very useful for parsing programs in programming languages.

54:56.980 --> 55:06.580
I was able to implement something similar to the debugger, to the PEG debugger that lpeg uses,

55:07.220 --> 55:12.740
but I was frustrated by some limitations of that debugger,

55:12.740 --> 55:16.180
and I implemented my own that is much better.

55:17.700 --> 55:23.860
And let me show something else.

55:23.860 --> 55:32.900
I was able to translate a good part of the Lua parser here to lpeg1.lua.

55:33.700 --> 55:39.940
I haven't finished yet, but I have most of the translation here.

55:43.140 --> 55:50.420
And after having all that, I was able to build other grammars very quickly.

55:51.380 --> 55:59.220
Writing new parsers finally became fun, and here's one example that I showed in the beginning.

56:03.620 --> 56:11.220
If I remember correctly, I took a figure from the Wikipedia.

56:11.220 --> 56:17.220
I don't have its link now, but I specified a grammar that parses

56:18.500 --> 56:21.140
exactly the example that appears in the Wikipedia.

56:21.140 --> 56:28.100
So with my grammar, considering that the top-level entry is statement,

56:28.100 --> 56:36.900
when I parse this string here, the result is this tree.

56:38.100 --> 56:41.780
And I can do some operations on that.

56:41.780 --> 56:45.780
I can define how this thing is to be converted into LaTeX.

56:47.460 --> 56:51.700
I can define other operations that convert trees into other trees.

56:52.580 --> 56:54.980
And here are some tests of these operations.

56:58.340 --> 57:00.660
This is what I showed in the beginning.

57:00.660 --> 57:04.100
I'm not going to explain all the details of this thing now.

57:07.460 --> 57:14.260
This show converts this thing into LaTeX in the way specified by these instructions here.

57:15.380 --> 57:16.340
Let's say that...

57:19.540 --> 57:20.260
Well, whatever.

57:22.740 --> 57:23.540
Really, whatever.

57:24.020 --> 57:28.500
And here's the result, the LaTeX result.

57:34.900 --> 57:46.260
And these diagrams here are generated by this file here that defines a simple grammar that parses

57:46.260 --> 57:53.460
this thing here, and then LaTeX it in a certain way, and then also tests this thing.

57:54.100 --> 58:00.900
To check if this code here, that is Lua code that generates an LPEG grammar,

58:02.900 --> 58:08.740
parses this subject here and returns the expected result.

58:12.740 --> 58:15.140
So this is the code that I wanted to show.

58:15.700 --> 58:21.700
I wanted to show many more things, but I wasn't able to prepare them before the conference.

58:22.580 --> 58:29.060
And I hope that soon, for some value of soon, I'll be able to create

58:29.620 --> 58:37.060
REPL-based tutorials for LPEG here and lpeg1.lua, where LPEG is something very famous.

58:37.060 --> 58:39.460
Here is a module of LPEG.

58:40.500 --> 58:48.100
I could also do something like this for LPEG-REX, and lpeg1.lua is the thing that I wrote,

58:48.820 --> 58:58.900
the one that has tests in comments, and the tests usually generate trees,

58:58.900 --> 59:00.900
and sometimes they generate tag code.

59:03.300 --> 59:04.420
Yeah, so that's it.

59:04.420 --> 59:07.860
I wanted to present much more, but I wasn't able to prepare it.

59:07.860 --> 59:08.660
So sorry.

59:08.660 --> 59:09.140
Thanks.

59:09.140 --> 59:10.260
Bye.