path: root/2023/captions/emacsconf-2023-repl--repls-in-strange-places-lua-latex-lpeg-lpegrex-tikz--eduardo-ochs--main.vtt

WEBVTT captioned by eduardo
Kind: captions
Language: en-GB

NOTE Intro

00:00:00.000 --> 00:00:03.079
Hi! My name is Eduardo Ochs

00:00:03.080 --> 00:00:05.519
and the title of this talk is: REPLs

00:00:05.520 --> 00:00:09.399
in strange places - Lua, LateX, LPeg, LPegRex,

00:00:09.400 --> 00:00:12.119
and TikZ. I'm the author of an Emacs

00:00:12.120 --> 00:00:14.879
package called eev, and this is a talk

00:00:14.880 --> 00:00:18.479
at the EmacsConf 2023, that is happening in

00:00:18.480 --> 00:00:22.519
December 2023, at the internets.

NOTE Diagrams

00:00:22.520 --> 00:00:23.079
This is one of the

00:00:23.080 --> 00:00:24.919
examples of diagrams that we are

00:00:24.920 --> 00:00:27.639
going to see - let me show how I generate

00:00:27.640 --> 00:00:30.599
it... one second,

00:00:30.600 --> 00:00:35.799
I have to use a smaller font here...

00:00:35.800 --> 00:00:39.159
this is a file called ParseTree2.lua...

00:00:39.160 --> 00:00:42.919
let me go back to this block of tests again...

00:00:42.920 --> 00:00:43.879
and now if I run

00:00:43.880 --> 00:00:47.759
this...

00:00:47.760 --> 00:00:50.159
we get these outputs here at the

00:00:50.160 --> 00:00:53.959
right, and then in this line here it

00:00:53.960 --> 00:00:58.119
generates a PDF, and if I type f8 here it

00:00:58.120 --> 00:01:03.919
shows the PDF in the lower right window.

NOTE eev

00:01:03.920 --> 00:01:06.079
Let me start by explaining

00:01:06.080 --> 00:01:09.839
briefly what is eev.

00:01:09.840 --> 00:01:12.239
First: it is something that

00:01:12.240 --> 00:01:14.759
appeared by accident in the mid-90s - I

00:01:14.760 --> 00:01:16.799
explained this story in my

00:01:16.800 --> 00:01:20.159
presentation at the EmacsConf 2019...

00:01:20.160 --> 00:01:23.159
it's a package... it's an Emacs

00:01:23.160 --> 00:01:26.519
package that is part of ELPA... it has at

00:01:26.520 --> 00:01:29.679
least 10 users - those are the ones

00:01:29.680 --> 00:01:32.559
that I know by name...

00:01:32.560 --> 00:01:38.439
eev means `emacs-execute-verbosely'...

00:01:38.440 --> 00:01:40.959
eev is something that treats eval-last-sexp

00:01:40.960 --> 00:01:43.999
as the central feature of Emacs...

00:01:44.000 --> 00:01:45.599
eev blurs the distinction between

00:01:45.600 --> 00:01:48.919
programmers and users, and it replaces

00:01:48.920 --> 00:01:51.679
the slogan "users should not be forced to

00:01:51.680 --> 00:01:53.679
see Lisp", that is something that Richard

00:01:53.680 --> 00:01:57.559
Stallman told me once, by "users should see

00:01:57.560 --> 00:02:00.479
Lisp instead of buttons" and "new users

00:02:00.480 --> 00:02:03.999
should see Lisp in the first 5 minutes"...

00:02:04.000 --> 00:02:05.279
I'm going to show

00:02:05.280 --> 00:02:08.959
some examples of that soon.

00:02:08.960 --> 00:02:11.959
Eev uses code in comments a lot,

00:02:11.960 --> 00:02:15.119
and also tests in comments...

00:02:15.120 --> 00:02:17.679
I changed my way of presenting it

00:02:17.680 --> 00:02:19.679
and it became very REPL-centric

00:02:19.680 --> 00:02:22.319
in the last few years, in the

00:02:22.320 --> 00:02:24.399
sense that I start by explaining its

00:02:24.400 --> 00:02:28.679
main features by its support for REPLs...

00:02:28.680 --> 00:02:31.079
eev supposes

00:02:31.080 --> 00:02:32.639
that we want to keep

00:02:32.640 --> 00:02:35.199
executable notes of everything - I'm also

00:02:35.200 --> 00:02:37.159
going to show examples of this in a

00:02:37.160 --> 00:02:40.479
second... eev has lots of "videos for

00:02:40.480 --> 00:02:43.559
people who hate videos", and it tries to

00:02:43.560 --> 00:02:46.159
do everything with very little magic and

00:02:46.160 --> 00:02:48.839
without black boxes - I'm going to explain

00:02:48.840 --> 00:02:50.319
many of these things very soon.

NOTE Another figure

00:02:50.320 --> 00:02:52.799
This is a figure that that I'm going

00:02:52.800 --> 00:02:57.119
to show in details soon, that is

00:02:57.120 --> 00:02:59.959
about something important about Lua...

00:02:59.960 --> 00:03:03.959
the font is very bad now, so let me

00:03:03.960 --> 00:03:07.559
change the font... the figure is this one...

00:03:07.560 --> 00:03:08.239
and...

00:03:08.240 --> 00:03:12.519
what most people do when they

00:03:12.520 --> 00:03:14.479
visit a file with something

00:03:14.480 --> 00:03:16.479
interesting on it is that they just go

00:03:16.480 --> 00:03:18.679
there and they set a bookmark there, or

00:03:18.680 --> 00:03:21.959
they put the position in a register...

00:03:21.960 --> 00:03:26.959
but I prefer to keep

00:03:26.960 --> 00:03:29.199
links to everything that is interesting

00:03:29.200 --> 00:03:32.119
as elisp hyperlinks. So, for example, this is

00:03:32.120 --> 00:03:35.079
an elisp hyperlink to a file, that goes

00:03:35.080 --> 00:03:37.599
to this anchor here, and to this string

00:03:37.600 --> 00:03:41.439
after this anchor... this is a variant

00:03:41.440 --> 00:03:44.239
that opens that file in the window

00:03:44.240 --> 00:03:45.479
at the right -

00:03:45.480 --> 00:03:48.879
here... and this is

00:03:48.880 --> 00:03:53.279
a sexp that changes the font. I

00:03:53.280 --> 00:03:56.679
have a command with a very short name

00:03:56.680 --> 00:03:59.439
that does that, but I

00:03:59.440 --> 00:04:02.959
prefer to keep that as a one-liner.

00:04:02.960 --> 00:04:06.919
About the videos... we can see

00:04:06.920 --> 00:04:10.079
the list of first-class videos of eev

00:04:10.080 --> 00:04:14.759
by executing this, M-x find-1stclassvideos,

00:04:14.760 --> 00:04:18.639
or by running this alias here, M-x 1c...

00:04:18.640 --> 00:04:20.679
and then what we see is this...

00:04:20.680 --> 00:04:24.239
the first sexp here

00:04:24.240 --> 00:04:26.959
regenerates this buffer - so we can make a

00:04:26.960 --> 00:04:29.399
mess here and then run this and the

00:04:29.400 --> 00:04:33.519
original buffer is regenerated again in

00:04:33.520 --> 00:04:34.959
a clean way...

00:04:34.960 --> 00:04:36.919
each of these things here

00:04:36.920 --> 00:04:40.879
opens a buffer with information about

00:04:40.880 --> 00:04:43.999
a video... let me take a specific

00:04:44.000 --> 00:04:49.159
example here... this video here is about

00:04:49.160 --> 00:04:51.039
one of the ancestors of this talk, that

00:04:51.040 --> 00:04:53.479
is a library that I wrote

00:04:53.480 --> 00:04:58.479
for creating diagrams in LaTeX using

00:04:58.480 --> 00:05:03.519
a package called Pict2e using REPLs...

00:05:03.520 --> 00:05:03.959
anyway...

00:05:03.960 --> 00:05:06.599
the thing is that if we

00:05:06.600 --> 00:05:11.599
run a sexp like this one and we don't

00:05:11.600 --> 00:05:14.679
have a local copy of the video eev

00:05:14.680 --> 00:05:17.599
will try to download to the local copy -

00:05:17.600 --> 00:05:20.119
and instead of doing that by asking

00:05:20.120 --> 00:05:21.719
something like "do you want me

00:05:21.720 --> 00:05:23.999
to download the local copy? Blah

00:05:24.000 --> 00:05:27.999
blah blah blah blah..." it simply opens a

00:05:28.000 --> 00:05:30.719
buffer like this, I mean, if we don't

00:05:30.720 --> 00:05:33.359
have a local copy yet it will open a

00:05:33.360 --> 00:05:36.359
buffer like this one, in which these

00:05:36.360 --> 00:05:39.199
things here in comments are links to the

00:05:39.200 --> 00:05:43.839
documentation... I mean, this thing here

00:05:43.840 --> 00:05:46.159
explains the idea of local copies

00:05:46.160 --> 00:05:48.759
of files from the internet...

00:05:48.760 --> 00:05:54.759
there are more details here, and here...

00:05:54.760 --> 00:05:57.159
and this is a script that we

00:05:57.160 --> 00:06:00.519
can execute line by line, so instead of

00:06:00.520 --> 00:06:02.359
this script being hidden behind the

00:06:02.360 --> 00:06:06.119
button that we just press after a

00:06:06.120 --> 00:06:08.399
question like "Do you want me to do

00:06:08.400 --> 00:06:10.239
something blah blah blah? Yes or no?"

00:06:10.240 --> 00:06:13.639
the script is visible here and we can

00:06:13.640 --> 00:06:17.599
execute it step by step... it creates a

00:06:17.600 --> 00:06:20.799
terminal with a shell here in the

00:06:20.800 --> 00:06:24.799
right window, and when we type f8 in

00:06:24.800 --> 00:06:26.839
one of these lines here the lines are

00:06:26.840 --> 00:06:29.119
sent... (...) so this is going

00:06:29.120 --> 00:06:31.759
to download a copy of the video... the

00:06:31.760 --> 00:06:36.639
wget says that I already have a copy of

00:06:36.640 --> 00:06:39.479
the video and its subtitles... and so on.

00:06:39.480 --> 00:06:43.879
And after getting a copy of the video

00:06:43.880 --> 00:06:48.159
we can run this sexp here and it displays

00:06:48.160 --> 00:06:49.919
the video.

00:06:49.920 --> 00:06:55.399
I said that eev has lots of

00:06:55.400 --> 00:06:58.399
"videos for people who hate videos", and

00:06:58.400 --> 00:06:59.679
the idea is that very few

00:06:59.680 --> 00:07:02.039
people are going to watch the videos in

00:07:02.040 --> 00:07:06.559
real time... and most of the people that

00:07:06.560 --> 00:07:08.279
I know - or: most of the people that

00:07:08.280 --> 00:07:10.639
are interested in eev in some

00:07:10.640 --> 00:07:13.479
way... they are going to watch just

00:07:13.480 --> 00:07:16.319
small sections of the video, and most of

00:07:16.320 --> 00:07:17.559
the time they're just going to read the

00:07:17.560 --> 00:07:20.799
subtitles of the video. So, for each

00:07:20.800 --> 00:07:23.479
one of the videos we have a page

00:07:23.480 --> 00:07:27.039
about the video... let me see if I

00:07:27.040 --> 00:07:29.999
have internet here... yes. This is a

00:07:30.000 --> 00:07:32.879
page...

00:07:32.880 --> 00:07:38.999
and usually these pages have a link

00:07:39.000 --> 00:07:40.759
to another page that

00:07:40.760 --> 00:07:43.239
has all the subtitles of the

00:07:43.240 --> 00:07:46.639
video... uh, wherever... in this one

00:07:46.640 --> 00:07:48.319
it's not so visible...

00:07:48.320 --> 00:07:50.599
but anyway, there are several

00:07:50.600 --> 00:07:52.479
ways of accessing the subtitles of the

00:07:52.480 --> 00:07:56.239
video, and one of the ways is by running

00:07:56.240 --> 00:07:57.519
this sexp here,

00:07:57.520 --> 00:08:01.559
that opens a file in Lua that is

00:08:01.560 --> 00:08:03.319
what I use to generate the

00:08:03.320 --> 00:08:04.479
subtitles.

00:08:04.480 --> 00:08:08.519
Anyway... by the way, these things... each

00:08:08.520 --> 00:08:12.039
one of these things here is a hyperlink

00:08:12.040 --> 00:08:15.239
to a position of the video, so if I type

00:08:15.240 --> 00:08:18.879
this the right way it goes to that

00:08:18.880 --> 00:08:24.119
position. Anyway, let me go back...

00:08:24.120 --> 00:08:27.279
also, the tutorials of eev... the

00:08:27.280 --> 00:08:31.359
"intros" of eev, that start with "find-" and

00:08:31.360 --> 00:08:34.279
end with "-intro", they have lots of blocks

00:08:34.280 --> 00:08:39.039
that say "[Video links:]", like this one, and

00:08:39.040 --> 00:08:41.359
these blocks have links to positions

00:08:41.360 --> 00:08:43.879
in videos, and if we don't have a local

00:08:43.880 --> 00:08:47.919
copy of the video yet the thing shows

00:08:47.920 --> 00:08:49.799
us a script that lets us download the

00:08:49.800 --> 00:08:50.599
local copy.

00:08:50.600 --> 00:08:54.399
Anyway, I said that I was going

00:08:54.400 --> 00:08:58.759
to explain what I mean by "magic" and

00:08:58.760 --> 00:08:59.639
"black boxes".

00:08:59.640 --> 00:09:03.119
this is something that I've been

00:09:03.120 --> 00:09:05.119
trying to explain for a long time, and I

00:09:05.120 --> 00:09:07.319
think that I got a very good explanation

00:09:07.320 --> 00:09:09.839
about that in a video that I made

00:09:09.840 --> 00:09:13.199
about something called eev-wconfig, that

00:09:13.200 --> 00:09:14.999
is a tool for configuring eev on

00:09:15.000 --> 00:09:19.199
Windows without "magic" - without buttons

00:09:19.200 --> 00:09:22.479
that do things without explaining what

00:09:22.480 --> 00:09:22.879
they're doing.

00:09:22.880 --> 00:09:25.799
This is a part of the subtitles

00:09:25.800 --> 00:09:28.039
of the video, let me read that...

00:09:28.040 --> 00:09:32.319
eev-wconfig is an attempt to solve the

00:09:32.320 --> 00:09:35.039
problem of how to install these things

00:09:35.040 --> 00:09:37.279
on Windows both without magic and with

00:09:37.280 --> 00:09:37.879
very little

00:09:37.880 --> 00:09:41.679
magic. Remember this slogan: "any

00:09:41.680 --> 00:09:44.359
sufficiently advanced technology is

00:09:44.360 --> 00:09:45.399
indistinguishable from

00:09:45.400 --> 00:09:49.679
magic". Here in this video I'm going to

00:09:49.680 --> 00:09:52.199
use the term magic as a shorthand

00:09:52.200 --> 00:09:55.239
for sufficiently advanced technology,

00:09:55.240 --> 00:09:57.799
that is something that is complex and

00:09:57.800 --> 00:10:00.199
non-obvious and that is

00:10:00.200 --> 00:10:02.119
indistinguishable from magic in the

00:10:02.120 --> 00:10:04.119
sense of being almost impossible to

00:10:04.120 --> 00:10:07.439
understand. And I'm also going to use a

00:10:07.440 --> 00:10:10.279
the term "black box" as a near-synonym for

00:10:10.280 --> 00:10:13.079
magic, and sometimes the term

00:10:13.080 --> 00:10:16.039
"black box" is more convenient even though

00:10:16.040 --> 00:10:17.919
it's a bit longer - it has more

00:10:17.920 --> 00:10:20.159
letters - because when I use the term

00:10:20.160 --> 00:10:22.599
black box it invites us to use

00:10:22.600 --> 00:10:25.479
expressions like "opening the black box",

00:10:25.480 --> 00:10:26.639
and I'm going to use that

00:10:26.640 --> 00:10:28.039
expression a lot.

00:10:28.040 --> 00:10:37.399
Now let me try to explain what is...

00:10:37.400 --> 00:10:41.039
sorry, let me change the font...

00:10:41.040 --> 00:10:45.479
what is Lua. Lua is a minimalistic

00:10:45.480 --> 00:10:49.999
language, in the sense of

00:10:50.000 --> 00:10:53.679
"batteries not included"... it uses

00:10:53.680 --> 00:10:55.999
associative tables for most of its data

00:10:56.000 --> 00:10:56.799
structures...

00:10:56.800 --> 00:11:00.239
and it is so minimalistic

00:11:00.240 --> 00:11:03.999
that its default print function, when

00:11:04.000 --> 00:11:06.679
we tell... when we create an associative

00:11:06.680 --> 00:11:09.559
table and we ask it to print...

00:11:09.560 --> 00:11:13.319
when we ask "print" to print an

00:11:13.320 --> 00:11:15.719
associative table it just prints the

00:11:15.720 --> 00:11:17.879
address of the table. Here are some

00:11:17.880 --> 00:11:21.599
examples... here is a table, and when we

00:11:21.600 --> 00:11:24.679
ask "print" to print it it just says

00:11:24.680 --> 00:11:26.839
that it's the table at this address here.

00:11:26.840 --> 00:11:30.119
So, one of things that that most

00:11:30.120 --> 00:11:32.919
people do when they start using Lua is

00:11:32.920 --> 00:11:35.239
that either they download a package with

00:11:35.240 --> 00:11:37.079
a pretty-printing function or they write

00:11:37.080 --> 00:11:39.359
their own pretty-printing functions. My

00:11:39.360 --> 00:11:41.519
own pretty-printing function is called

00:11:41.520 --> 00:11:46.159
PP, with upper case letters, and it works

00:11:46.160 --> 00:11:47.679
like this...

00:11:47.680 --> 00:11:50.279
and it prints associative tables

00:11:50.280 --> 00:11:53.839
in a way like this. It says that for

00:11:53.840 --> 00:11:57.479
the key 1 the the value associated to

00:11:57.480 --> 00:12:00.039
it is 2, for the key 2 the value is

00:12:00.040 --> 00:12:04.959
3, and for the key 3 the value is 5.

00:12:04.960 --> 00:12:11.079
When I started using Lua one of my

00:12:11.080 --> 00:12:13.839
favorite languages was also a language

00:12:13.840 --> 00:12:15.639
that used associative tables a lot -

00:12:15.640 --> 00:12:16.839
it was called Icon...

00:12:16.840 --> 00:12:21.039
and I had to write my own

00:12:21.040 --> 00:12:23.839
pretty-printing functions for Icon, so

00:12:23.840 --> 00:12:27.319
I just had to port my pretty-printing

00:12:27.320 --> 00:12:29.719
functions to Lua... and my first

00:12:29.720 --> 00:12:32.239
version looked at something like this... it

00:12:32.240 --> 00:12:35.999
just had some some global functions... lots

00:12:36.000 --> 00:12:39.639
of them, actually...

00:12:39.640 --> 00:12:42.279
and after a while I rewrote it, and I

00:12:42.280 --> 00:12:44.599
rewrote it again, and again, and again, and

00:12:44.600 --> 00:12:47.279
this is one of the versions of that,

00:12:47.280 --> 00:12:49.879
is not even the default at this

00:12:49.880 --> 00:12:51.559
point...

00:12:51.560 --> 00:12:54.119
"Tos" is for "to string"...

00:12:54.120 --> 00:12:58.279
and this is a demo...

00:12:58.280 --> 00:13:01.919
it's very modular, so it's easy to replace

00:13:01.920 --> 00:13:05.639
parts of it, or to toggle flags... and this

00:13:05.640 --> 00:13:08.119
is an example. If I try to print the

00:13:08.120 --> 00:13:09.999
table of methods for a certain

00:13:10.000 --> 00:13:14.359
class... I will need a smaller font...

00:13:14.360 --> 00:13:16.799
it prints the table like this, with the

00:13:16.800 --> 00:13:20.039
names of the methods and then links to

00:13:20.040 --> 00:13:21.719
the source code of the functions...

00:13:21.720 --> 00:13:25.399
these links only make sense in Emacs and

00:13:25.400 --> 00:13:25.959
in eev...

00:13:25.960 --> 00:13:30.359
and when we run a link like this one...

00:13:30.360 --> 00:13:32.039
it shows the source code in the

00:13:32.040 --> 00:13:35.079
window at the right. So, for some

00:13:35.080 --> 00:13:37.879
functions the source code is three lines,

00:13:37.880 --> 00:13:39.999
for other ones it's one line... and

00:13:40.000 --> 00:13:44.879
whatever. Anyway, let me go

00:13:44.880 --> 00:13:47.039
back... Lua can be used in many different

00:13:47.040 --> 00:13:50.359
styles... most people hate other people's

00:13:50.360 --> 00:13:53.839
styles... when I started using it in the

00:13:53.840 --> 00:13:57.679
year 2000 I learned most of the basic

00:13:57.680 --> 00:13:59.839
language in a single day - it was very

00:13:59.840 --> 00:14:02.119
similar to things that I was already

00:14:02.120 --> 00:14:05.639
using... and then I rewrote the the mini-

00:14:05.640 --> 00:14:10.279
language that I was using to

00:14:10.280 --> 00:14:13.559
generate the HTML for my pages

00:14:13.560 --> 00:14:16.199
in Lua... actually I had to rewrite it

00:14:16.200 --> 00:14:18.079
many times, but the first version I

00:14:18.080 --> 00:14:21.519
certainly did in my first weeks or first

00:14:21.520 --> 00:14:22.519
months using Lua...

00:14:22.520 --> 00:14:27.279
In the beginning I was just using

00:14:27.280 --> 00:14:30.159
it for writing programs that either

00:14:30.160 --> 00:14:32.759
didn't take any input at all - because

00:14:32.760 --> 00:14:35.079
the input was already in the source file -

00:14:35.080 --> 00:14:40.599
or that worked as Unix programs,

00:14:40.600 --> 00:14:43.199
that would read files

00:14:43.200 --> 00:14:45.279
and process these files in some way

00:14:45.280 --> 00:14:48.999
and output something.

00:14:49.000 --> 00:14:52.319
I mentioned the "basic language" here...

00:14:52.320 --> 00:14:54.719
I only learned how to use closures,

00:14:54.720 --> 00:14:58.479
metatables, and coroutines many years later...

00:14:58.480 --> 00:15:02.399
in the beginning, when I started using Lua,

00:15:02.400 --> 00:15:04.199
it didn't have a package manager...

00:15:04.200 --> 00:15:06.799
it appeared later, it is called

00:15:06.800 --> 00:15:10.119
Luarocks... it has had this package

00:15:10.120 --> 00:15:13.279
manager for several years, most

00:15:13.280 --> 00:15:15.719
of the rocks for Luarocks are poorly

00:15:15.720 --> 00:15:18.959
documented and hacker-unfriendly,

00:15:18.960 --> 00:15:21.199
so you can't rely just on the

00:15:21.200 --> 00:15:23.679
documentation and you can't rely just on the

00:15:23.680 --> 00:15:26.719
source code, because, I mean... if you are

00:15:26.720 --> 00:15:29.159
a genius of course you can, but for

00:15:29.160 --> 00:15:31.479
people who are either lazy, or dumb, or

00:15:31.480 --> 00:15:34.279
whatever, like me, or unfocused...

00:15:34.280 --> 00:15:36.759
the source code is hard to

00:15:36.760 --> 00:15:38.959
understand and hard to tinker with.

00:15:38.960 --> 00:15:43.319
Some rocks are excellent. The

00:15:43.320 --> 00:15:46.599
best rocks are well documented

00:15:46.600 --> 00:15:48.719
but they are hacker-unfriendly

00:15:48.720 --> 00:15:50.999
in a sense that I hope that

00:15:51.000 --> 00:15:52.879
I'll be able to explain soon.

00:15:52.880 --> 00:15:54.999
The best rocks use local

00:15:55.000 --> 00:15:58.799
variables and metatables a lot -

00:15:58.800 --> 00:16:02.519
so if you are beginner

00:16:02.520 --> 00:16:03.799
learning Lua you're not going to

00:16:03.800 --> 00:16:06.159
understand what their source code do...

00:16:06.160 --> 00:16:08.519
they use lots of dirty tricks.

NOTE Object orientation in Lua

00:16:08.520 --> 00:16:12.479
Let me talk a bit about object

00:16:12.480 --> 00:16:14.959
orientation in Lua. It can be done in

00:16:14.960 --> 00:16:15.879
many ways...

00:16:15.880 --> 00:16:18.999
the main book about Lua, called

00:16:19.000 --> 00:16:21.439
"Programming in Lua", by one of the authors

00:16:21.440 --> 00:16:23.959
of the language, Roberto Ierusalimschy,

00:16:23.960 --> 00:16:26.679
presents several ways of doing

00:16:26.680 --> 00:16:29.279
object orientation in Lua... I hated all

00:16:29.280 --> 00:16:33.199
of these ways - and also the ways that I

00:16:33.200 --> 00:16:34.519
tried from the rocks.

00:16:34.520 --> 00:16:38.559
And then I wrote my own way

00:16:38.560 --> 00:16:40.639
of doing object orientation in Lua... it's

00:16:40.640 --> 00:16:43.559
very minimalistic, it's in this file here,

00:16:43.560 --> 00:16:48.679
eoo.lua... the main code is just this five

00:16:48.680 --> 00:16:49.639
lines here...

00:16:49.640 --> 00:16:53.439
and here's an example of how it works.

00:16:53.440 --> 00:16:58.439
Here we define the class Vector,

00:16:58.440 --> 00:17:02.719
with some metamethods...

00:17:02.720 --> 00:17:05.959
this metamethod here will tell Lua

00:17:05.960 --> 00:17:08.319
what to do when the

00:17:08.320 --> 00:17:12.639
user asks to add two vectors, this one

00:17:12.640 --> 00:17:15.919
here tells Lua what to do when the user

00:17:15.920 --> 00:17:18.479
asks Lua to convert a vector to a string,

00:17:18.480 --> 00:17:21.439
and... whatever, this one is

00:17:21.440 --> 00:17:24.039
something that I'm going to explain in a

00:17:24.040 --> 00:17:27.479
second. So, here we create a vector with

00:17:27.480 --> 00:17:30.279
these coordinates, 3 and 4... here we create

00:17:30.280 --> 00:17:33.319
another Vector... if we "print" here then Lua

00:17:33.320 --> 00:17:36.439
uses this function here, in the __tostring...

00:17:36.440 --> 00:17:39.759
if we add the two vectors it uses this

00:17:39.760 --> 00:17:43.479
function here, in the __add metamethod, and

00:17:43.480 --> 00:17:45.359
if we run the method :norm...

00:17:45.360 --> 00:17:49.959
it is defined here, in the table __index.

00:17:49.960 --> 00:17:57.999
Anyway...

00:17:58.000 --> 00:18:02.439
Even this thing being so small I used

00:18:02.440 --> 00:18:04.719
to forget how its innards worked all

00:18:04.720 --> 00:18:08.119
the time. Actually I always forget how

00:18:08.120 --> 00:18:09.759
things work and I have to remember them

00:18:09.760 --> 00:18:12.479
somehow... and I have to have

00:18:12.480 --> 00:18:15.959
tricks for remembering, and tricks for

00:18:15.960 --> 00:18:18.719
summarizing things, and diagrams, and so

00:18:18.720 --> 00:18:22.199
on. And every time that I forgot how this

00:18:22.200 --> 00:18:24.799
thing worked I went back to the

00:18:24.800 --> 00:18:26.879
source code, and then I looked at the

00:18:26.880 --> 00:18:29.039
diagrams... or, of course, in the

00:18:29.040 --> 00:18:31.719
first times I had to draw the diagrams...

00:18:31.720 --> 00:18:35.239
and I run the examples, and of course in

00:18:35.240 --> 00:18:36.479
in the beginning I thought that the code

00:18:36.480 --> 00:18:39.119
was clear and my examples were very

00:18:39.120 --> 00:18:41.559
brief, and so I had to rewrite the

00:18:41.560 --> 00:18:44.719
examples many times until they became,

00:18:44.720 --> 00:18:45.639
let's say...

00:18:45.640 --> 00:18:47.759
perfect.

00:18:47.760 --> 00:18:52.599
I was saying that Lua can be used in

00:18:52.600 --> 00:18:56.359
many ways, and in my way of using Lua - in

00:18:56.360 --> 00:18:59.439
my favorite way - everything can be

00:18:59.440 --> 00:19:02.159
inspected and modified from REPLs,

00:19:02.160 --> 00:19:06.319
like we can do in Emacs and in SmallTalk,

00:19:06.320 --> 00:19:08.519
or sort of. So, in my

00:19:08.520 --> 00:19:10.239
favorite way of using Lua there's no

00:19:10.240 --> 00:19:12.679
security at all, everything can be

00:19:12.680 --> 00:19:14.919
changed at all times.

00:19:14.920 --> 00:19:19.119
Of course most people hate that...

NOTE My init file

00:19:19.120 --> 00:19:22.599
My init file has lots of classes... by the

00:19:22.600 --> 00:19:26.079
way, instead of keeping many small files

00:19:26.080 --> 00:19:29.639
with many things I put lots of stuff

00:19:29.640 --> 00:19:31.279
in just one big init file.

00:19:31.280 --> 00:19:34.599
My init file has lots of classes,

00:19:34.600 --> 00:19:37.959
and lots of global functions, and

00:19:37.960 --> 00:19:41.799
lots of cruft - and people hate that,

00:19:41.800 --> 00:19:44.559
of course. This is an example...

00:19:44.560 --> 00:19:46.439
this is the index at the top

00:19:46.440 --> 00:19:48.359
of my init file,

00:19:48.360 --> 00:19:53.119
the classes start here, and then

00:19:53.120 --> 00:19:59.119
we have some functions, and

00:19:59.120 --> 00:20:01.199
then we have functions that load

00:20:01.200 --> 00:20:03.839
certain packages, and then we have... cruft.

00:20:03.840 --> 00:20:04.919
Whatever.

00:20:04.920 --> 00:20:08.119
Most people think that my style

00:20:08.120 --> 00:20:10.279
of using Lua is dirty, and dangerous...

00:20:10.280 --> 00:20:12.959
and they wouldn't touch my Lua code

00:20:12.960 --> 00:20:15.479
with a 10 feet pole... but most of the

00:20:15.480 --> 00:20:18.599
things that I'm going to present here in

00:20:18.600 --> 00:20:23.199
this presentation are ideas that should

00:20:23.200 --> 00:20:28.159
be easy to port to other environments

00:20:28.160 --> 00:20:32.279
and other languages, especially the

00:20:32.280 --> 00:20:35.279
diagrams... so the code is not so important.

NOTE LaTeX and LuaLaTeX

00:20:35.280 --> 00:20:39.039
Now let me talk a bit about LuaLaTeX,

00:20:39.040 --> 00:20:41.359
that is LaTeX with a Lua interpreter

00:20:41.360 --> 00:20:44.559
embedded inside, and two ways

00:20:44.560 --> 00:20:48.839
of generating pictures in LaTeX: TikZ,

00:20:48.840 --> 00:20:54.439
that is very famous, and Pict2e, that is not

00:20:54.440 --> 00:20:57.359
very famous and that is very low level...

00:20:57.360 --> 00:21:02.359
and I think that not many people use it.

00:21:02.360 --> 00:21:04.119
I said before that when I

00:21:04.120 --> 00:21:06.919
learned Lua I realized that it was

00:21:06.920 --> 00:21:09.199
very good for writing little

00:21:09.200 --> 00:21:14.919
languages. I was doing my PhD at the

00:21:14.920 --> 00:21:19.839
time and typesetting the diagrams for

00:21:19.840 --> 00:21:24.039
my PhD thesis was very boring, so

00:21:24.040 --> 00:21:29.879
one of the things that I did was that I

00:21:29.880 --> 00:21:34.439
created a little language for typesetting

00:21:34.440 --> 00:21:36.359
the diagrams for me. it was

00:21:36.360 --> 00:21:38.879
called Dednat because initially

00:21:38.880 --> 00:21:41.039
it only generated diagrams for

00:21:41.040 --> 00:21:43.759
Natural Deduction, and then it had

00:21:43.760 --> 00:21:45.119
several versions...

00:21:45.120 --> 00:21:46.679
these are the slides for my

00:21:46.680 --> 00:21:52.159
presentation about Dednat6... "Dednat6 is

00:21:52.160 --> 00:21:56.159
an extensible semi-preprocessor for

00:21:56.160 --> 00:22:01.159
LuaLaTeX that understands diagrams in

00:22:01.160 --> 00:22:05.359
ASCII art"... in the sense that when I have

00:22:05.360 --> 00:22:11.399
a .tex file that has this, and when

00:22:11.400 --> 00:22:13.279
Dednat6 is loaded,

00:22:13.280 --> 00:22:15.039
when I give the right commands

00:22:15.040 --> 00:22:19.559
Dednat6 interprets this block here as

00:22:19.560 --> 00:22:22.559
something that defines this

00:22:22.560 --> 00:22:28.239
diagram... oops, sorry, it interprets this

00:22:28.240 --> 00:22:30.599
diagram here, this diagram in

00:22:30.600 --> 00:22:34.879
comments here, as something that defines

00:22:34.880 --> 00:22:39.759
a diagram called foo... a deduction called

00:22:39.760 --> 00:22:41.439
foo, and it generates this code here...

00:22:41.440 --> 00:22:44.239
so that we can just invoke

00:22:44.240 --> 00:22:45.519
the definition of the

00:22:45.520 --> 00:22:47.719
deduction by typing \ded{foo}.

00:22:47.720 --> 00:22:50.759
And Dednat6 also

00:22:50.760 --> 00:22:57.159
supports another language for typesetting

00:22:57.160 --> 00:22:59.639
bidimensional diagrams with

00:22:59.640 --> 00:23:05.399
arrows and stuff for category Theory and

00:23:05.400 --> 00:23:08.519
blah blah blah... the specifications of

00:23:08.520 --> 00:23:12.039
these diagrams look like this...

00:23:12.040 --> 00:23:14.559
here is a... sorry, here is a very good

00:23:14.560 --> 00:23:16.719
example, this is a huge diagram...

00:23:16.720 --> 00:23:18.599
sorry, one second...

00:23:18.600 --> 00:23:20.399
so, the source code that generates

00:23:20.400 --> 00:23:25.119
this diagram here is just this thing at

00:23:25.120 --> 00:23:32.039
the left, so it's very visual... we can

00:23:32.040 --> 00:23:35.679
typeset the diagram in ASCII art here and

00:23:35.680 --> 00:23:38.759
then in this part here we tell how

00:23:38.760 --> 00:23:41.279
the nodes are to be joined, which

00:23:41.280 --> 00:23:43.799
arrows have to to have annotations, and

00:23:43.800 --> 00:23:45.039
so on...

00:23:45.040 --> 00:23:46.799
and this language is extensible in

00:23:46.800 --> 00:23:48.679
the sense that... uh, where's that...

00:23:48.680 --> 00:23:52.559
here: comments that start with "%:"

00:23:52.560 --> 00:23:54.119
are interpreted as

00:23:54.120 --> 00:23:56.079
definitions for tree diagrams,

00:23:56.080 --> 00:23:58.559
lines that start with "%D"

00:23:58.560 --> 00:24:00.639
define 2D diagrams with arrows and

00:24:00.640 --> 00:24:04.279
stuff, and lines that start with "%L"

00:24:04.280 --> 00:24:06.759
contain blocks of Lua code

00:24:06.760 --> 00:24:09.479
that we can use to extend the interpreter

00:24:09.480 --> 00:24:10.439
on-the-fly...

00:24:10.440 --> 00:24:12.679
anyway, here are some recent

00:24:12.680 --> 00:24:15.319
examples of diagrams that I used

00:24:15.320 --> 00:24:19.839
Dednat6 to typeset... this diagram

00:24:19.840 --> 00:24:21.919
here was generated by this

00:24:21.920 --> 00:24:22.879
specification here...

00:24:22.880 --> 00:24:27.239
and this diagram here with the

00:24:27.240 --> 00:24:30.719
curved arrows was generated by this

00:24:30.720 --> 00:24:32.719
specification here.

00:24:32.720 --> 00:24:39.079
So, Dednat6 was very easy to extend,

00:24:39.080 --> 00:24:41.879
and at some point I started to use it

00:24:41.880 --> 00:24:44.679
to generate diagrams using Pict2e -

00:24:44.680 --> 00:24:47.359
mainly for the classes that I give

00:24:47.360 --> 00:24:50.079
at the University... I teach mathematics and

00:24:50.080 --> 00:24:57.239
whatever... in a bad place. Whatever...

00:24:57.240 --> 00:25:00.039
Let me show an animation... here is a

00:25:00.040 --> 00:25:02.479
diagram that I generated with Dednat6,

00:25:02.480 --> 00:25:06.319
and it is a flip book animation, like... we

00:25:06.320 --> 00:25:09.279
type PgUp and PgDn and we go

00:25:09.280 --> 00:25:11.119
to the next page of the book and to the

00:25:11.120 --> 00:25:12.439
previous page of the book...

00:25:12.440 --> 00:25:16.279
and here is the source code that generates

00:25:16.280 --> 00:25:19.159
that. This source code is not very visual,

00:25:19.160 --> 00:25:22.559
so it's quite clumsy to edit that

00:25:22.560 --> 00:25:27.519
diagram directly in the .tex file like

00:25:27.520 --> 00:25:28.079
that...

NOTE Manim

00:25:28.080 --> 00:25:30.199
These diagrams were inspired

00:25:30.200 --> 00:25:33.039
by something called my Manim, that...

00:25:33.040 --> 00:25:37.559
I forgot the name of the guy, but

00:25:37.560 --> 00:25:41.479
it's a guy that makes many videos about

00:25:41.480 --> 00:25:44.839
Mathematics, and he created this library

00:25:44.840 --> 00:25:48.599
called Manim for generating his

00:25:48.600 --> 00:25:51.839
animations, and other people adapted

00:25:51.840 --> 00:25:55.919
his library to make it more accessible...

00:25:55.920 --> 00:25:59.359
I tried to learn it, but

00:25:59.360 --> 00:26:01.199
each animation, even an animation

00:26:01.200 --> 00:26:03.679
that has very few frames... each

00:26:03.680 --> 00:26:07.319
animation took ages to render, so it

00:26:07.320 --> 00:26:11.159
wasn't fun... and animations in PDFs can

00:26:11.160 --> 00:26:13.639
be rendered in seconds. So these

00:26:13.640 --> 00:26:18.679
things were fun for me, because my laptop

00:26:18.680 --> 00:26:24.359
is very very slow, and my Manim was not fun.

NOTE Generating diagrams from REPLs

00:26:24.360 --> 00:26:27.359
Anyway, writing code like this

00:26:27.360 --> 00:26:32.719
inside a .tex file was not very

00:26:32.720 --> 00:26:35.519
fun because it was hard to

00:26:35.520 --> 00:26:38.719
debug... so in 2022 I started to play

00:26:38.720 --> 00:26:41.319
with ways of generating these

00:26:41.320 --> 00:26:43.839
diagrams from REPLs, and I found a

00:26:43.840 --> 00:26:47.319
way for Pict2e and a way for TikZ...

00:26:47.320 --> 00:26:50.159
each one of these ways became a video...

00:26:50.160 --> 00:26:53.679
if you go to the list of first-class

00:26:53.680 --> 00:26:57.719
videos of eev you're going to see

00:26:57.720 --> 00:26:59.919
that there's a video about Pict2e here

00:26:59.920 --> 00:27:03.399
here and a video about TikZ...

00:27:03.400 --> 00:27:05.759
here you have some some information

00:27:05.760 --> 00:27:09.839
like length, an explanation, etc...

00:27:09.840 --> 00:27:11.719
and here are the pages for these videos.

00:27:11.720 --> 00:27:15.999
My page about the video about Pict2e

00:27:16.000 --> 00:27:20.079
looks like this, it has some diagrams...

00:27:20.080 --> 00:27:23.919
whatever... and this one is much

00:27:23.920 --> 00:27:26.679
nicer, and a lot of people

00:27:26.680 --> 00:27:30.599
watched that video... I mean, I think

00:27:30.600 --> 00:27:33.719
that 250 people watched it - for me that's

00:27:33.720 --> 00:27:35.599
a million of people...

00:27:35.600 --> 00:27:39.159
and this video is about how to

00:27:39.160 --> 00:27:44.079
extract diagrams from the manual... from

00:27:44.080 --> 00:27:46.599
the TikZ manual and how to run those

00:27:46.600 --> 00:27:49.759
examples in a REPL and modify

00:27:49.760 --> 00:27:53.159
them bit by bit... this is a a

00:27:53.160 --> 00:27:57.439
screenshot... but let me go back.

00:27:57.440 --> 00:28:00.959
At that point these things were just

00:28:00.960 --> 00:28:03.239
prototypes, the code was not very nice...

00:28:03.240 --> 00:28:07.519
and in this year I wrote... I was able

00:28:07.520 --> 00:28:12.399
to unify those two ways of generating PDFs,

00:28:12.400 --> 00:28:16.039
the one for TikZ and the one for Pict2e,

00:28:16.040 --> 00:28:18.719
and I unified them with many other

00:28:18.720 --> 00:28:20.879
things that generated diagrams.

00:28:20.880 --> 00:28:24.279
The basis of these things is

00:28:24.280 --> 00:28:29.319
something called Show2.lua... I'm not going

00:28:29.320 --> 00:28:35.759
to show its details now, but its

00:28:35.760 --> 00:28:39.079
extension that generates TikZ code

00:28:39.080 --> 00:28:43.039
is just this, so we can specify a

00:28:43.040 --> 00:28:45.799
diagram with just a block like this,

00:28:45.800 --> 00:28:49.079
and then uh if we

00:28:49.080 --> 00:28:54.239
run :show00() it returns a string

00:28:54.240 --> 00:28:56.199
that is just the body... the inner

00:28:56.200 --> 00:29:00.279
body of the .tex file, if we run this we

00:29:00.280 --> 00:29:02.999
see the whole .tex file, and if we run

00:29:03.000 --> 00:29:05.119
this we save the .tex file and we

00:29:05.120 --> 00:29:08.119
compile the .tex file to generate a PDF...

00:29:08.120 --> 00:29:10.959
and if we run this we show the PDF in

00:29:10.960 --> 00:29:14.239
the lower right window.

00:29:14.240 --> 00:29:17.759
And that's the same thing for all

00:29:17.760 --> 00:29:20.199
my recent programs that generate

00:29:20.200 --> 00:29:22.439
PDFs - they are all

00:29:22.440 --> 00:29:26.199
integrated... here is the one that...

00:29:26.200 --> 00:29:29.359
the basis for all my modules that generate

00:29:29.360 --> 00:29:30.719
diagrams with Pict2e...

00:29:30.720 --> 00:29:34.879
its demos are not very interesting,

00:29:34.880 --> 00:29:36.799
so let me show some demos of

00:29:36.800 --> 00:29:39.759
extensions that do interesting things...

00:29:39.760 --> 00:29:45.319
so, this is a diagram that I created

00:29:45.320 --> 00:29:47.479
by editing it in a REPL...

00:29:47.480 --> 00:29:51.279
I create several Pict objects here...

00:29:51.280 --> 00:29:54.479
and if I execute this it

00:29:54.480 --> 00:29:59.959
compiles an object, generates a PDF, and

00:29:59.960 --> 00:30:04.759
if I tap this... here is the PDF.

00:30:04.760 --> 00:30:07.599
And if I just ask Lua to

00:30:07.600 --> 00:30:10.079
display what is "pux", here,

00:30:10.080 --> 00:30:15.719
it shows the source code in Pict2e

00:30:15.720 --> 00:30:17.999
of the diagram... and the

00:30:18.000 --> 00:30:20.959
nice thing is that it is indented, so

00:30:20.960 --> 00:30:23.599
it's easy to debug the Pict2e code.

00:30:23.600 --> 00:30:25.919
If anyone is interested the

00:30:25.920 --> 00:30:28.639
module that does the tricks for

00:30:28.640 --> 00:30:31.879
indentation is very easy to understand...

00:30:31.880 --> 00:30:35.959
it has lots of tests and test blocks,

00:30:35.960 --> 00:30:38.599
and I think that its data

00:30:38.600 --> 00:30:42.079
structures are easy to understand.

00:30:42.080 --> 00:30:44.359
Anyway... here is another

00:30:44.360 --> 00:30:51.359
example. The :show() is

00:30:51.360 --> 00:30:56.439
here... it generates a 3D diagram.

NOTE Parsers

00:30:56.440 --> 00:31:06.279
Now let me talk about parsers and

00:31:06.280 --> 00:31:09.559
REPLs in VERY strange places... I mean,

00:31:09.560 --> 00:31:13.359
using REPLs to build parsers step by step

00:31:13.360 --> 00:31:17.959
and" replacing parts by more complex

00:31:17.960 --> 00:31:23.039
parts. So, I said that Lua is very

00:31:23.040 --> 00:31:28.279
minimalistic, and everybody knows that

00:31:28.280 --> 00:31:30.759
implementations of regular expressions

00:31:30.760 --> 00:31:32.479
are big and complex..

00:31:32.480 --> 00:31:34.679
so, instead of coming with

00:31:34.680 --> 00:31:37.439
full regular expressions Lua comes with

00:31:37.440 --> 00:31:39.879
something called "patterns" and a

00:31:39.880 --> 00:31:43.839
library function called "string.match".

00:31:43.840 --> 00:31:44.599
Here is

00:31:44.600 --> 00:31:50.319
a copy of the part of the manual that

00:31:50.320 --> 00:31:53.399
explains the syntax... a part of the

00:31:53.400 --> 00:31:57.159
syntax of of patterns... here's how

00:31:57.160 --> 00:31:59.279
string.match is described in the

00:31:59.280 --> 00:32:03.199
manual - it's just this... "looks for

00:32:03.200 --> 00:32:05.359
the first match of pattern in the string

00:32:05.360 --> 00:32:08.039
as blah blah blah"... and then we have to

00:32:08.040 --> 00:32:10.159
go to the other section of the menual

00:32:10.160 --> 00:32:11.479
that explains patterns.

00:32:11.480 --> 00:32:20.079
Lua patterns are so simple,

00:32:20.080 --> 00:32:23.159
so limited, that they don't even

00:32:23.160 --> 00:32:26.519
have the the alternation operator...

00:32:26.520 --> 00:32:29.759
here is how it is described in the

00:32:29.760 --> 00:32:31.599
elisp manual -

00:32:31.600 --> 00:32:36.039
backslash-pipe specifies

00:32:36.040 --> 00:32:40.359
an alternative, blah blah blah.

00:32:40.360 --> 00:32:42.879
When we want to to build more

00:32:42.880 --> 00:32:45.319
complex... regular expressions,

00:32:45.320 --> 00:32:49.199
patterns, grammars, etc... we have to use

00:32:49.200 --> 00:32:52.679
an external library for that... no,

00:32:52.680 --> 00:32:56.279
sorry, a library that is external

00:32:56.280 --> 00:32:58.239
but that was written by one of the

00:32:58.240 --> 00:33:00.879
authors of Lua itself. This library

00:33:00.880 --> 00:33:05.879
is called Lpeg, and its manual says...

00:33:05.880 --> 00:33:09.599
"Lpeg is a new pattern matching library for

00:33:09.600 --> 00:33:12.039
Lua based on Parsing Expression Grammars

00:33:12.040 --> 00:33:18.759
(PEGs)". The manual is very terse, I

00:33:18.760 --> 00:33:21.559
found it incredibly hard to read... it

00:33:21.560 --> 00:33:25.439
doesn't have any diagrams - it has some

00:33:25.440 --> 00:33:29.759
examples, though... and the Lua Wiki

00:33:29.760 --> 00:33:33.879
has a big page called Lpeg Tutorial

00:33:33.880 --> 00:33:35.359
with lots of examples...

00:33:35.360 --> 00:33:38.879
but it it also doesn't have

00:33:38.880 --> 00:33:41.199
diagrams and I found some things

00:33:41.200 --> 00:33:42.719
incredibly hard to understand.

00:33:42.720 --> 00:33:45.879
For example, this is something that is in

00:33:45.880 --> 00:33:48.879
the the manual of Lpeg that I saw and I

00:33:48.880 --> 00:33:51.639
thought: "Wow, great! This makes all sense

00:33:51.640 --> 00:33:53.159
and is going to be very useful!"...

00:33:53.160 --> 00:33:54.199
it's a way to to build

00:33:54.200 --> 00:33:57.199
grammars that can be recursive,

00:33:57.200 --> 00:34:01.359
and they sort of can encode BNF

00:34:01.360 --> 00:34:03.439
grammars... we just have to translate the

00:34:03.440 --> 00:34:06.479
BNF a bit to get rid of some

00:34:06.480 --> 00:34:08.079
recursions and to translate them to

00:34:08.080 --> 00:34:08.999
something else.

00:34:09.000 --> 00:34:11.919
And the manual also has some things

00:34:11.920 --> 00:34:15.159
that I thought: "Oh, no! I don't have any

00:34:15.160 --> 00:34:18.359
idea of what this thing does"... and in fact

00:34:18.360 --> 00:34:20.399
I saw these things for the first

00:34:20.400 --> 00:34:22.359
time more than 10 years ago and they

00:34:22.360 --> 00:34:26.079
only started to make sense one year ago.

00:34:26.080 --> 00:34:30.519
One example is group captures.

00:34:30.520 --> 00:34:36.359
Lpeg also comes with a

00:34:36.360 --> 00:34:38.719
module called the Re module... let me

00:34:38.720 --> 00:34:41.719
pronounce as it in Portuguese - the Re

00:34:41.720 --> 00:34:45.759
module... its manual says: "The Re

00:34:45.760 --> 00:34:48.199
module (provided by the file re.lua in the

00:34:48.200 --> 00:34:51.159
distribution) supports a somewhat conventional

00:34:51.160 --> 00:34:56.239
regular expression syntax for pattern usage

00:34:56.240 --> 00:34:58.679
within lpeg"... and

00:34:58.680 --> 00:35:03.519
this is a quick reference... this

00:35:03.520 --> 00:35:06.319
thing is very brief, it has some nice

00:35:06.320 --> 00:35:08.919
examples but it's hard to understand anyway...

00:35:08.920 --> 00:35:13.199
and here are some comments about

00:35:13.200 --> 00:35:17.279
my attempts to learn Re.lua. This is

00:35:17.280 --> 00:35:20.639
a class... in this case it's a very small

00:35:20.640 --> 00:35:24.839
class... this file implements a :pm()

00:35:24.840 --> 00:35:28.679
method - I'm going to show examples of

00:35:28.680 --> 00:35:32.239
other :pm() methods very soon - so, this is

00:35:32.240 --> 00:35:35.799
a :pm() method for Re.lua that lets us

00:35:35.800 --> 00:35:38.719
compare the syntax of Lua patterns, Lpeg,

00:35:38.720 --> 00:35:43.999
and Re... let's see this example here... so,

00:35:44.000 --> 00:35:47.319
if we run this it loads my version of

00:35:47.320 --> 00:35:52.799
lpeg... no, sorry, my version of lpegrex...

00:35:52.800 --> 00:35:57.119
and it shows that when we apply

00:35:57.120 --> 00:36:01.199
the :pm() method to this Lua pattern, this

00:36:01.200 --> 00:36:04.879
lpeg pattern, and this Re pattern

00:36:04.880 --> 00:36:07.999
they all give the same results. So we can

00:36:08.000 --> 00:36:10.799
use this thing... this kind of thing here

00:36:10.800 --> 00:36:14.119
to show how to translate from Lua

00:36:14.120 --> 00:36:16.519
patterns, that are familiar because

00:36:16.520 --> 00:36:18.519
they're similar to regular expressions,

00:36:18.520 --> 00:36:20.199
only weaker...

00:36:20.200 --> 00:36:24.799
to lpeg, that is super weird

00:36:24.800 --> 00:36:27.759
and to Re, that is not so weird.

00:36:27.760 --> 00:36:35.159
Anyway, the comment says that in 2012

00:36:35.160 --> 00:36:37.519
I had a project that needed a

00:36:37.520 --> 00:36:40.479
precedence passer that could parse

00:36:40.480 --> 00:36:43.239
arithmetical expressions with the right

00:36:43.240 --> 00:36:46.639
precedences... and at that point I was

00:36:46.640 --> 00:36:49.919
still struggling with pure lpeg, and I

00:36:49.920 --> 00:36:52.359
couldn't do much with it, so I tried to

00:36:52.360 --> 00:36:55.519
learn Re.lua instead, and I wrote this old

00:36:55.520 --> 00:36:56.319
class here...

00:36:56.320 --> 00:37:01.039
that allowed me to use a preprocessor

00:37:01.040 --> 00:37:03.279
on patterns for Lua. And the thing is that

00:37:03.280 --> 00:37:04.879
with this preprocessor I could

00:37:04.880 --> 00:37:07.839
specify precedence grammars using this

00:37:07.840 --> 00:37:11.879
thing here, that worked, but was super

00:37:11.880 --> 00:37:15.999
clumsy... and I gave up after a few attempts.

00:37:16.000 --> 00:37:21.879
and in 2022 I heard about something

00:37:21.880 --> 00:37:23.239
called lpegrex,

00:37:23.240 --> 00:37:29.799
that was a... a kind of extension or Re,

00:37:29.800 --> 00:37:32.879
and it was much more powerful than re.lua,

00:37:32.880 --> 00:37:34.919
but after a while I realized that it

00:37:34.920 --> 00:37:37.639
had the same defects as re.lua...

00:37:37.640 --> 00:37:40.839
and let me explain that, because

00:37:40.840 --> 00:37:44.439
it has all to do with the things about

00:37:44.440 --> 00:37:48.039
black boxes and magic that I told in the

00:37:48.040 --> 00:37:52.919
beginning. Both... I mean, sorry, neither

00:37:52.920 --> 00:37:57.199
re.lua or lpegrex had some features that

00:37:57.200 --> 00:38:00.799
I needed... they didn't let us explore...

00:38:00.800 --> 00:38:03.679
sorry, they received a pattern that was

00:38:03.680 --> 00:38:06.839
specified as a string, and it converted

00:38:06.840 --> 00:38:09.679
that into an lpeg pattern, but it didn't

00:38:09.680 --> 00:38:12.559
let us explore the the lpeg patterns

00:38:12.560 --> 00:38:15.159
that it generated...

00:38:15.160 --> 00:38:18.759
their code was written in a way

00:38:18.760 --> 00:38:21.319
that was REPL-unfriendly - I

00:38:21.320 --> 00:38:24.279
couldn't modify parts of the code

00:38:24.280 --> 00:38:28.399
bit by bit in a REPL and try to change

00:38:28.400 --> 00:38:31.719
the code without changing the

00:38:31.720 --> 00:38:34.199
original file... the code was very

00:38:34.200 --> 00:38:36.839
hard to explore, to hack, and to extend -

00:38:36.840 --> 00:38:39.159
in my opinion... the documentation was not

00:38:39.160 --> 00:38:43.319
very clear... and I sent one or two messages

00:38:43.320 --> 00:38:47.159
to the the developer of lpegrex and...

00:38:47.160 --> 00:38:50.759
he was too busy to help me. He

00:38:50.760 --> 00:38:53.959
answered it very briefly, and, uh, to be

00:38:53.960 --> 00:38:56.599
honest I felt... rejected. I felt that I

00:38:56.600 --> 00:38:58.679
wasn't doing anything interesting...

00:38:58.680 --> 00:39:03.399
whatever, whatever...

00:39:03.400 --> 00:39:09.239
So, in 2022 I was trying to learn lpegrex

00:39:09.240 --> 00:39:11.559
because I was thinking that it would

00:39:11.560 --> 00:39:13.719
solve my problems - but it didn't...

00:39:13.720 --> 00:39:16.479
it didn't have the features that I needed,

00:39:16.480 --> 00:39:20.919
it was hard to extend, hard to explore,

00:39:20.920 --> 00:39:23.279
and hard to debug, and I

00:39:23.280 --> 00:39:25.039
decided to rewrite it in a more

00:39:25.040 --> 00:39:30.639
hacker-friendly way - in the sense that...

00:39:30.640 --> 00:39:33.759
was modular, and I could replace any

00:39:33.760 --> 00:39:35.399
part of the module from a REPL...

NOTE ELpeg1.lua

00:39:35.400 --> 00:39:43.679
My version of it was called ELpeg1.lua...

00:39:43.680 --> 00:39:47.679
and I decided that in my version I

00:39:47.680 --> 00:39:49.639
wouldn't have the part that

00:39:49.640 --> 00:39:54.879
receives a grammar specified as a string

00:39:54.880 --> 00:39:57.519
and converts that to lpeg... I would

00:39:57.520 --> 00:40:00.959
just have the backend part, that are the

00:40:00.960 --> 00:40:03.999
functions in lpeg that let us specify

00:40:04.000 --> 00:40:05.479
powerful grammars.

00:40:05.480 --> 00:40:11.759
Let me go back. Let me explain a

00:40:11.760 --> 00:40:15.519
bit about lpeg... Lua has

00:40:15.520 --> 00:40:21.599
coercions: the + expects to receive

00:40:21.600 --> 00:40:23.999
true numbers, and if one of its arguments,

00:40:24.000 --> 00:40:26.999
or both of them, are strings, it converts

00:40:27.000 --> 00:40:29.839
the string... the strings to numbers so in

00:40:29.840 --> 00:40:33.519
this case here, 2+"3",

00:40:33.520 --> 00:40:36.159
it returns the number 5,

00:40:36.160 --> 00:40:39.359
and this is the concatenation

00:40:39.360 --> 00:40:42.119
operator... it expects to receive

00:40:42.120 --> 00:40:44.999
strings, so in this case it will

00:40:45.000 --> 00:40:47.359
convert the number 2 to the string "2",

00:40:47.360 --> 00:40:50.279
and the concatenation of thes two

00:40:50.280 --> 00:40:54.479
things will be 23... oops, sorry, "23"

00:40:54.480 --> 00:40:56.279
as a string.

00:40:56.280 --> 00:40:58.519
Lpeg also has some coercions.

00:40:58.520 --> 00:41:01.759
I usually set these

00:41:01.760 --> 00:41:05.799
globals to let me write my grammars

00:41:05.800 --> 00:41:09.719
in a very compact way, so instead

00:41:09.720 --> 00:41:14.759
of lpeg.B, lpeg.C, etc I use these globals,

00:41:14.760 --> 00:41:18.359
like uppercase B, uppercase C, and so on...

00:41:18.360 --> 00:41:21.679
and with these globals I can write

00:41:21.680 --> 00:41:26.759
things like this: C(1)*"_"...

00:41:26.760 --> 00:41:33.199
and lpeg knows that lpeg.C...

00:41:33.200 --> 00:41:38.879
it sort of expands this to lpeg.C,

00:41:38.880 --> 00:41:42.039
but lpeg.C expects to receive

00:41:42.040 --> 00:41:44.839
an lpeg pattern, and 1 is not yet an

00:41:44.840 --> 00:41:47.879
lpeg pattern, so it is coerced into an

00:41:47.880 --> 00:41:51.799
lpeg pattern by calling lpeg.P,

00:41:51.800 --> 00:41:55.679
so this short thing here becomes

00:41:55.680 --> 00:42:03.399
equivalent to lpeg.C(lpeg.P(1)), and the

00:42:03.400 --> 00:42:07.399
multiplication, when at least one of its

00:42:07.400 --> 00:42:10.759
arguments is an lpeg pattern... it expects

00:42:10.760 --> 00:42:13.199
to receive two lpeg patterns, and in

00:42:13.200 --> 00:42:15.239
this case the one at the right is

00:42:15.240 --> 00:42:18.319
just a string, so it is coerced to an lpeg

00:42:18.320 --> 00:42:20.079
pattern by using lpeg.P.

00:42:20.080 --> 00:42:25.599
With this idea we can sort of

00:42:25.600 --> 00:42:28.439
understand the comparison here. I mean,

00:42:28.440 --> 00:42:31.719
let me run it again... this first part is

00:42:31.720 --> 00:42:34.679
very similar to a regular expression

00:42:34.680 --> 00:42:35.359
here at the left...

00:42:35.360 --> 00:42:39.759
and when we apply this... Lua pattern

00:42:39.760 --> 00:42:43.639
to this subject here the result

00:42:43.640 --> 00:42:47.799
is this thing here, this thing, this

00:42:47.800 --> 00:42:54.319
thing and this thing... I'm going to

00:42:54.320 --> 00:42:56.119
call each one of these results

00:42:56.120 --> 00:42:59.519
"captures", so each of these things

00:42:59.520 --> 00:43:03.319
between parentheses "captures" a substring

00:43:03.320 --> 00:43:06.039
of the original string and these

00:43:06.040 --> 00:43:08.559
captured substrings are returned in a

00:43:08.560 --> 00:43:11.839
certain order. Here is how to express the

00:43:11.840 --> 00:43:12.759
same thing in lpeg...

00:43:12.760 --> 00:43:15.919
it's very cryptic but it's a

00:43:15.920 --> 00:43:20.719
good way to understand the some basic

00:43:20.720 --> 00:43:23.879
operators of lpeg, I mean we can look at

00:43:23.880 --> 00:43:26.479
the manual and understand and

00:43:26.480 --> 00:43:30.519
what C, S and R do, and also

00:43:30.520 --> 00:43:37.959
exponentiation... and this strange thing

00:43:37.960 --> 00:43:41.319
here receives this string here, runs

00:43:41.320 --> 00:43:43.279
a function that I have defined, that

00:43:43.280 --> 00:43:46.039
converts it to an object of a certain

00:43:46.040 --> 00:43:47.759
class, and that class

00:43:47.760 --> 00:43:52.399
represents Re patterns, so this thing

00:43:52.400 --> 00:43:54.479
is treated as a pattern for re.lua,

00:43:54.480 --> 00:43:56.479
and it is matched against the string,

00:43:56.480 --> 00:43:59.439
and it returns the same thing as the

00:43:59.440 --> 00:44:02.559
other one.

00:44:02.560 --> 00:44:05.519
Also, this thing here also has a

00:44:05.520 --> 00:44:08.479
comparison with lpegrex, but these

00:44:08.480 --> 00:44:11.559
patterns are very trivial, they

00:44:11.560 --> 00:44:13.359
don't do anything very strange...

00:44:13.360 --> 00:44:15.759
so let's go back and see what

00:44:15.760 --> 00:44:18.239
kinds of very strange things there are.

00:44:18.240 --> 00:44:26.559
Here is the page of lpegrex at github,

00:44:26.560 --> 00:44:29.719
here's the documentation...

00:44:29.720 --> 00:44:32.439
it's relatively brief,

00:44:32.440 --> 00:44:35.239
it explains lpegrex as being an

00:44:35.240 --> 00:44:39.719
extension of Re.lua, so it explains

00:44:39.720 --> 00:44:42.879
mainly the additional features... here is a

00:44:42.880 --> 00:44:45.119
quick reference that explains only the

00:44:45.120 --> 00:44:46.359
additional features...

00:44:46.360 --> 00:44:49.639
some of the these things

00:44:49.640 --> 00:44:50.919
I was able to understand

00:44:50.920 --> 00:44:57.559
by struggling a lot, and some I wasn't

00:44:57.560 --> 00:45:02.439
able to even by spending several evenings

00:45:02.440 --> 00:45:04.319
try to to build examples...

00:45:04.320 --> 00:45:12.879
and this is something very nice. Lpegrex

00:45:12.880 --> 00:45:15.879
comes with some example parsers... and

00:45:15.880 --> 00:45:18.679
here is a parser that parses the Lua

00:45:18.680 --> 00:45:22.479
grammar - I mean, this is the the grammar

00:45:22.480 --> 00:45:25.959
for Lua 5.4 at the end of the

00:45:25.960 --> 00:45:31.199
reference manual... it's just this... this

00:45:31.200 --> 00:45:34.799
is in a kind of BNF, and this is the BNF

00:45:34.800 --> 00:45:35.599
translated

00:45:35.600 --> 00:45:39.919
to the language of lpegrex, so this

00:45:39.920 --> 00:45:43.039
thing uses many constructions that are

00:45:43.040 --> 00:45:47.999
in re.lua and some extra constructions that

00:45:48.000 --> 00:45:50.959
are described here... and with these

00:45:50.960 --> 00:45:54.239
examples I was able to to understand

00:45:54.240 --> 00:45:55.159
some of the...

00:45:55.160 --> 00:45:58.079
of these things here that are

00:45:58.080 --> 00:46:00.239
described here in the quick

00:46:00.240 --> 00:46:04.719
reference - but not all.

00:46:04.720 --> 00:46:11.279
So, I wasn't able to use lpegrex

00:46:11.280 --> 00:46:14.279
by itself, because some things didn't

00:46:14.280 --> 00:46:16.199
make much sense, and I decided to

00:46:16.200 --> 00:46:18.759
reimplement it in my own style,

00:46:18.760 --> 00:46:23.679
because that would be a way to map...

00:46:23.680 --> 00:46:26.839
to at the very least map what I had

00:46:26.840 --> 00:46:29.559
understood and what I didn't, learn

00:46:29.560 --> 00:46:32.999
one feature at a time, do comparisons, and

00:46:33.000 --> 00:46:35.319
so on.

00:46:35.320 --> 00:46:38.799
Here I pointed to two features of lpeg...

00:46:38.800 --> 00:46:41.679
in one I said "Oh, great! This thing can

00:46:41.680 --> 00:46:44.319
be used to to define grammars, even

00:46:44.320 --> 00:46:45.959
recursive grammars", and so on...

00:46:45.960 --> 00:46:49.759
and this is an "Oh, no!" feature - one

00:46:49.760 --> 00:46:51.759
thing that didn't make any sense at all...

00:46:51.760 --> 00:46:56.439
group captures. One thing that I did to

00:46:56.440 --> 00:46:59.039
understand group captures was to

00:46:59.040 --> 00:47:02.319
represent them as diagrams. Of course in

00:47:02.320 --> 00:47:05.359
the beginning I was drawing these

00:47:05.360 --> 00:47:08.919
diagrams by hand, but then I realized

00:47:08.920 --> 00:47:11.559
that I could use the bits of lpeg

00:47:11.560 --> 00:47:14.759
that I already knew to build a grammar

00:47:14.760 --> 00:47:17.479
that would parse a little language and

00:47:17.480 --> 00:47:20.999
generate these diagrams in LaTeX, and I was

00:47:21.000 --> 00:47:21.919
able to make this.

00:47:21.920 --> 00:47:25.279
In this diagram here

00:47:25.280 --> 00:47:30.719
this thing above the arrow is Lua code...

00:47:30.720 --> 00:47:33.759
a piece of Lua code that

00:47:33.760 --> 00:47:37.119
specifies an lpeg pattern... this

00:47:37.120 --> 00:47:39.559
thing here at the top is the string that

00:47:39.560 --> 00:47:43.039
is being matched, and the things below

00:47:43.040 --> 00:47:46.599
the underbraces are the captures that

00:47:46.600 --> 00:47:50.639
each thing... sorry, that each thing

00:47:50.640 --> 00:47:51.319
captures.

00:47:51.320 --> 00:47:58.479
For example, this underbrace here

00:47:58.480 --> 00:48:00.279
corresponds to this pattern here,

00:48:00.280 --> 00:48:02.879
that parses a single character but

00:48:02.880 --> 00:48:05.559
doesn't return any captures, this thing

00:48:05.560 --> 00:48:08.119
here parses a single "b" and doesn't

00:48:08.120 --> 00:48:11.239
return any captures, this thing here

00:48:11.240 --> 00:48:14.399
parses a single character and captures

00:48:14.400 --> 00:48:16.879
it, and this thing here parses the

00:48:16.880 --> 00:48:21.319
character "d" and captures it... and this

00:48:21.320 --> 00:48:24.439
other thing here transforms this

00:48:24.440 --> 00:48:27.279
pattern into another pattern...

00:48:27.280 --> 00:48:33.119
returns first a capture with all

00:48:33.120 --> 00:48:35.079
the string that was parsed by this

00:48:35.080 --> 00:48:37.399
pattern here, and then all the captures

00:48:37.400 --> 00:48:41.079
returned by this thing here before

00:48:41.080 --> 00:48:42.959
the ":".

00:48:42.960 --> 00:48:45.479
So, this was a way to build

00:48:45.480 --> 00:48:48.599
concrete examples for things that the

00:48:48.600 --> 00:48:52.159
lpag manual was explaining in a very terse

00:48:52.160 --> 00:48:55.799
way, and it worked for me - some things

00:48:55.800 --> 00:48:56.999
that were very

00:48:57.000 --> 00:48:59.839
mysterious started to make sense, and I

00:48:59.840 --> 00:49:03.199
started to have intelligent questions

00:49:03.200 --> 00:49:06.079
to ask in the mailing list.

00:49:06.080 --> 00:49:10.959
And with that I was able to

00:49:10.960 --> 00:49:12.959
understand what are group captures,

00:49:12.960 --> 00:49:17.879
and group captures that receive a name...

00:49:17.880 --> 00:49:22.719
Well, let me explain what this does.

00:49:22.720 --> 00:49:27.119
This thing here captures... sorry, parses

00:49:27.120 --> 00:49:29.359
the empty string and returns this as a

00:49:29.360 --> 00:49:32.959
constant... so, this is something that

00:49:32.960 --> 00:49:35.799
doesn't exist in regular expressions...

00:49:35.800 --> 00:49:38.639
it parses nothing and

00:49:38.640 --> 00:49:41.839
returns this as a capture... then this

00:49:41.840 --> 00:49:44.599
thing here returns these two

00:49:44.600 --> 00:49:47.159
constants here, and parses the empty

00:49:47.160 --> 00:49:51.279
string, and this thing here converts

00:49:51.280 --> 00:49:54.159
the results of this thing here into a

00:49:54.160 --> 00:49:57.639
group capture, and stores it in the label

00:49:57.640 --> 00:50:03.279
"d"... and then here's another constant

00:50:03.280 --> 00:50:03.719
capture.

NOTE Building lists

00:50:03.720 --> 00:50:05.679
And I realized that these things

00:50:05.680 --> 00:50:08.599
here were similar to how Lua

00:50:08.600 --> 00:50:09.839
specifies building lists...

00:50:09.840 --> 00:50:16.239
when we build... sorry, tables. When

00:50:16.240 --> 00:50:18.759
we build a table, and we say that the

00:50:18.760 --> 00:50:21.879
first element of the table is here, this

00:50:21.880 --> 00:50:23.559
element is put at the end of the table...

00:50:23.560 --> 00:50:29.399
when after the that would say d=42...

00:50:29.400 --> 00:50:31.199
we are putting the 42

00:50:31.200 --> 00:50:34.559
in the the slot whose key is "d".

00:50:34.560 --> 00:50:38.999
This was happening with lpeg captures,

00:50:39.000 --> 00:50:43.359
but there was something very strange...

00:50:43.360 --> 00:50:46.199
these group captures could hold

00:50:46.200 --> 00:50:49.199
more than one capture - more than one

00:50:49.200 --> 00:50:51.759
value... so there was something between

00:50:51.760 --> 00:50:58.039
lists and tables. I started to use this

00:50:58.040 --> 00:51:00.479
notation to...

00:51:00.480 --> 00:51:04.959
explain in my notation what they

00:51:04.960 --> 00:51:08.159
were doing... many things started

00:51:08.160 --> 00:51:10.239
to make sense, many mysterious

00:51:10.240 --> 00:51:12.879
sentences in the manual started to

00:51:12.880 --> 00:51:14.439
make sense... but some didn't...

00:51:14.440 --> 00:51:19.679
but at least I was able to send

00:51:19.680 --> 00:51:22.319
some intelligent questions to the

00:51:22.320 --> 00:51:25.199
mailing lis,t and the author of Lua and

00:51:25.200 --> 00:51:27.359
lpeg answered some of them...

00:51:27.360 --> 00:51:31.519
he was not very happy about my

00:51:31.520 --> 00:51:34.959
questions - he... told me that those

00:51:34.960 --> 00:51:37.679
diagrams were a waste of time, the

00:51:37.680 --> 00:51:40.559
manual was perfectly clear, and so on...

00:51:40.560 --> 00:51:44.919
whatever - but I was able to...

00:51:44.920 --> 00:51:48.879
so, it was weird, but I was able to

00:51:48.880 --> 00:51:51.799
understand lots of things from his

00:51:51.800 --> 00:51:56.519
answers. This is a copy of one of

00:51:56.520 --> 00:51:58.239
my messages, then there's another one,

00:51:58.240 --> 00:52:01.239
another one, some of them had diagrams...

00:52:01.240 --> 00:52:04.359
then he complained about these diagrams,

00:52:04.360 --> 00:52:08.439
he said that these things here, that look

00:52:08.440 --> 00:52:11.119
like table constructors, "do not exist"...

00:52:11.120 --> 00:52:17.199
whatever... anyway, once I understood

00:52:17.200 --> 00:52:20.679
group captures many features

00:52:20.680 --> 00:52:23.359
were very easy to understand

00:52:23.360 --> 00:52:26.039
and I started to be able to use lpeg to

00:52:26.040 --> 00:52:28.159
to build some very interesting things...

00:52:28.160 --> 00:52:33.039
I was able to reproduce some

00:52:33.040 --> 00:52:36.359
of the features that I saw in lpegrex -

00:52:36.360 --> 00:52:41.079
remember that this... where is that?

00:52:41.080 --> 00:52:46.119
this is the syntax of Lua... here -

00:52:46.120 --> 00:52:48.959
I was able to understand

00:52:48.960 --> 00:52:52.479
how these things here were translated to

00:52:52.480 --> 00:52:55.359
lpeg code... to lpeg patterns

00:52:55.360 --> 00:52:58.239
by using group captures in a certain

00:52:58.240 --> 00:53:03.039
way... I was able to implement them

00:53:03.040 --> 00:53:04.759
in ELpeg1.lua...

00:53:04.760 --> 00:53:08.719
and after some time I was able to use

00:53:08.720 --> 00:53:12.879
ELpeg1.lua to build grammars that

00:53:12.880 --> 00:53:14.159
were able to parse

00:53:14.160 --> 00:53:18.679
arithmetical expressions with the

00:53:18.680 --> 00:53:20.959
right precedence... and here's an example

00:53:20.960 --> 00:53:23.319
in which I built the grammar step by step...

00:53:23.320 --> 00:53:29.239
and I test the current grammar, and I

00:53:29.240 --> 00:53:35.079
replace a bit, and then I test the new

00:53:35.080 --> 00:53:36.599
grammar and so on...

00:53:36.600 --> 00:53:39.079
and you can see that the result is

00:53:39.080 --> 00:53:43.359
always a tree that is drawn in a

00:53:43.360 --> 00:53:44.239
nice two dimensional way...

00:53:44.240 --> 00:53:48.919
At this point these powers here

00:53:48.920 --> 00:53:50.559
are returned as a list,

00:53:50.560 --> 00:53:53.119
as an operation "pow"

00:53:53.120 --> 00:53:57.559
with several arguments, here... and then

00:53:57.560 --> 00:54:00.519
I apply a kind of parsing combinator,

00:54:00.520 --> 00:54:03.719
here... that transforms these trees into

00:54:03.720 --> 00:54:08.199
other trees and with these combinators

00:54:08.200 --> 00:54:12.199
here I can specify that the "^" is

00:54:12.200 --> 00:54:14.639
associative in a certain direction...

00:54:14.640 --> 00:54:17.519
that the "/" is associative in

00:54:17.520 --> 00:54:20.119
another direction... the "-" uses

00:54:20.120 --> 00:54:23.079
the same direction as a the "/",

00:54:23.080 --> 00:54:26.079
and so on... and they have the

00:54:26.080 --> 00:54:29.679
right precedences.

00:54:29.680 --> 00:54:34.559
So, here are the tests...

00:54:34.560 --> 00:54:38.119
here is my file ELpeg1.lua... it has

00:54:38.120 --> 00:54:41.719
several classes, each class has tests

00:54:41.720 --> 00:54:42.279
after it...

00:54:42.280 --> 00:54:46.239
I was able to implement something

00:54:46.240 --> 00:54:50.519
that lpegrex has, that is called

00:54:50.520 --> 00:54:53.519
"keywords", that is very useful for parsing

00:54:53.520 --> 00:54:56.479
programs in programming languages...

00:54:56.480 --> 00:54:59.439
I was able to implement something

00:54:59.440 --> 00:55:02.639
similar to the debugger... to the

00:55:02.640 --> 00:55:07.999
lpeg debugger lpeg uses... I was

00:55:08.000 --> 00:55:11.399
frustrated by some limitations of

00:55:11.400 --> 00:55:16.839
the lpeg debugger, and I implemented

00:55:16.840 --> 00:55:23.439
my own that is, uh... much better!...

00:55:23.440 --> 00:55:24.759
Let me show something else... I was

00:55:24.760 --> 00:55:27.119
able to translate a good part of the

00:55:27.120 --> 00:55:33.039
Lua parser, here, to ELpeg1.lua... I haven't

00:55:33.040 --> 00:55:38.399
finished yet, but I have most of the

00:55:38.400 --> 00:55:39.719
the translation here...

00:55:39.720 --> 00:55:47.279
and after having all that I was able to

00:55:47.280 --> 00:55:50.319
build other grammars very quickly...

00:55:50.320 --> 00:55:55.239
writing new parsers finally became fun.

00:55:55.240 --> 00:55:58.719
And here's one example that I showed in the

00:55:58.720 --> 00:56:00.639
beginning.

00:56:00.640 --> 00:56:05.799
If I remember correctly...

00:56:05.800 --> 00:56:10.639
I took a figure from the Wikipedia...

00:56:10.640 --> 00:56:12.439
I don't have its link now...

00:56:12.440 --> 00:56:17.079
but I specified a grammar that parses

00:56:17.080 --> 00:56:20.119
exactly the example that appears

00:56:20.120 --> 00:56:20.839
in the Wikipedia...

00:56:20.840 --> 00:56:24.679
so, with my grammar, considering that

00:56:24.680 --> 00:56:28.719
the top level entry is "Stmt", when I

00:56:28.720 --> 00:56:30.679
parse this string here

00:56:30.680 --> 00:56:36.599
the result is this tree...

00:56:36.600 --> 00:56:41.119
and I can do some operations on that,

00:56:41.120 --> 00:56:44.039
I can define how this thing is to be

00:56:44.040 --> 00:56:45.639
converted into LaTeX,

00:56:45.640 --> 00:56:49.399
I can define other operations

00:56:49.400 --> 00:56:52.999
that convert trees into other trees, and

00:56:53.000 --> 00:56:54.879
here are some tests of these operations...

00:56:54.880 --> 00:57:00.359
This is what I showed in the beginning...

00:57:00.360 --> 00:57:02.759
I'm not going to explain all the details

00:57:02.760 --> 00:57:03.999
of this thing now...

00:57:04.000 --> 00:57:09.199
this :show() converts this thing

00:57:09.200 --> 00:57:11.919
into LaTeX in the way specified by these

00:57:11.920 --> 00:57:16.159
instructions here, that says that...

00:57:16.160 --> 00:57:25.239
well, whatever...

00:57:25.240 --> 00:57:32.959
and here's the result - the LaTeXed result...

00:57:32.960 --> 00:57:41.759
and these diagrams here are generated by

00:57:41.760 --> 00:57:46.719
this file here, that defines a simple

00:57:46.720 --> 00:57:48.479
grammar that parses this thing here,

00:57:48.480 --> 00:57:51.999
and then LaTeXes it in a certain way, and

00:57:52.000 --> 00:57:56.399
and also tests to check if this code here...

00:57:56.400 --> 00:58:01.999
this Lua code that generates an lpeg grammar...

00:58:02.000 --> 00:58:05.799
parses this subject here and

00:58:05.800 --> 00:58:08.599
returns the expected result...

00:58:08.600 --> 00:58:12.239
So: this is the code that I

00:58:12.240 --> 00:58:16.719
wanted to show. I wanted to show many

00:58:16.720 --> 00:58:19.919
more things but I wasn't able to prepare

00:58:19.920 --> 00:58:23.919
them before the conference... and I hope

00:58:23.920 --> 00:58:27.519
that soon - for some value of "soon" -

00:58:27.520 --> 00:58:30.399
I'll be able to create REPL-based

00:58:30.400 --> 00:58:33.919
tutorials for lpeg, Re, and ELpeg1.lua...

00:58:33.920 --> 00:58:36.319
where lpeg is something very famous,

00:58:36.320 --> 00:58:39.199
Re is a module of lpeg...

00:58:39.200 --> 00:58:42.399
I could also do something like this

00:58:42.400 --> 00:58:47.799
for lpegrex... and ELpeg1.lua is

00:58:47.800 --> 00:58:51.159
the thing that I wrote, the one that

00:58:51.160 --> 00:58:56.799
has test in comments, and the tests

00:58:56.800 --> 00:58:59.519
usually generate trees, and sometimes

00:58:59.520 --> 00:59:00.879
they generate TeX code.

00:59:00.880 --> 00:59:04.959
Yeah, so that's it! I wanted to

00:59:04.960 --> 00:59:07.159
present much more but I wasn't able to

00:59:07.160 --> 00:59:10.480
prepare it... so: sorry, thanks, bye! =)