path: root/2021/captions/emacsconf-2021-exec--org-as-an-executable-format--tom-gillespie--main.vtt

WEBVTT

00:00.320 --> 00:00:03.679
Hello all! Welcome to EmacsConf 2021.

00:00:03.679 --> 00:00:05.040
I'm Tom Gillespie.

00:00:05.040 --> 00:00:06.799
Thank you to the organizers for

00:00:06.799 --> 00:00:07.680
all your hard work,

00:00:07.680 --> 00:00:08.639
and for inviting me to

00:00:08.639 --> 00:00:10.021
give this short talk on

00:00:10.021 --> 00:00:12.240
"Org as an executable format".

00:00:12.240 --> 00:00:13.840
The links to the talk page,

00:00:13.840 --> 00:00:16.000
the GitHub page for the project,

00:00:16.000 --> 00:00:18.880
and the package on MELPA

00:18.880 --> 00:00:20.160
are listed on the right.

00:00:20.160 --> 00:00:21.760
Let's start with one of the motivating

00:00:21.760 --> 00:00:25.920
use cases for executable Org files.

00:25.920 --> 00:00:29.339
Many users keep global configuration

00:00:29.339 --> 00:00:31.840
for Org in an init.el file,

00:00:31.840 --> 00:00:33.520
which works for many workflows.

00:00:33.520 --> 00:00:36.239
However, for reproducible research,

00:00:36.239 --> 00:00:37.600
this is a challenge

00:00:37.600 --> 00:00:39.280
because if an Org file is

00:00:39.280 --> 00:00:41.440
dissociated from the init.el file,

00:00:41.440 --> 00:00:43.040
then often it will no longer

00:00:43.040 --> 00:00:44.640
function as expected.

00:44.640 --> 00:46.719
One potential solution to this problem

00:46.719 --> 00:48.160
is to be able to include all of the

00:48.160 --> 00:00:50.239
global configuration for Emacs

00:00:50.239 --> 00:00:52.960
and the environment in the Org file itself,

00:00:52.960 --> 00:00:53.840
in which case when you

00:00:53.840 --> 00:00:55.440
go to reuse the Org file,

00:00:55.440 --> 00:00:58.640
it will work as expected.

00:58.640 --> 00:01:00.480
What does an executable Org file

00:01:00.480 --> 00:01:02.559
look like in action?

01:02.559 --> 01:05.280
Here's a demo of an executable Org file

01:05.280 --> 01:09.680
running in Bash, Dash, Zsh, and PowerShell.

01:09.680 --> 00:01:14.799
So, we are currently in Bash,

01:14.799 --> 01:19.360
and we can run our demo,

01:19.360 --> 00:01:21.119
and it will print some stuff,

00:01:21.119 --> 00:01:22.640
and wait for input.

01:22.640 --> 00:01:24.144
We can also run it in Dash,

00:01:24.144 --> 00:01:25.720
which is the default for Debian

00:01:25.720 --> 00:01:29.840
and derivatives.

01:29.840 --> 01:32.320
Same program works as expected.

01:32.320 --> 00:01:38.560
Zsh also. And lastly PowerShell,

01:38.560 --> 00:01:41.439
if we try to run demo.org itself,

00:01:41.439 --> 00:01:42.640
we see (that) we get an error

00:01:42.640 --> 00:01:43.764
because PowerShell cares

00:01:43.764 --> 00:01:45.439
about file extensions,

01:45.439 --> 00:01:49.840
so, if we symlink to ps1,

00:01:49.840 --> 00:01:51.680
then it works as expected,

00:01:51.680 --> 00:01:53.341
and there are ways to alias this,

00:01:53.341 --> 00:01:55.044
so that you can run it as a program

00:01:55.044 --> 00:01:58.640
without the ps1 extension.

01:58.640 --> 02:03.920
So, how does this work?

02:03.920 --> 00:02:05.759
There are three components

00:02:05.759 --> 00:02:07.352
to an executable Org file

00:02:07.352 --> 00:02:08.560
that all need to be present

00:02:08.560 --> 00:02:10.080
in order for this to work.

02:10.080 --> 02:11.920
Starting from the top of the file,

02:11.920 --> 02:14.239
we have a shebang block.

02:14.239 --> 00:02:16.640
Next we have an Org Babel block

00:02:16.640 --> 00:02:17.760
written in Emacs Lisp,

00:02:17.760 --> 00:02:20.000
which is what we actually saw executing,

00:02:20.000 --> 00:02:20.959
and then there are some

00:02:20.959 --> 00:02:23.520
eval local variables or Elvs

02:23.520 --> 02:25.200
that are involved in making this

02:25.200 --> 02:26.800
actually executable.

02:26.800 --> 02:29.760
Let's start with the shebang block.

02:29.760 --> 00:02:33.280
Org syntax does not have support

00:02:33.280 --> 00:02:34.959
for shebang lines.

02:34.959 --> 02:37.120
However, it supports the shebang block.

02:37.120 --> 02:39.440
This is because Org comments, blocks,

02:39.440 --> 00:02:41.760
keywords, etc. that start with the

00:02:41.760 --> 00:02:43.920
sharp sign have the same syntax as

00:02:43.920 --> 00:02:46.720
comments in POSIX and PowerShell.

02:46.720 --> 00:02:53.280
This block is in fact valid

02:53.280 --> 00:02:55.040
Bash, Dash, Zsh, PowerShell,

00:02:55.040 --> 00:02:57.280
and maybe some other shells as well.

02:57.280 --> 00:03:02.480
In essence what it does is,

03:02.480 --> 00:03:03.440
perform some setup

00:03:03.440 --> 00:03:06.080
to avoid polluting standard output,

00:03:06.080 --> 00:03:07.516
and then it runs Emacs

00:03:07.516 --> 00:03:08.959
to load the file itself.

00:03:08.959 --> 00:03:12.640
The Elisp that is passed on the command line

00:03:12.640 --> 00:03:14.959
is explicated over here on the right,

03:14.959 --> 00:03:17.920
and in essence what it does is,

00:03:17.920 --> 00:03:20.480
to keep the startup time minimal

00:03:20.480 --> 00:03:21.760
and as low as possible,

00:03:21.760 --> 00:03:24.080
it loads the absolute bare minimum

00:03:24.080 --> 00:03:25.120
needed for Babel,

00:03:25.120 --> 00:03:27.519
and then it calls hack-local-variables

00:03:27.519 --> 00:03:31.680
triggering the eval-local-variables.

03:31.680 --> 00:03:33.614
What do the eval-local-variables do,

00:03:33.614 --> 00:03:34.799
and how those work?

00:03:34.799 --> 00:03:36.319
The essence of the approach is to

00:03:36.319 --> 00:03:38.720
use org-confirm-babel-evaluate

00:03:38.720 --> 00:03:40.480
to allow Babel execution.

00:03:40.480 --> 00:03:43.360
We can't set it to nil because that is

00:03:43.360 --> 00:03:45.308
an arbitrary code execution vector,

00:03:45.308 --> 00:03:48.000
which we don't want if we're sharing files.

03:48.000 --> 00:03:49.555
Instead what we do is,

00:03:49.555 --> 00:03:52.000
we use the fact that it can be a function,

00:03:52.000 --> 00:03:55.280
and we normalize the block of code,

00:03:55.280 --> 00:03:57.320
we checksum it, and then we check

00:03:57.320 --> 00:03:59.280
that it matches this checksum up here

00:03:59.280 --> 00:04:00.400
at the top of the file,

00:04:00.400 --> 00:04:02.640
and then sort of inside there,

04:02.640 --> 00:04:04.159
inside of org-sbe,

00:04:04.159 --> 00:04:06.560
which is Org source block evaluate,

04:06.560 --> 04:08.959
we call the block.

04:08.959 --> 00:04:11.120
The actual implementation of this

00:04:11.120 --> 00:04:12.497
is somewhat more complicated.

00:04:12.497 --> 00:04:14.799
However, it's small enough to fit in

00:04:14.799 --> 00:04:17.519
the local variables at the end of the file.

04:17.519 --> 00:04:19.040
One thing to note is that

00:04:19.040 --> 00:04:20.799
if you are using PowerShell,

00:04:20.799 --> 00:04:23.919
PowerShell parses the whole file

00:04:23.919 --> 00:04:25.040
which means that

00:04:25.040 --> 00:04:27.759
for any normal Org content,

00:04:27.759 --> 00:04:28.639
you need to put it in

04:28.639 --> 00:04:30.240
a multi-line PowerShell comment,

00:04:30.240 --> 00:04:31.199
and close it.

00:04:31.199 --> 00:04:32.371
So, once we hit

00:04:32.371 --> 00:04:34.160
hack-local-variables at the end,

00:04:34.160 --> 00:04:37.120
we run the eval-local-variables block,

00:04:37.120 --> 00:04:40.880
then we enter this Elisp block,

04:40.880 --> 00:04:42.720
and you can write whatever you want.

00:04:42.720 --> 00:04:44.160
All the power of Org Babel is

00:04:44.160 --> 00:04:45.360
now at your fingertips

00:04:45.360 --> 00:04:47.199
in order to do what you need

04:47.199 --> 04:48.800
for this file.

04:48.800 --> 00:04:50.320
Finally, let's do a quick demo

00:04:50.320 --> 00:04:52.453
of how to use this to make

00:04:52.453 --> 00:04:54.800
your own Org files executable.

04:54.800 --> 00:05:01.840
Orgstrap is available on MELPA as mentioned,

05:01.840 --> 00:05:06.080
and it can be installed using package.el

00:05:06.080 --> 00:05:11.280
by calling package-install orgstrap.

05:11.280 --> 05:13.520
It will download, and it will install.

05:13.520 --> 00:05:18.720
Then you can open an existing file

05:18.720 --> 00:05:21.919
or a new file. In this case,

00:05:21.919 --> 00:05:25.199
let me open a file called example.org.

05:25.199 --> 00:05:26.996
And then orgstrap provides

00:05:26.996 --> 00:05:29.360
command called orgstrap-init.

05:29.360 --> 00:05:30.953
What orgstrap-init does is,

00:05:30.953 --> 00:05:33.759
it populates a file with the machinery

05:33.759 --> 00:05:36.639
needed to run an orgstrap block.

00:05:36.639 --> 00:05:38.560
We're just going to do a message

05:38.560 --> 05:43.440
"hello orgstrap!".

05:43.440 --> 00:05:46.160
If you look up at the top,

00:05:46.160 --> 00:05:47.386
you will see that the

00:05:47.386 --> 00:05:50.320
orgstrap-block-checksum will change

05:50.320 --> 00:05:51.520
when I save the file.

00:05:51.520 --> 00:05:53.840
This makes it much easier to author files

00:05:53.840 --> 00:05:55.120
with orgstrap blocks.

00:05:55.120 --> 00:05:56.560
And then we need one last piece

00:05:56.560 --> 00:05:57.876
of machinery,

00:05:57.876 --> 00:06:00.400
which is the shebang block.

00:06:00.400 --> 00:06:01.520
I am just going to steal

00:06:01.520 --> 00:06:02.560
the shebang block from

00:06:02.560 --> 00:06:06.080
this other file over here

06:06.080 --> 00:06:07.039
since it is available.

00:06:07.039 --> 00:06:08.880
You can also get it from shebang.org,

00:06:08.880 --> 00:06:10.800
and I have plans to add a command

00:06:10.800 --> 00:06:12.560
to insert this into the file directly,

00:06:12.560 --> 00:06:15.600
which may actually be done by the time

00:06:15.600 --> 00:06:19.520
this video is actually posted and visible.

06:19.520 --> 00:06:21.120
There's one last step,

00:06:21.120 --> 00:06:24.160
which is that we need to run dired

06:24.160 --> 00:06:27.520
in order to… There we go.

06:27.520 --> 06:31.039
So, we use Shift m, capital m,

06:31.039 --> 00:06:32.800
in order to make our

00:06:32.800 --> 00:06:35.520
example file executable,

06:35.520 --> 00:06:42.140
and then if we come back to here,

00:06:42.140 --> 00:06:47.360
we see that example.org is now executable,

00:06:47.360 --> 00:06:48.000
and we can run it.

06:48.000 --> 06:50.560
"hello orgstrap!", and we're done.

06:50.560 --> 06:52.639
So, that's the basic workflow

06:52.639 --> 06:54.880
for getting orgstrap files

06:54.880 --> 00:06:56.240
to be executable.

00:06:56.240 --> 00:06:58.960
I will be around to answer questions live,

00:06:58.960 --> 00:07:00.800
and I will be also available in

00:07:00.800 --> 00:07:03.280
the #emacsconf IRC channel all day.

00:07:03.280 --> 00:07:04.960
I hope you have found this useful,

00:07:04.960 --> 00:07:08.160
and thank you very much for watching.

00:07:08.160 --> 00:07:09.160
[captions by bhavin192 (Bhavin Gandhi)]