path: root/2025/captions/emacsconf-2025-org-babel--making-orgbabel-reactive--abhinav-tushar--main.vtt

WEBVTT captioned by abhinav

NOTE What are reactive notebooks?

00:00:01.120 --> 00:00:03.033
Hello, everyone. My name is Abhinav,

00:00:03.034 --> 00:00:03.900
and I'm going to talk about

00:00:03.901 --> 00:00:07.140
how to make Org Babel reactive. So reactivity here

00:00:07.240 --> 00:00:10.000
means reactivity in the sense of reactive notebooks.

00:00:10.001 --> 00:00:11.600
So if you used Org Babel,

00:00:11.601 --> 00:00:13.933
you might also have used Jupyter notebooks,

00:00:13.934 --> 00:00:16.100
which are basically notebooks primarily for

00:00:16.200 --> 00:00:16.933
Python programming,

00:00:16.934 --> 00:00:20.100
where you have these text and code blocks interleaved,

00:00:20.101 --> 00:00:23.157
and then you can execute every code block independently,

00:00:23.158 --> 00:00:25.858
and then you control the order of execution manually,

00:00:25.859 --> 00:00:27.199
or you can just run the code blocks

00:00:27.200 --> 00:00:29.699
from top to bottom. But with reactive notebooks,

00:00:29.700 --> 00:00:32.927
what happens is that there's another way of running

00:00:32.928 --> 00:00:35.329
which is basically by having all these

00:00:35.330 --> 00:00:36.900
dependent code blocks automatically get

00:00:37.000 --> 00:00:38.900
executed whenever you make a change.

00:00:38.901 --> 00:00:40.774
So for example, if you change a variable,

00:00:40.775 --> 00:00:42.060
everything else that's dependent on

00:00:42.160 --> 00:00:44.433
that variable will be executed automatically.

00:00:44.434 --> 00:00:49.041
I'll show you an example of what that looks like.

NOTE Reactivity demo

00:00:49.042 --> 00:00:51.762
Right, here's an example reactive Notebook.

00:00:51.763 --> 00:00:53.460
So this is called Observable.

00:00:53.560 --> 00:00:54.863
Observable is this tool made by

00:00:54.864 --> 00:00:57.679
the creator of d3.js which is

00:00:57.680 --> 00:01:01.499
a famous JavaScript charting library. So here, the

00:01:01.500 --> 00:01:03.667
interface is very similar to Jupyter Notebook.

00:01:03.668 --> 00:01:06.407
You basically are having these cells

00:01:06.408 --> 00:01:08.508
and each cell could be a text cell, like here,

00:01:08.509 --> 00:01:09.588
this is a Markdown cell

00:01:09.589 --> 00:01:11.609
and then there are these code blocks.

00:01:11.610 --> 00:01:15.250
Now each code cell is basically defining a variable.

00:01:15.251 --> 00:01:17.740
This is important in reactive notebooks because

00:01:17.840 --> 00:01:21.140
each cell is connected to other cell via this variable

00:01:21.240 --> 00:01:23.552
usage. So here data is defined,

00:01:23.553 --> 00:01:25.012
then there is filtered which is defined

00:01:25.013 --> 00:01:27.620
which is dependent on data, and then this plot is

00:01:27.720 --> 00:01:29.133
dependent on filtered.

00:01:29.134 --> 00:01:31.153
So now, in a classical notebook, what I will do is

00:01:31.154 --> 00:01:34.394
if I change something here, let's say from 1 to 2,

00:01:34.395 --> 00:01:34.854
I will have to run this, and then run this plot block again

00:01:34.855 --> 00:01:40.335
to make the change be visible.

00:01:40.336 --> 00:01:42.055
But in a reactive notebook, what happens is

00:01:42.056 --> 00:01:44.396
I can just change this from some value

00:01:44.397 --> 00:01:46.256
to some value, and then execute,

00:01:46.257 --> 00:01:48.817
and then every descendant is also executed,

00:01:48.818 --> 00:01:50.940
because that's how the reactivity works.

00:01:51.040 --> 00:01:51.937
You change this variable,

00:01:51.938 --> 00:01:53.080
so this should also be changed,

00:01:53.081 --> 00:01:55.238
because this is dependent on this variable.

00:01:55.239 --> 00:01:56.858
Now this is really helpful

00:01:56.859 --> 00:01:58.900
if you have a very complex and messy notebook

00:01:59.000 --> 00:02:01.199
which is what actually happens in reality.

00:02:01.200 --> 00:02:03.480
You end up doing an exploratory analysis,

00:02:03.481 --> 00:02:05.959
and you have these code blocks lying here and there.

00:02:05.960 --> 00:02:07.101
Then you change something

00:02:07.102 --> 00:02:09.281
and then you have to keep something in your mind

00:02:09.282 --> 00:02:11.362
that if I change this, I need to run

00:02:11.363 --> 00:02:13.023
these five code blocks again

00:02:13.024 --> 00:02:15.604
to finally get to the result that I want to see.

00:02:15.605 --> 00:02:20.467
Stale state causes a lot of issues in Jupyter Notebooks.

00:02:20.468 --> 00:02:23.788
So this is really good for reactivity, sorry reproducibility,

00:02:23.789 --> 00:02:26.630
but this is also really good for

00:02:26.631 --> 00:02:28.599
just having this exploration

00:02:28.600 --> 00:02:30.117
that you're trying to do. For example,

00:02:30.118 --> 00:02:31.761
you're changing something and it's really easy

00:02:31.762 --> 00:02:34.887
to just see that change happening in real time

00:02:34.888 --> 00:02:38.498
in your outcome variables, right?

NOTE Org-Babel

00:02:38.499 --> 00:02:41.920
So I was wondering how to introduce this reactivity in Org Mode.

00:02:41.921 --> 00:02:43.200
And here's how it will look like.

00:02:43.201 --> 00:02:46.302
So this is a demo Org Mode file.

00:02:46.303 --> 00:02:48.603
There are many Org Babel blocks here.

00:02:48.604 --> 00:02:49.563
So you start from here.

00:02:49.564 --> 00:02:52.085
Let's say this is a code block. It has a name.

00:02:52.086 --> 00:02:53.665
And then there's another code block,

00:02:53.666 --> 00:02:55.426
which is dependent on the previous one,

00:02:55.427 --> 00:02:57.807
as you can see here, and so on.

00:02:57.808 --> 00:02:59.368
And then finally, there's a plot here,

00:02:59.369 --> 00:03:00.889
which is a gnuplot code.

00:03:00.890 --> 00:03:02.550
And you can see the image here.

00:03:02.551 --> 00:03:04.131
Now, what happens usually is that

00:03:04.132 --> 00:03:05.196
if I change this value from,

00:03:05.197 --> 00:03:09.199
let's say, 113 to 112, nothing happens on its own right?

00:03:09.200 --> 00:03:12.199
There's an extra step of execution that I will have to do

00:03:12.200 --> 00:03:15.079
so I will do that, and then the value is changed.

00:03:15.080 --> 00:03:17.699
Now the problem is that only this value is changed and

00:03:17.700 --> 00:03:21.079
if I go down and see the image, nothing will have changed.

NOTE Running the whole buffer

00:03:21.080 --> 00:03:23.079
So what I can do is basically,

00:03:23.080 --> 00:03:24.818
a really simple thing is that,

00:03:24.819 --> 00:03:26.500
a simple trick is to basically

00:03:26.600 --> 00:03:29.445
enable a hook, like, add a hook

00:03:29.446 --> 00:03:30.525
whenever you're saving the buffer,

00:03:30.526 --> 00:03:31.866
you just run the full buffer again,

00:03:31.867 --> 00:03:34.287
like run all the code blocks automatically.

00:03:34.288 --> 00:03:36.849
Now if you do that, you can basically make a change somewhere

00:03:36.850 --> 00:03:37.889
and then you can, you know,

00:03:37.890 --> 00:03:41.071
see how everything else is changing

00:03:41.072 --> 00:03:42.712
which gives you some sort of reactivity,

00:03:42.713 --> 00:03:43.972
but there's still a lot of computation

00:03:43.973 --> 00:03:45.973
that's being wasted.

00:03:45.974 --> 00:03:49.595
You might not want to change or run this code block again

00:03:49.596 --> 00:03:51.900
when something down there is changing.

NOTE Caching

00:03:51.901 --> 00:03:54.567
So to counter that, you can actually add caching.

00:03:54.568 --> 00:03:57.133
So if you add caching to any code block,

00:03:57.134 --> 00:03:59.800
that code block will only be executed again

00:03:59.801 --> 00:04:02.300
if that code has changed or

00:04:02.400 --> 00:04:04.755
the input variables have changed.

00:04:04.756 --> 00:04:06.336
But the other problem is that

00:04:06.337 --> 00:04:08.659
you don't want caching to be enabled for a lot of cases

00:04:08.660 --> 00:04:10.840
where the code block is actually dependent on

00:04:10.841 --> 00:04:12.722
external state, like for example,

00:04:12.723 --> 00:04:15.024
some sort of randomness or time.

00:04:15.025 --> 00:04:17.433
So caching also is, you know, kind of,

00:04:17.434 --> 00:04:18.967
it's, like, an important thing to use,

00:04:18.968 --> 00:04:21.660
but it's probably not giving you the complete answer.

NOTE Computation dependencies

00:04:21.760 --> 00:04:25.973
So what we can instead do is basically figure out

00:04:25.974 --> 00:04:28.554
the whole computation dependencies here.

00:04:28.555 --> 00:04:31.275
So let's say if I look at this buffer,

00:04:31.276 --> 00:04:35.076
here's how all the blocks are connected.

00:04:35.077 --> 00:04:37.656
So as you can see the plot code block

00:04:37.657 --> 00:04:40.117
is dependent on c and then legendpg,

00:04:40.118 --> 00:04:43.918
and they themselves are dependent on these other nodes.

00:04:43.919 --> 00:04:47.279
So when I make a change in b, I only want b to run

00:04:47.280 --> 00:04:50.844
and then c and then plot. I don't want anything else to run.

00:04:50.845 --> 00:04:54.267
So what I did was I wrote a small minor mode for Org Mode

00:04:54.268 --> 00:04:55.368
which does exactly this.

00:04:55.369 --> 00:04:57.769
So whenever you are in a code block

00:04:57.770 --> 00:04:59.871
and you are making a change and then you save it,

00:04:59.872 --> 00:05:01.913
it will just follow the trail from that code block

00:05:01.914 --> 00:05:05.355
to every other descendant which is going to be impacted,

00:05:05.356 --> 00:05:09.719
and it just runs all of them, and nothing else gets executed.

00:05:09.720 --> 00:05:13.119
So to see it in action, I will just enable that mode.

00:05:13.120 --> 00:05:17.021
Yeah, right. So now here, if I change this 113 to 112

00:05:17.022 --> 00:05:21.243
and I save, this code, this variable gets changed.

00:05:21.244 --> 00:05:23.744
It's the same value because I did not update it again.

00:05:23.745 --> 00:05:25.719
And you can also see b also got changed

00:05:25.720 --> 00:05:29.667
because it's just following all the execution order and so on.

00:05:29.668 --> 00:05:31.727
The plot also got updated.

00:05:31.728 --> 00:05:34.068
We will be able to see more clearly

00:05:34.069 --> 00:05:36.402
once I change something more substantial.

00:05:36.402 --> 00:05:36.402
So here's another variable.

00:05:36.403 --> 00:05:41.332
So I added a small toggle button here,

00:05:41.333 --> 00:05:43.468
which is again part of the minor mode.

00:05:43.469 --> 00:05:45.209
So since this is nil, if I toggle it,

00:05:45.210 --> 00:05:49.300
it will become true. And this variable dictates whether

00:05:49.400 --> 00:05:51.174
the plot will have the legend or not.

00:05:51.175 --> 00:05:54.457
So if I toggle it to be t, now it's t

00:05:54.458 --> 00:05:57.900
and you can see that the plot has legend that's visible.

00:05:57.901 --> 00:06:03.139
If I toggle it back again to nil, the legend is gone.

00:06:03.140 --> 00:06:04.533
Now this is nice, this...

NOTE Making this even better

00:06:04.534 --> 00:06:06.380
This is already pretty helpful for me

00:06:06.480 --> 00:06:10.179
but what we can do is we can make it even better.

00:06:10.180 --> 00:06:11.400
So one of the nicer ideas

00:06:11.401 --> 00:06:13.015
from these reactive notebooks

00:06:13.016 --> 00:06:16.078
is this idea of having an infinite canvas

00:06:16.079 --> 00:06:19.022
where you don't look at the document model,

00:06:19.023 --> 00:06:20.623
you look at the whole document

00:06:20.624 --> 00:06:25.008
as a canvas of multiple connected documents.

00:06:25.009 --> 00:06:26.589
One good thing that happens there is that

00:06:26.590 --> 00:06:29.550
you can basically have a piece of code somewhere

00:06:29.551 --> 00:06:30.410
and then piece of code

00:06:30.411 --> 00:06:32.499
somewhere very different position in the document,

00:06:32.500 --> 00:06:34.732
but you can put them together in the canvas

00:06:34.733 --> 00:06:36.933
and then see them side by side.

00:06:36.934 --> 00:06:38.294
So here also, let's say

00:06:38.295 --> 00:06:41.996
if I want to just have this image shown up at the top,

00:06:41.997 --> 00:06:45.857
what I can do is like I can pop this out,

00:06:45.858 --> 00:06:49.938
which opens a child frame, and then I can just go here.

00:06:49.939 --> 00:06:52.460
This child frame is showing the same image.

00:06:52.461 --> 00:06:55.502
So there's no change. So if I toggle this variable here,

00:06:55.503 --> 00:06:58.423
you can see that the image is updated.

00:06:58.424 --> 00:07:02.199
If I toggle it back to nil, the image, the legend is gone.

00:07:02.200 --> 00:07:03.367
And you can obviously, you know,

00:07:03.368 --> 00:07:08.690
you can make a lot of things come up as child frames.

00:07:08.691 --> 00:07:09.430
This is the same image.

00:07:09.431 --> 00:07:11.291
So even if you go down to the document,

00:07:11.292 --> 00:07:13.810
you will see the same image.

00:07:13.811 --> 00:07:18.174
So yeah, this is what I have right now.

00:07:18.175 --> 00:07:21.956
I'm definitely looking forward to making it more useful,

00:07:21.957 --> 00:07:25.599
probably including more kinds of child frames,

00:07:25.600 --> 00:07:29.965
maybe like making the whole document an infinite canvas.

NOTE Wrapping up

00:07:29.966 --> 00:07:32.099
Alright, so that's the talk.

00:07:32.100 --> 00:07:33.346
If you're interested in the codebase,

00:07:33.347 --> 00:07:34.446
here's the homepage

00:07:34.447 --> 00:07:35.546
for the project [https://dev.lepisma.xyz/git/ob-rx].

00:07:35.547 --> 00:07:37.566
So the next steps for me are basically

00:07:37.567 --> 00:07:40.647
making my workflow easier in matplotlib,

00:07:40.648 --> 00:07:42.587
which is a Python-based library,

00:07:42.588 --> 00:07:45.348
and d3.js, which is for JavaScript.

00:07:45.349 --> 00:07:47.888
For the JS thing, I might have to add

00:07:47.889 --> 00:07:49.540
the interactive JS child frames,

00:07:49.640 --> 00:07:51.829
and I am also looking forward to building something

00:07:51.830 --> 00:07:53.969
which can replicate the work

00:07:53.970 --> 00:07:56.750
of the Observable's infinite canvas,

00:07:56.751 --> 00:07:57.490
because that's something

00:07:57.491 --> 00:08:00.619
which I found really useful in my work with

00:08:00.620 --> 00:08:02.240
just JS visualizations.

00:08:02.340 --> 00:08:05.540
So yeah, happy to take questions on Etherpad

00:08:05.560 --> 00:08:08.240
and thank you for your time.