WEBVTT
00:00:00.320 --> 00:00:03.280
Joe: Hi, I'm Joe Corneli.
00:00:03.280 --> 00:00:06.879
This is work I did with Ray Puzio and
Cameron Smith.
00:00:06.879 --> 00:00:11.123
They're the main protagonists in
this story.
00:00:11.123 --> 00:00:14.960
They are researchers who've been working
00:00:14.960 --> 00:00:17.490
on theoretical biology.
00:00:17.490 --> 00:00:21.357
In a typical project, they may use
Maxima and Julia.
00:00:21.357 --> 00:00:24.800
Their work combines biology, physics and
computer science.
00:00:24.800 --> 00:00:29.439
The latest work-in-progress is on
branching processes for
00:00:29.439 --> 00:00:30.800
cancer modeling.
00:00:30.800 --> 00:00:34.719
How can Emacs possibly help?
00:00:34.719 --> 00:00:37.360
Let's have a look. Moving code
00:00:37.360 --> 00:00:38.399
and data between these different
00:00:38.399 --> 00:00:39.680
programs by hand is
00:00:39.680 --> 00:00:42.000
annoying. Separate workflows for
00:00:42.000 --> 00:00:44.399
writing up notes and preparing
publications
00:00:44.399 --> 00:00:46.000
is perhaps even more annoying. All of
00:00:46.000 --> 00:00:48.640
it is time consuming and error-prone.
00:00:48.640 --> 00:00:52.000
So what about maybe using Jupyter?
00:00:52.000 --> 00:00:53.760
We found something called Script of
00:00:53.760 --> 00:00:55.199
Scripts. It solves some of those
00:00:55.199 --> 00:00:58.399
problems because you can use
00:00:58.399 --> 00:01:01.120
Maxima and Julia together, but we were
00:01:01.120 --> 00:01:02.640
quite happy to explore Emacs-based
00:01:02.640 --> 00:01:04.890
solutions, being Emcas enthusiasts
00:01:04.890 --> 00:01:07.760
We even got Cameron to be
enthusiastic about
00:01:07.760 --> 00:01:10.057
doing Emacs, so that went nice.
00:01:10.057 --> 00:01:12.400
Here's a little feature grid
00:01:12.400 --> 00:01:15.360
of Emacs + Org versus your
00:01:15.360 --> 00:01:18.390
generic tools that are in a
00:01:18.390 --> 00:01:20.123
different, more general ecosystem.
00:01:20.123 --> 00:01:21.520
As you can see, it's quite
00:01:21.520 --> 00:01:22.957
feature-complete. You've got your
00:01:22.957 --> 00:01:25.520
maxima-mode, julia-mode. You can use both of them
00:01:25.520 --> 00:01:27.119
inside of org-mode.
00:01:27.119 --> 00:01:28.720
You can present things with org-tree-slide.
00:01:28.720 --> 00:01:31.490
You can set up a wiki inside of org-roam.
00:01:31.490 --> 00:01:33.680
This is one I found
00:01:33.680 --> 00:01:35.759
rather recently. You can even use
00:01:35.759 --> 00:01:37.759
compatibly with org-roam, something called
00:01:37.759 --> 00:01:42.159
logseq, which is in the browser, so that's
nice.
00:01:42.159 --> 00:01:44.957
You can do real-time collaborative
editing,
00:01:44.957 --> 00:01:47.657
either in a kind of pairing style
00:01:47.657 --> 00:01:49.280
or in a more Etherpad style.
00:01:49.280 --> 00:01:51.520
Obviously, you can manage your references.
00:01:51.520 --> 00:01:53.457
You can typeset whatever you want.
00:01:53.457 --> 00:01:55.759
You can publish work in progress on
a blog.
00:01:55.759 --> 00:01:57.439
Firn is another one of these
00:01:57.439 --> 00:01:59.790
external Org Mode tools.
00:01:59.790 --> 00:02:00.560
It's not actually in Emacs,
00:02:00.560 --> 00:02:03.557
but works with Org Mode stuff. And, you know...
00:02:03.557 --> 00:02:05.657
So we're good to go with all of that.
00:02:05.657 --> 00:02:07.423
So what does that look like? Well,
00:02:07.423 --> 00:02:09.423
here's a little example from before
00:02:09.423 --> 00:02:12.800
they were doing... before we started
really thinking
00:02:12.800 --> 00:02:13.890
seriously about this stuff.
00:02:13.890 --> 00:02:15.599
So this is just Maxima.
00:02:15.599 --> 00:02:17.440
Well, Maxima doesn't have a long running
00:02:17.440 --> 00:02:19.280
process by default. If you've ever used
00:02:19.280 --> 00:02:22.480
Python, you have something called
sessions.
00:02:22.480 --> 00:02:23.920
They don't have that for Maxima, at least
00:02:23.920 --> 00:02:24.959
not by default.
00:02:24.959 --> 00:02:27.599
So how... What was the workaround?
00:02:27.599 --> 00:02:29.360
There's this thing called
00:02:29.360 --> 00:02:32.480
solve-for-u here
00:02:32.480 --> 00:02:34.879
that shows up down below again in these
00:02:34.879 --> 00:02:36.319
angle brackets, which you've seen maybe
00:02:36.319 --> 00:02:37.360
in someone else's talk,
00:02:37.360 --> 00:02:40.480
which means go to the previous
00:02:40.480 --> 00:02:41.920
thing that was named solve-for-u
00:02:41.920 --> 00:02:43.920
and do that all over again, so they do
00:02:43.920 --> 00:02:45.280
that over again.
00:02:45.280 --> 00:02:47.440
Here's the little Maxima code for
00:02:47.440 --> 00:02:48.990
defining usol, so you've now
00:02:48.990 --> 00:02:51.723
defined usol, and then you can use it
00:02:51.723 --> 00:02:54.000
in the next expression. You get out a
nice juicy
00:02:54.000 --> 00:02:56.560
zero at the end. It's a little bit
00:02:57.200 --> 00:02:58.959
like a partridge in a pear tree to have to
00:02:58.959 --> 00:03:00.640
redefine everything every time.
00:03:00.640 --> 00:03:02.490
So this is clearly at the level of
00:03:02.490 --> 00:03:04.159
work-around. Maybe just
00:03:04.159 --> 00:03:06.223
one more time looking through
00:03:06.223 --> 00:03:11.599
that stuff.
00:03:11.599 --> 00:03:13.760
Sorry. So, looking through that stuff,
00:03:13.760 --> 00:03:15.280
this is... We're going to need something
00:03:15.280 --> 00:03:17.200
like that, probably, for stitching
00:03:17.200 --> 00:03:19.599
Maxima and Julia together. so it's
00:03:19.599 --> 00:03:20.890
good to look a little bit
00:03:20.890 --> 00:03:22.590
about how that might work.
00:03:22.590 --> 00:03:23.920
First of all, you can cache
00:03:23.920 --> 00:03:25.680
results, so if you wanted to save the
00:03:25.680 --> 00:03:28.480
date out of block one at a certain
time and
00:03:28.480 --> 00:03:30.990
then use it again later...
00:03:30.990 --> 00:03:33.280
At the time when I ran this code,
00:03:33.280 --> 00:03:34.640
you can see I've got two slightly
00:03:34.640 --> 00:03:36.623
different time stamps down below.
00:03:36.623 --> 00:03:38.323
One's the cached result, and the other
00:03:38.323 --> 00:03:40.319
was the result of reevaluating
00:03:40.319 --> 00:03:42.640
the block. So you can move things around.
00:03:43.760 --> 00:03:46.000
That's going to be useful. But you know,
00:03:46.000 --> 00:03:46.923
that's not really the main problem.
00:03:46.923 --> 00:03:48.080
The main problem is
00:03:48.080 --> 00:03:50.799
making Maxima long-running.
00:03:50.799 --> 00:03:53.920
The core of this talk is a new observant
00:03:53.920 --> 00:03:56.400
facility, which is a general purpose
00:03:56.400 --> 00:03:59.280
way to do that kind of thing, which
00:03:59.280 --> 00:04:01.823
involves a very simple change to ob-core.
00:04:01.823 --> 00:04:04.239
We'll give a quick
00:04:04.239 --> 00:04:05.360
overview of that and show
00:04:05.360 --> 00:04:08.480
an example. So here's the example,
00:04:08.480 --> 00:04:11.760
a very simple sort of silly example.
00:04:11.760 --> 00:04:12.957
What does it mean to have
00:04:12.957 --> 00:04:13.990
a long-running process?
00:04:13.990 --> 00:04:14.640
Here, I've set this
00:04:14.640 --> 00:04:18.560
display2d to be false, which just
means that
00:04:18.560 --> 00:04:19.440
things are going to come
00:04:19.440 --> 00:04:22.320
come across in 1d. Then I ask it to
00:04:22.320 --> 00:04:25.199
expand something. I get LaTeX
by default.
00:04:25.199 --> 00:04:27.280
So that's what it means. It's that I've sent
00:04:27.280 --> 00:04:28.639
something in and it's going to come
00:04:28.639 --> 00:04:30.240
across in one view, which is great.
00:04:30.240 --> 00:04:32.080
Maybe you'll also notice that there's
00:04:32.080 --> 00:04:34.560
no semicolon, if you're a Maxima fan,
00:04:34.560 --> 00:04:36.720
and things are coming across as TeX.
00:04:36.720 --> 00:04:38.400
So those were some little bonus features.
00:04:38.400 --> 00:04:40.320
I'll show you how that works later.
00:04:41.040 --> 00:04:45.440
The change to ob-core is as follows.
00:04:45.440 --> 00:04:48.880
Actually, this should say...
00:04:48.880 --> 00:04:51.520
Instead of stream here, it should say
00:04:51.520 --> 00:04:54.800
servant. Sorry. We tried an experimental
00:04:54.800 --> 00:04:56.160
version which was called stream, so now it's
00:04:56.160 --> 00:04:58.160
called servant. But all it does is it
00:04:58.160 --> 00:05:01.520
overrides org-babel-execute lang for
00:05:01.520 --> 00:05:02.639
arbitrary lang
00:05:02.639 --> 00:05:05.919
if you have a servant
00:05:05.919 --> 00:05:07.840
in your params. So that's the
00:05:07.840 --> 00:05:09.759
change that hasn't been
00:05:09.759 --> 00:05:11.919
pushed out or sent as a patch to anybody,
00:05:11.919 --> 00:05:13.759
but it's a pretty minor change.
00:05:13.759 --> 00:05:16.960
Here's an overview without the code.
00:05:16.960 --> 00:05:19.080
Just a high level overview of
00:05:19.080 --> 00:05:20.720
observant.el.
00:05:20.720 --> 00:05:22.160
It stores information about these
00:05:22.160 --> 00:05:24.479
processes in a hash table.
00:05:24.479 --> 00:05:26.080
It can do pre-processing and
00:05:26.080 --> 00:05:27.600
post-processing.
00:05:27.600 --> 00:05:29.759
It does all these things. It stores
00:05:29.759 --> 00:05:30.720
the output.
00:05:30.720 --> 00:05:32.479
I mentioned here that, in principle, we
00:05:32.479 --> 00:05:34.080
could store lots of output and have a
00:05:34.080 --> 00:05:35.657
kind of browsable history,
00:05:35.657 --> 00:05:37.790
although we don't do that
presently.
00:05:37.790 --> 00:05:38.790
But that's what observant does.
00:05:38.790 --> 00:05:40.639
It does what you might expect.
00:05:41.440 --> 00:05:46.190
Here's the Maxima on-ramp
00:05:46.190 --> 00:05:48.160
to get Maxima brought in.
00:05:48.160 --> 00:05:49.257
You have to obviously have
00:05:49.257 --> 00:05:51.360
a Maxima process you can call.
00:05:51.360 --> 00:05:54.960
puthash... this is the preprocessing
00:05:54.960 --> 00:05:57.840
thing I mentioned, adding in some Tex
00:05:57.840 --> 00:05:58.960
and adding in--
00:05:58.960 --> 00:06:01.520
or deleting, rather--a substring.
00:06:01.520 --> 00:06:03.759
Here is why you delete the
substring.
00:06:03.759 --> 00:06:06.960
It's because Maxima thinks it's a good
00:06:06.960 --> 00:06:08.240
idea to tell you false
00:06:08.240 --> 00:06:10.080
once you run check on things.
00:06:10.080 --> 00:06:11.759
You've got to delete that back out to
00:06:11.759 --> 00:06:13.680
get something coherent out of it.
00:06:13.680 --> 00:06:16.960
So this is how to set up Maxima.
00:06:16.960 --> 00:06:19.157
That's enough, really, of the demo.
00:06:19.157 --> 00:06:20.000
It's not really a demo for
00:06:20.000 --> 00:06:21.919
show and tell, but as this is an
00:06:21.919 --> 00:06:23.600
experience report, I wanted to talk about
00:06:23.600 --> 00:06:25.440
the experience of doing this.
00:06:25.440 --> 00:06:28.080
Some negatives, like we tried to
00:06:28.080 --> 00:06:30.160
get Emacs Jupyter working
00:06:30.160 --> 00:06:34.190
prior to working on observant.
00:06:34.190 --> 00:06:36.000
We couldn't get it doing
00:06:36.000 --> 00:06:37.919
everything we wanted, despite a bit of
00:06:37.919 --> 00:06:40.160
heavy lifting and debugging and stuff.
00:06:40.160 --> 00:06:42.880
So that's not finished. That was a bit
difficult.
00:06:42.880 --> 00:06:45.360
On the other hand, working on
00:06:45.360 --> 00:06:47.759
observant was fun, pretty lightweight,
00:06:47.759 --> 00:06:48.479
and easy.
00:06:48.479 --> 00:06:50.400
We got some experience co-editing
00:06:50.400 --> 00:06:52.400
things with these real-time tools.
00:06:52.400 --> 00:06:55.919
Obviously, the stack is somewhat work in
progress.
00:06:55.919 --> 00:06:58.000
I just wanted to give a shout out to
00:06:58.000 --> 00:07:00.800
crdt which was really fun,
00:07:00.800 --> 00:07:03.919
and Qiantan was making
00:07:03.919 --> 00:07:06.057
bug fixes for that as we go.
00:07:06.057 --> 00:07:08.960
Similarly, for firn and logseq,
the maintainers
00:07:08.960 --> 00:07:10.160
were really responsive,
00:07:10.160 --> 00:07:12.960
so that was nice. We did try to
00:07:12.960 --> 00:07:14.560
get Emacs running in the browser,
00:07:14.560 --> 00:07:15.840
thinking it would be really nice for
00:07:15.840 --> 00:07:19.290
people who didn't want to install it
00:07:19.290 --> 00:07:21.120
to get a chance to just try it,
00:07:21.120 --> 00:07:23.120
but actually, browsers capture things
00:07:23.120 --> 00:07:27.120
like C-n, so that was a bit annoying.
00:07:27.120 --> 00:07:28.479
But we did get lots of great feedback
00:07:28.479 --> 00:07:29.759
and interaction with people, including
00:07:29.759 --> 00:07:31.599
around this conference. Thank you
00:07:31.599 --> 00:07:33.759
to those who we've had discussions with.
00:07:35.599 --> 00:07:37.680
So, future work. Okay, so... Maybe you
00:07:37.680 --> 00:07:41.039
remember, I gave a talk a few years back
on Arxana.
00:07:41.039 --> 00:07:44.479
What might this have to do with Org
Mode?
00:07:44.479 --> 00:07:45.919
That's always the question one asks
00:07:45.919 --> 00:07:47.039
about Arxana.
00:07:47.039 --> 00:07:48.720
Arxana... One of the things it does
00:07:48.720 --> 00:07:50.319
is transclusions, and so that could be
00:07:50.319 --> 00:07:51.680
actually very helpful
00:07:51.680 --> 00:07:54.000
in connection with this "combined notes
00:07:54.000 --> 00:07:55.520
and write-up" workflow. So you might have
00:07:55.520 --> 00:07:58.400
an Org Mode. Some of these
00:07:58.400 --> 00:08:00.800
results we got back as raw results
00:08:00.800 --> 00:08:03.199
could go right into your write-up in a
00:08:03.919 --> 00:08:07.520
convenient way, at a level above--
00:08:07.520 --> 00:08:09.039
transparently, a level above the notebook.
00:08:09.039 --> 00:08:11.280
So you'd have the notebook alongside the
00:08:11.280 --> 00:08:13.440
write-up in that case,
00:08:13.440 --> 00:08:15.599
which is a variation on the
00:08:15.599 --> 00:08:17.423
literate programming workflow.
00:08:17.423 --> 00:08:19.120
This is speculative. Who knows?
00:08:19.120 --> 00:08:20.000
The other thought is,
00:08:20.000 --> 00:08:22.080
it just relates to the idea of network
00:08:22.080 --> 00:08:23.520
programming. So we can imagine these
00:08:23.520 --> 00:08:25.990
networks of computational nodes
00:08:25.990 --> 00:08:27.623
sitting inside of org-roam,
00:08:27.623 --> 00:08:28.800
calling each other.
00:08:28.800 --> 00:08:31.199
You would want to maintain some
00:08:31.199 --> 00:08:33.357
kind of model of that process.
00:08:33.357 --> 00:08:36.640
A general question is: how do we have a
remote control for
00:08:36.640 --> 00:08:37.957
long-running processes?
00:08:37.957 --> 00:08:38.320
You could do that
00:08:38.320 --> 00:08:39.857
in Lisp or Clojure,
00:08:39.857 --> 00:08:41.657
but maybe we could have something
00:08:41.657 --> 00:08:44.080
a little bit like that here.
00:08:44.080 --> 00:08:45.839
Conclusions: what have we actually
00:08:45.839 --> 00:08:48.080
addressed? Well, we addressed
00:08:48.080 --> 00:08:50.080
accessing any long-running process with
00:08:50.080 --> 00:08:51.600
a simple Org Mode interface.
00:08:51.600 --> 00:08:52.959
Obviously, we're not the only people to
00:08:52.959 --> 00:08:54.880
think about notebooks, but we think that
00:08:54.880 --> 00:08:56.880
Emacs has some advantages
00:08:56.880 --> 00:08:58.880
related to reproducible research and
00:08:58.880 --> 00:09:00.757
interdisciplinary collaboration.
00:09:00.757 --> 00:09:02.590
Let's just say that we think
00:09:02.590 --> 00:09:04.880
something is reproducible if it's
actually teachable
00:09:04.880 --> 00:09:07.200
to someone new and they can do it.
00:09:07.200 --> 00:09:09.190
Org Mode seems very useful for that.
00:09:09.190 --> 00:09:11.680
Many of the other talks have touched
on this.
00:09:11.680 --> 00:09:14.399
Interdisciplinary collaboration is great.
00:09:14.399 --> 00:09:16.000
This was an interdisciplinary
00:09:16.000 --> 00:09:17.839
collaboration on some level, but
00:09:17.839 --> 00:09:19.680
what about future work for
00:09:19.680 --> 00:09:21.190
bringing in scenario planners,
00:09:21.190 --> 00:09:22.320
simulation scientists,
00:09:22.320 --> 00:09:24.480
and local farmers, and building something
00:09:24.480 --> 00:09:26.023
that they can all use
00:09:26.023 --> 00:09:27.857
that's more than the sum of the
parts?
00:09:27.857 --> 00:09:30.720
So a little future work for everybody
else here.
00:09:30.720 --> 00:09:32.160
We think science should be widely
00:09:32.160 --> 00:09:34.560
teachable, shareable, semi-automated,
00:09:34.560 --> 00:09:36.720
transdisciplinary, and real-time
00:09:36.720 --> 00:09:40.399
like EmacsConf. So you can get in touch
00:09:40.399 --> 00:09:43.279
via these methods. The code--which
00:09:43.279 --> 00:09:45.200
is very much early stage work in
progress,
00:09:45.200 --> 00:09:46.640
as this was meant to be an experience
00:09:46.640 --> 00:09:48.800
report, not a "it's all done, here, it is
00:09:48.800 --> 00:09:49.920
polished" report--
00:09:49.920 --> 00:09:51.680
it's also online if you'd like to
00:09:51.680 --> 00:09:53.360
have a look. That's the end of the
00:09:53.360 --> 00:09:54.560
talk. I don't know if there's time
00:09:54.560 --> 00:09:57.440
for questions or not, but um I'm at your
disposal now.
00:09:57.440 --> 00:10:00.240
Thank you.
00:10:00.240 --> 00:10:04.079
(Amin: Many thanks for the tough job.
00:10:04.079 --> 00:10:07.120
Let's see. We have about I think four
00:10:07.120 --> 00:10:08.880
minutes for questions,
00:10:08.880 --> 00:10:10.880
and we have a couple of questions on
00:10:10.880 --> 00:10:12.320
the pad. Would you like to read them
00:10:12.320 --> 00:10:14.240
yourself or should I read them to you?)
00:10:14.240 --> 00:10:16.423
Just for the sake of easy management
00:10:16.423 --> 00:10:18.079
why don't you read them out, if
that's okay?
00:10:18.079 --> 00:10:21.279
(Amin: yeah, sure. They ask, "Have you looked
00:10:21.279 --> 00:10:25.440
into trying Sage Math? I've long wanted
00:10:25.440 --> 00:10:33.760
to use Sage Math in Org files.")
00:10:33.760 --> 00:10:36.959
Ray: Right. I wrote the answer that
00:10:36.959 --> 00:10:39.279
it should be possible because one can
00:10:39.279 --> 00:10:44.839
call it from a command.
00:10:44.839 --> 00:10:48.190
(Amin: okay, and I see there's
00:10:48.190 --> 00:10:50.079
another Sage Math question that you seem
to have answered
00:10:50.079 --> 00:10:52.100
as well, so I guess I won't repeat that.
00:10:54.880 --> 00:10:56.959
There's... "Let's not forget about embedded
00:10:56.959 --> 00:11:00.640
Calc in Emacs.")
00:11:00.640 --> 00:11:05.040
Joe: So the first demos actually were with
Calc.
00:11:05.040 --> 00:11:06.640
That's useful. Although I think it was a
00:11:06.640 --> 00:11:08.240
different--kind of a different command
line.
00:11:08.240 --> 00:11:11.839
Ray: Well, that was UNIX Calc.
00:11:11.839 --> 00:11:13.839
Joe: So, sure, there is calc, so that...
00:11:15.680 --> 00:11:19.120
Ray: Calc is already in Org Mode.
00:11:25.680 --> 00:11:39.760
(Amin: Still looking for questions.
00:11:39.760 --> 00:11:41.760
Okay, I think that's about it. I don't see
00:11:41.760 --> 00:11:44.320
any questions on the Etherpad.
00:11:44.320 --> 00:11:47.440
And let's see...
00:11:47.440 --> 00:11:53.040
Anything on irc?
00:11:53.040 --> 00:11:55.760
Nothing but praises and everyone
00:11:55.760 --> 00:11:57.290
thanking you. Thank you.)
00:11:57.290 --> 00:11:59.120
Ray: all right, you're welcome.
00:11:59.120 --> 00:12:00.240
Joe: Thanks a lot!
00:12:00.240 --> 00:12:01.923
We'll see you guys around then.
00:12:01.923 --> 00:12:06.800
Amin: Cheers, and see you around!
