WEBVTT

00:00:00.320 --> 00:00:03.280
um so hi I'm joe corneli

00:00:03.280 --> 00:00:05.920
uh this is work I did with ray puzio and

00:00:05.920 --> 00:00:06.879
cameron smith

00:00:06.879 --> 00:00:10.559
um and they're the main protagonists in

00:00:10.559 --> 00:00:11.599
this story they

00:00:11.599 --> 00:00:14.960
are uh researchers who've been working

00:00:14.960 --> 00:00:17.840
on theoretical biology um so in a

00:00:17.840 --> 00:00:20.160
typical project they may use manxima

00:00:20.160 --> 00:00:23.760
and julia their work for biology physics

00:00:23.760 --> 00:00:24.800
and computer science

00:00:24.800 --> 00:00:27.199
computer science and the latest work in

00:00:27.199 --> 00:00:29.439
progress is on branching processes for

00:00:29.439 --> 00:00:30.800
cancer modeling so

00:00:30.800 --> 00:00:34.719
how can um Emacs possibly help

00:00:34.719 --> 00:00:37.360
let's let's have a look uh moving code

00:00:37.360 --> 00:00:38.399
and data between these different

00:00:38.399 --> 00:00:39.680
programs by hand is

00:00:39.680 --> 00:00:42.000
annoying on separate workflows for

00:00:42.000 --> 00:00:43.200
writing up notes and preparing

00:00:43.200 --> 00:00:44.399
publications

00:00:44.399 --> 00:00:46.000
is perhaps even more annoying all of

00:00:46.000 --> 00:00:48.640
it's time consuming and error-prone

00:00:48.640 --> 00:00:52.000
um so what about maybe using jupiter

00:00:52.000 --> 00:00:53.760
uh we found something called script of

00:00:53.760 --> 00:00:55.199
scripts and it solves some of those

00:00:55.199 --> 00:00:58.399
problems because you can use

00:00:58.399 --> 00:01:01.120
maximum and julia together but we were

00:01:01.120 --> 00:01:02.640
quite happy to explore emacs based

00:01:02.640 --> 00:01:05.199
solutions being emax enthusiasts and we

00:01:05.199 --> 00:01:05.840
even

00:01:05.840 --> 00:01:07.760
got cameron to be enthusiastic about

00:01:07.760 --> 00:01:09.200
doing emacs so that

00:01:09.200 --> 00:01:11.600
went nice so just here's a little

00:01:11.600 --> 00:01:12.400
feature grid

00:01:12.400 --> 00:01:15.360
of emacs org versus just sort of your

00:01:15.360 --> 00:01:16.400
generic

00:01:16.400 --> 00:01:19.040
um tools that are in a different more

00:01:19.040 --> 00:01:20.960
general ecosystem so as you can see it's

00:01:20.960 --> 00:01:21.520
quite

00:01:21.520 --> 00:01:23.360
feature complete you've got your maximo

00:01:23.360 --> 00:01:25.520
mode julia mode you can use both of them

00:01:25.520 --> 00:01:27.119
inside of org mode

00:01:27.119 --> 00:01:28.720
you can present things with word tree

00:01:28.720 --> 00:01:30.240
slide you can set up a

00:01:30.240 --> 00:01:33.280
wiki inside of orgrome this is one I

00:01:33.280 --> 00:01:33.680
found

00:01:33.680 --> 00:01:35.759
rather recently you can even use

00:01:35.759 --> 00:01:37.759
compatibly with orgrome something called

00:01:37.759 --> 00:01:38.799
log seek

00:01:38.799 --> 00:01:41.520
which is in the browser um so that's

00:01:41.520 --> 00:01:42.159
nice

00:01:42.159 --> 00:01:44.320
um you can do real-time collaborative

00:01:44.320 --> 00:01:45.840
editing um

00:01:45.840 --> 00:01:47.840
either in a kind of pairing style or in

00:01:47.840 --> 00:01:49.280
a more etherpad style

00:01:49.280 --> 00:01:51.520
obviously you can manage your references

00:01:51.520 --> 00:01:54.159
you can typeset whatever you want um you

00:01:54.159 --> 00:01:55.759
can publish work in progress on a blog

00:01:55.759 --> 00:01:57.439
and the fern is another one of these

00:01:57.439 --> 00:01:58.159
external

00:01:58.159 --> 00:02:00.560
org mode tools it's not actually any mac

00:02:00.560 --> 00:02:01.360
but works with

00:02:01.360 --> 00:02:03.680
org mode stuff and you know so we're

00:02:03.680 --> 00:02:04.399
good to go

00:02:04.399 --> 00:02:06.640
uh with all of that so what does that

00:02:06.640 --> 00:02:08.000
look like well here's

00:02:08.000 --> 00:02:09.679
a little example from before they were

00:02:09.679 --> 00:02:11.599
doing um

00:02:11.599 --> 00:02:12.800
before we started really thinking

00:02:12.800 --> 00:02:14.560
seriously about this stuff so this is

00:02:14.560 --> 00:02:15.599
just maxima

00:02:15.599 --> 00:02:17.440
well maximo doesn't have a long running

00:02:17.440 --> 00:02:19.280
process by default if you've ever used

00:02:19.280 --> 00:02:20.160
python

00:02:20.160 --> 00:02:22.480
uh you have something called sessions uh

00:02:22.480 --> 00:02:23.920
they don't have that for maxim at least

00:02:23.920 --> 00:02:24.959
not by default

00:02:24.959 --> 00:02:27.599
um so how what was the workaround

00:02:27.599 --> 00:02:29.360
there's this thing called uh

00:02:29.360 --> 00:02:32.480
solve for you here and um

00:02:32.480 --> 00:02:34.879
that shows up down below again in these

00:02:34.879 --> 00:02:36.319
angle brackets which you've seen maybe

00:02:36.319 --> 00:02:37.360
in someone else's talk

00:02:37.360 --> 00:02:40.480
uh which means go to the previous uh

00:02:40.480 --> 00:02:41.920
thing that was named soul for you

00:02:41.920 --> 00:02:43.920
and do that all over again so they do

00:02:43.920 --> 00:02:45.280
that over again

00:02:45.280 --> 00:02:47.440
and here's the little maxima code for

00:02:47.440 --> 00:02:49.519
defining you saw so you've now defined

00:02:49.519 --> 00:02:50.560
you saw

00:02:50.560 --> 00:02:51.920
and then you can use it in the next

00:02:51.920 --> 00:02:54.000
expression you get out a nice juicy

00:02:54.000 --> 00:02:56.560
zero at the end but it's a little bit a

00:02:56.560 --> 00:02:57.200
little bit

00:02:57.200 --> 00:02:58.959
like cartridge in a pear tree to have to

00:02:58.959 --> 00:03:00.640
redefine everything every time

00:03:00.640 --> 00:03:02.560
so this is clearly at the level of work

00:03:02.560 --> 00:03:04.159
around um maybe just

00:03:04.159 --> 00:03:06.879
one more time going looking through that

00:03:06.879 --> 00:03:07.920
um

00:03:07.920 --> 00:03:11.599
that stuff um

00:03:11.599 --> 00:03:13.760
sorry so uh looking through that stuff

00:03:13.760 --> 00:03:15.280
this is we're going to need something

00:03:15.280 --> 00:03:17.200
like that probably for stitching

00:03:17.200 --> 00:03:19.599
maxima and julie julia together so it's

00:03:19.599 --> 00:03:20.319
good to

00:03:20.319 --> 00:03:21.680
look a little bit about how that might

00:03:21.680 --> 00:03:23.920
work so first of all you can cache

00:03:23.920 --> 00:03:25.680
results so if you wanted to save the

00:03:25.680 --> 00:03:26.480
date

00:03:26.480 --> 00:03:28.480
out of block one at a certain time and

00:03:28.480 --> 00:03:29.920
then use

00:03:29.920 --> 00:03:32.239
use it again later so at the time when I

00:03:32.239 --> 00:03:33.280
ran this code

00:03:33.280 --> 00:03:34.640
you can see I've got two slightly

00:03:34.640 --> 00:03:36.720
different time stamps down below one's

00:03:36.720 --> 00:03:38.560
the cached result and the other was the

00:03:38.560 --> 00:03:40.319
result of reevaluating

00:03:40.319 --> 00:03:42.640
the block so you can move things around

00:03:42.640 --> 00:03:43.760
um and that's

00:03:43.760 --> 00:03:46.000
that's going to be useful but you know

00:03:46.000 --> 00:03:47.280
that's not really the main problem the

00:03:47.280 --> 00:03:48.080
main problem is

00:03:48.080 --> 00:03:50.799
making maxima long running so kind of

00:03:50.799 --> 00:03:51.440
the

00:03:51.440 --> 00:03:53.920
core of this talk is a new observant

00:03:53.920 --> 00:03:56.400
facility which is a general purpose

00:03:56.400 --> 00:03:59.280
way to do that kind of thing which

00:03:59.280 --> 00:04:00.560
involves a very simple

00:04:00.560 --> 00:04:04.239
change to obcor uh so we'll give a quick

00:04:04.239 --> 00:04:05.360
overview of that and show

00:04:05.360 --> 00:04:08.480
an example um so here's the example

00:04:08.480 --> 00:04:11.760
um a very simple sort of silly example

00:04:11.760 --> 00:04:13.040
uh what does it mean to have a long

00:04:13.040 --> 00:04:14.640
running process here I've set this

00:04:14.640 --> 00:04:15.920
display2d

00:04:15.920 --> 00:04:18.560
to be false um which just means that

00:04:18.560 --> 00:04:19.440
things are going to come

00:04:19.440 --> 00:04:22.320
come across in 1d and then I ask it to

00:04:22.320 --> 00:04:23.040
expand

00:04:23.040 --> 00:04:25.199
uh something and I get latex by default

00:04:25.199 --> 00:04:27.280
so so that's what it means is I've sent

00:04:27.280 --> 00:04:28.639
something in and it's going to come

00:04:28.639 --> 00:04:30.240
across in one view which is great

00:04:30.240 --> 00:04:32.080
um maybe you'll also notice that there's

00:04:32.080 --> 00:04:34.560
no semicolon if you're a maxima fan

00:04:34.560 --> 00:04:36.720
um and things are coming across as tech

00:04:36.720 --> 00:04:38.400
so those were some little bonus features

00:04:38.400 --> 00:04:40.320
and I'll show you how that works later

00:04:40.320 --> 00:04:41.040
so

00:04:41.040 --> 00:04:45.440
um the change to obcor is as follows uh

00:04:45.440 --> 00:04:48.880
um we uh actually this should say uh

00:04:48.880 --> 00:04:51.520
instead of stream here it should say um

00:04:51.520 --> 00:04:52.479
servant

00:04:52.479 --> 00:04:54.800
sorry uh we tried an experimental

00:04:54.800 --> 00:04:56.160
version what's called stream so now it's

00:04:56.160 --> 00:04:58.160
called servant but all it does is it

00:04:58.160 --> 00:05:01.520
overrides uh or babel execute laying for

00:05:01.520 --> 00:05:02.639
arbitrary laying

00:05:02.639 --> 00:05:05.919
if you have um a servant

00:05:05.919 --> 00:05:07.840
in your in your params so that's the

00:05:07.840 --> 00:05:09.759
change that hasn't been

00:05:09.759 --> 00:05:11.919
pushed out or sent as a patch to anybody

00:05:11.919 --> 00:05:13.759
but it's a pretty minor change

00:05:13.759 --> 00:05:16.960
um here's an overview without the code

00:05:16.960 --> 00:05:19.080
of just a high level overview of

00:05:19.080 --> 00:05:20.720
observant.el so

00:05:20.720 --> 00:05:22.160
it stores information about these

00:05:22.160 --> 00:05:24.479
processes in a hash table

00:05:24.479 --> 00:05:26.080
it can do pre-processing and

00:05:26.080 --> 00:05:27.600
post-processing

00:05:27.600 --> 00:05:29.759
um it does all these things it stores

00:05:29.759 --> 00:05:30.720
the output

00:05:30.720 --> 00:05:32.479
I mentioned here that in principle we

00:05:32.479 --> 00:05:34.080
could store lots of output and have a

00:05:34.080 --> 00:05:35.280
kind of browsable

00:05:35.280 --> 00:05:36.560
history although we don't do that

00:05:36.560 --> 00:05:38.880
presently um but that's what observant

00:05:38.880 --> 00:05:40.639
does is it does what you might expect

00:05:40.639 --> 00:05:41.440
and here's the

00:05:41.440 --> 00:05:45.440
here's the maxima kind of um

00:05:45.440 --> 00:05:48.160
on-ramp uh to get maxima brought in so

00:05:48.160 --> 00:05:49.600
you have to obviously have a maximum

00:05:49.600 --> 00:05:51.360
process you can call

00:05:51.360 --> 00:05:54.960
um put hash uh this is the preprocessing

00:05:54.960 --> 00:05:57.840
thing I mentioned adding in some tech um

00:05:57.840 --> 00:05:58.960
and adding in

00:05:58.960 --> 00:06:01.520
uh or deleting rather a substring and

00:06:01.520 --> 00:06:03.120
here here is why you delete the

00:06:03.120 --> 00:06:03.759
substring

00:06:03.759 --> 00:06:06.960
is because um maxima thinks it's a good

00:06:06.960 --> 00:06:08.240
idea to tell you false

00:06:08.240 --> 00:06:10.080
once you once you run check on things

00:06:10.080 --> 00:06:11.759
you've got to delete that back out to

00:06:11.759 --> 00:06:13.680
get something kind of coherent out of it

00:06:13.680 --> 00:06:16.960
but so this is how to set up maximal um

00:06:16.960 --> 00:06:19.280
that's enough really of the demo is not

00:06:19.280 --> 00:06:20.000
really a demos for

00:06:20.000 --> 00:06:21.919
show and tell but uh this is an

00:06:21.919 --> 00:06:23.600
experience report I wanted to talk about

00:06:23.600 --> 00:06:25.440
the experience of doing this

00:06:25.440 --> 00:06:28.080
so some some negatives like we tried to

00:06:28.080 --> 00:06:30.160
get emacs jupiter working

00:06:30.160 --> 00:06:33.199
prior to prior to working on the um

00:06:33.199 --> 00:06:36.000
observant and we couldn't get it doing

00:06:36.000 --> 00:06:37.919
everything we wanted despite a bit of

00:06:37.919 --> 00:06:40.160
heavy lifting and debugging and stuff so

00:06:40.160 --> 00:06:41.840
that's not not finished that was a bit

00:06:41.840 --> 00:06:42.880
difficult

00:06:42.880 --> 00:06:45.360
um on the other hand working on

00:06:45.360 --> 00:06:47.759
observing was fun and pretty lightweight

00:06:47.759 --> 00:06:48.479
and easy

00:06:48.479 --> 00:06:50.400
um we got some experience co-editing

00:06:50.400 --> 00:06:52.400
things with these real-time tools

00:06:52.400 --> 00:06:54.479
obviously the stack is somewhat work in

00:06:54.479 --> 00:06:55.919
progress um

00:06:55.919 --> 00:06:58.000
so I just wanted to give a shout out to

00:06:58.000 --> 00:07:00.800
crdt which was really fun

00:07:00.800 --> 00:07:03.919
and champion was making bug fiction

00:07:03.919 --> 00:07:07.039
bug fixes for that as we go similarly

00:07:07.039 --> 00:07:08.960
for fern and log seek the maintainers

00:07:08.960 --> 00:07:10.160
were really responsive

00:07:10.160 --> 00:07:12.960
um and so that was nice we did try to

00:07:12.960 --> 00:07:14.560
get emacs running in the browser

00:07:14.560 --> 00:07:15.840
thinking it would be really nice for

00:07:15.840 --> 00:07:16.880
people who

00:07:16.880 --> 00:07:19.520
um didn't want to install it to get a

00:07:19.520 --> 00:07:21.120
chance to just try it

00:07:21.120 --> 00:07:23.120
uh but actually browsers capture things

00:07:23.120 --> 00:07:24.479
like uh control n

00:07:24.479 --> 00:07:27.120
and so that's that was a bit annoying

00:07:27.120 --> 00:07:28.479
but we did get lots of great feedback

00:07:28.479 --> 00:07:29.759
and interaction with people including

00:07:29.759 --> 00:07:31.599
around this conference so thank you

00:07:31.599 --> 00:07:33.759
to those who we've had discussions with

00:07:33.759 --> 00:07:35.599
um

00:07:35.599 --> 00:07:37.680
so future work okay so maybe you

00:07:37.680 --> 00:07:38.639
remember I didn't

00:07:38.639 --> 00:07:41.039
give a talk a few years back on arcana

00:07:41.039 --> 00:07:42.400
so what what

00:07:42.400 --> 00:07:44.479
might this have to do with org mode and

00:07:44.479 --> 00:07:45.919
that's always the question one asks

00:07:45.919 --> 00:07:47.039
about arcata

00:07:47.039 --> 00:07:48.720
well so arcana one of the things it does

00:07:48.720 --> 00:07:50.319
is transclusions and so that could be

00:07:50.319 --> 00:07:51.680
actually very helpful

00:07:51.680 --> 00:07:54.000
in connection with this combined notes

00:07:54.000 --> 00:07:55.520
and write-up workflow so you might have

00:07:55.520 --> 00:07:56.479
a

00:07:56.479 --> 00:07:58.400
forward mode uh and some of these

00:07:58.400 --> 00:08:00.800
results we got back as raw results

00:08:00.800 --> 00:08:03.199
could go right into your write up in a

00:08:03.199 --> 00:08:03.919
kind of

00:08:03.919 --> 00:08:07.520
convenient way at a level above um

00:08:07.520 --> 00:08:09.039
transparently a level above the notebook

00:08:09.039 --> 00:08:11.280
so you'd have the notebook alongside the

00:08:11.280 --> 00:08:13.440
write-up in that case

00:08:13.440 --> 00:08:15.599
which is a sort of variation on the

00:08:15.599 --> 00:08:16.800
literate programming

00:08:16.800 --> 00:08:19.120
workflow this is speculative who knows

00:08:19.120 --> 00:08:20.000
the other thought is

00:08:20.000 --> 00:08:22.080
it just relates to the idea of network

00:08:22.080 --> 00:08:23.520
programming so we can imagine these

00:08:23.520 --> 00:08:24.800
networks of

00:08:24.800 --> 00:08:26.639
computational nodes maybe sitting inside

00:08:26.639 --> 00:08:28.800
of orgrome calling each other

00:08:28.800 --> 00:08:31.199
um and you would want to maintain some

00:08:31.199 --> 00:08:32.560
kind of model of that

00:08:32.560 --> 00:08:34.959
process and a sort of general question

00:08:34.959 --> 00:08:36.640
is how do we have a remote control for

00:08:36.640 --> 00:08:38.320
long-running processes you could do that

00:08:38.320 --> 00:08:40.240
in lisp or closure but

00:08:40.240 --> 00:08:42.479
maybe we could have something a little

00:08:42.479 --> 00:08:44.080
bit like that here

00:08:44.080 --> 00:08:45.839
um so conclusions what have we actually

00:08:45.839 --> 00:08:48.080
addressed well uh we addressed uh

00:08:48.080 --> 00:08:50.080
accessing any long-running process with

00:08:50.080 --> 00:08:51.600
a simple or mode interface

00:08:51.600 --> 00:08:52.959
obviously we're not the only people to

00:08:52.959 --> 00:08:54.880
think about notebooks but we think that

00:08:54.880 --> 00:08:56.880
emacs has some advantages

00:08:56.880 --> 00:08:58.880
related to reproducible research and

00:08:58.880 --> 00:09:01.120
interdisciplinary collaboration so let's

00:09:01.120 --> 00:09:01.680
just

00:09:01.680 --> 00:09:03.120
say that we think something is

00:09:03.120 --> 00:09:04.880
reproducible if it's actually teachable

00:09:04.880 --> 00:09:07.200
to someone new and they can do it

00:09:07.200 --> 00:09:09.279
org mode seems very useful for that many

00:09:09.279 --> 00:09:11.680
of the other talks have touched on this

00:09:11.680 --> 00:09:14.399
interdisciplinary collaboration is great

00:09:14.399 --> 00:09:16.000
this was an interdisciplinary

00:09:16.000 --> 00:09:17.839
collaboration on some level but

00:09:17.839 --> 00:09:19.680
what about you know future work for

00:09:19.680 --> 00:09:21.600
bringing in scenario planners simulation

00:09:21.600 --> 00:09:22.320
scientists

00:09:22.320 --> 00:09:24.480
and local farmers and building something

00:09:24.480 --> 00:09:25.519
that they can all

00:09:25.519 --> 00:09:27.200
use that's more than the sum of the

00:09:27.200 --> 00:09:29.200
parts so a little

00:09:29.200 --> 00:09:30.720
future work for everybody else here so

00:09:30.720 --> 00:09:32.160
we think science should be widely

00:09:32.160 --> 00:09:33.120
teachable

00:09:33.120 --> 00:09:34.560
shareable semi-automated

00:09:34.560 --> 00:09:36.720
transdisciplinary and real time

00:09:36.720 --> 00:09:40.399
like EmacsConf so you can get in touch

00:09:40.399 --> 00:09:43.279
uh via these methods and the code which

00:09:43.279 --> 00:09:44.560
is very much early stage work in

00:09:44.560 --> 00:09:45.200
progress

00:09:45.200 --> 00:09:46.640
as this was meant to be an experience

00:09:46.640 --> 00:09:48.800
report not a it's all done here it is

00:09:48.800 --> 00:09:49.920
polished report

00:09:49.920 --> 00:09:51.680
um it's also online if you'd like to

00:09:51.680 --> 00:09:53.360
have a look so that's the end of the

00:09:53.360 --> 00:09:54.560
talk and I don't know if there's time

00:09:54.560 --> 00:09:55.279
for questions

00:09:55.279 --> 00:09:57.440
or not but um I'm at your disposal now

00:09:57.440 --> 00:10:00.240
thank you you are now unmuted

00:10:00.240 --> 00:10:04.079
um many thanks for the tough job

00:10:04.079 --> 00:10:07.120
uh let's see we have about I think four

00:10:07.120 --> 00:10:08.880
minutes four questions

00:10:08.880 --> 00:10:10.880
um and we have a couple of questions on

00:10:10.880 --> 00:10:12.320
the pad would you like to read them

00:10:12.320 --> 00:10:14.240
yourself or should I read them to you

00:10:14.240 --> 00:10:16.560
uh just for sake of easy management why

00:10:16.560 --> 00:10:18.079
don't you read them out if that's okay

00:10:18.079 --> 00:10:21.279
yeah sure uh so they ask have you looked

00:10:21.279 --> 00:10:22.240
into trying

00:10:22.240 --> 00:10:25.440
sage math I've long uh long wanted

00:10:25.440 --> 00:10:33.760
nothing org files

00:10:33.760 --> 00:10:36.959
right and I and I wrote the answer that

00:10:36.959 --> 00:10:39.279
we should be possible because one can

00:10:39.279 --> 00:10:44.839
call it from a command

00:10:44.839 --> 00:10:46.399
okay

00:10:46.399 --> 00:10:48.640
and I see there's another sagemath

00:10:48.640 --> 00:10:50.079
question that you seem to have answered

00:10:50.079 --> 00:10:52.100
as well so I guess I won't repeat that

00:10:52.100 --> 00:10:54.880
[Music]

00:10:54.880 --> 00:10:56.959
there's let's not forget about embedded

00:10:56.959 --> 00:11:00.640
calc in emacs

00:11:00.640 --> 00:11:03.680
so so the first demos actually were with

00:11:03.680 --> 00:11:05.040
calc so that's that's

00:11:05.040 --> 00:11:06.640
useful uh although I think it was a

00:11:06.640 --> 00:11:07.839
different kind of a different command

00:11:07.839 --> 00:11:08.240
line

00:11:08.240 --> 00:11:11.839
well that was the next calc yeah

00:11:11.839 --> 00:11:13.839
so sure there is there is calc so that

00:11:13.839 --> 00:11:15.680
would be calculus and

00:11:15.680 --> 00:11:19.120
calc is already in org mode yeah

00:11:19.120 --> 00:11:25.680
yeah yeah

00:11:25.680 --> 00:11:39.760
um still looking for questions

00:11:39.760 --> 00:11:41.760
okay I think that's about it I don't see

00:11:41.760 --> 00:11:44.320
any questions on the ether pad

00:11:44.320 --> 00:11:47.440
and let's see

00:11:47.440 --> 00:11:53.040
anything on irc

00:11:53.040 --> 00:11:55.760
um nothing but phrases and everyone

00:11:55.760 --> 00:11:56.720
thanking you

00:11:56.720 --> 00:11:59.120
thank you all right you're welcome

00:11:59.120 --> 00:12:00.240
thanks a lot uh

00:12:00.240 --> 00:12:02.880
we'll see you guys around then cheers

00:12:02.880 --> 00:12:06.800
and see you around