0:00:00.320,0:00:05.920
um so hi i'm joe corneli
0:00:03.280,0:00:06.879
uh this is work i did with ray puzio and
0:00:05.920,0:00:10.559
cameron smith
0:00:06.879,0:00:11.599
um and they're the main protagonists in
0:00:10.559,0:00:14.960
this story they
0:00:11.599,0:00:17.840
are uh researchers who've been working
0:00:14.960,0:00:20.160
on theoretical biology um so in a
0:00:17.840,0:00:23.760
typical project they may use manxima
0:00:20.160,0:00:24.800
and julia their work for biology physics
0:00:23.760,0:00:27.199
and computer science
0:00:24.800,0:00:29.439
computer science and the latest work in
0:00:27.199,0:00:30.800
progress is on branching processes for
0:00:29.439,0:00:34.719
cancer modeling so
0:00:30.800,0:00:37.360
how can um emacs possibly help
0:00:34.719,0:00:38.399
let's let's have a look uh moving code
0:00:37.360,0:00:39.680
and data between these different
0:00:38.399,0:00:42.000
programs by hand is
0:00:39.680,0:00:43.200
annoying on separate workflows for
0:00:42.000,0:00:44.399
writing up notes and preparing
0:00:43.200,0:00:46.000
publications
0:00:44.399,0:00:48.640
is perhaps even more annoying all of
0:00:46.000,0:00:52.000
it's time consuming and error-prone
0:00:48.640,0:00:53.760
um so what about maybe using jupiter
0:00:52.000,0:00:55.199
uh we found something called script of
0:00:53.760,0:00:58.399
scripts and it solves some of those
0:00:55.199,0:01:01.120
problems because you can use
0:00:58.399,0:01:02.640
maximum and julia together but we were
0:01:01.120,0:01:05.199
quite happy to explore emacs based
0:01:02.640,0:01:05.840
solutions being emax enthusiasts and we
0:01:05.199,0:01:07.760
even
0:01:05.840,0:01:09.200
got cameron to be enthusiastic about
0:01:07.760,0:01:11.600
doing emacs so that
0:01:09.200,0:01:12.400
went nice so just here's a little
0:01:11.600,0:01:15.360
feature grid
0:01:12.400,0:01:16.400
of emacs org versus just sort of your
0:01:15.360,0:01:19.040
generic
0:01:16.400,0:01:20.960
um tools that are in a different more
0:01:19.040,0:01:21.520
general ecosystem so as you can see it's
0:01:20.960,0:01:23.360
quite
0:01:21.520,0:01:25.520
feature complete you've got your maximo
0:01:23.360,0:01:27.119
mode julia mode you can use both of them
0:01:25.520,0:01:28.720
inside of org mode
0:01:27.119,0:01:30.240
you can present things with word tree
0:01:28.720,0:01:33.280
slide you can set up a
0:01:30.240,0:01:33.680
wiki inside of orgrome this is one i
0:01:33.280,0:01:35.759
found
0:01:33.680,0:01:37.759
rather recently you can even use
0:01:35.759,0:01:38.799
compatibly with orgrome something called
0:01:37.759,0:01:41.520
log seek
0:01:38.799,0:01:42.159
which is in the browser um so that's
0:01:41.520,0:01:44.320
nice
0:01:42.159,0:01:45.840
um you can do real-time collaborative
0:01:44.320,0:01:47.840
editing um
0:01:45.840,0:01:49.280
either in a kind of pairing style or in
0:01:47.840,0:01:51.520
a more etherpad style
0:01:49.280,0:01:54.159
obviously you can manage your references
0:01:51.520,0:01:55.759
you can typeset whatever you want um you
0:01:54.159,0:01:57.439
can publish work in progress on a blog
0:01:55.759,0:01:58.159
and the fern is another one of these
0:01:57.439,0:02:00.560
external
0:01:58.159,0:02:01.360
org mode tools it's not actually any mac
0:02:00.560,0:02:03.680
but works with
0:02:01.360,0:02:04.399
org mode stuff and you know so we're
0:02:03.680,0:02:06.640
good to go
0:02:04.399,0:02:08.000
uh with all of that so what does that
0:02:06.640,0:02:09.679
look like well here's
0:02:08.000,0:02:11.599
a little example from before they were
0:02:09.679,0:02:12.800
doing um
0:02:11.599,0:02:14.560
before we started really thinking
0:02:12.800,0:02:15.599
seriously about this stuff so this is
0:02:14.560,0:02:17.440
just maxima
0:02:15.599,0:02:19.280
well maximo doesn't have a long running
0:02:17.440,0:02:20.160
process by default if you've ever used
0:02:19.280,0:02:22.480
python
0:02:20.160,0:02:23.920
uh you have something called sessions uh
0:02:22.480,0:02:24.959
they don't have that for maxim at least
0:02:23.920,0:02:27.599
not by default
0:02:24.959,0:02:29.360
um so how what was the workaround
0:02:27.599,0:02:32.480
there's this thing called uh
0:02:29.360,0:02:34.879
solve for you here and um
0:02:32.480,0:02:36.319
that shows up down below again in these
0:02:34.879,0:02:37.360
angle brackets which you've seen maybe
0:02:36.319,0:02:40.480
in someone else's talk
0:02:37.360,0:02:41.920
uh which means go to the previous uh
0:02:40.480,0:02:43.920
thing that was named soul for you
0:02:41.920,0:02:45.280
and do that all over again so they do
0:02:43.920,0:02:47.440
that over again
0:02:45.280,0:02:49.519
and here's the little maxima code for
0:02:47.440,0:02:50.560
defining you saw so you've now defined
0:02:49.519,0:02:51.920
you saw
0:02:50.560,0:02:54.000
and then you can use it in the next
0:02:51.920,0:02:56.560
expression you get out a nice juicy
0:02:54.000,0:02:57.200
zero at the end but it's a little bit a
0:02:56.560,0:02:58.959
little bit
0:02:57.200,0:03:00.640
like cartridge in a pear tree to have to
0:02:58.959,0:03:02.560
redefine everything every time
0:03:00.640,0:03:04.159
so this is clearly at the level of work
0:03:02.560,0:03:06.879
around um maybe just
0:03:04.159,0:03:07.920
one more time going looking through that
0:03:06.879,0:03:11.599
um
0:03:07.920,0:03:13.760
that stuff um
0:03:11.599,0:03:15.280
sorry so uh looking through that stuff
0:03:13.760,0:03:17.200
this is we're going to need something
0:03:15.280,0:03:19.599
like that probably for stitching
0:03:17.200,0:03:20.319
maxima and julie julia together so it's
0:03:19.599,0:03:21.680
good to
0:03:20.319,0:03:23.920
look a little bit about how that might
0:03:21.680,0:03:25.680
work so first of all you can cache
0:03:23.920,0:03:26.480
results so if you wanted to save the
0:03:25.680,0:03:28.480
date
0:03:26.480,0:03:29.920
out of block one at a certain time and
0:03:28.480,0:03:32.239
then use
0:03:29.920,0:03:33.280
use it again later so at the time when i
0:03:32.239,0:03:34.640
ran this code
0:03:33.280,0:03:36.720
you can see i've got two slightly
0:03:34.640,0:03:38.560
different time stamps down below one's
0:03:36.720,0:03:40.319
the cached result and the other was the
0:03:38.560,0:03:42.640
result of reevaluating
0:03:40.319,0:03:43.760
the block so you can move things around
0:03:42.640,0:03:46.000
um and that's
0:03:43.760,0:03:47.280
that's going to be useful but you know
0:03:46.000,0:03:48.080
that's not really the main problem the
0:03:47.280,0:03:50.799
main problem is
0:03:48.080,0:03:51.440
making maxima long running so kind of
0:03:50.799,0:03:53.920
the
0:03:51.440,0:03:56.400
core of this talk is a new observant
0:03:53.920,0:03:59.280
facility which is a general purpose
0:03:56.400,0:04:00.560
way to do that kind of thing which
0:03:59.280,0:04:04.239
involves a very simple
0:04:00.560,0:04:05.360
change to obcor uh so we'll give a quick
0:04:04.239,0:04:08.480
overview of that and show
0:04:05.360,0:04:11.760
an example um so here's the example
0:04:08.480,0:04:13.040
um a very simple sort of silly example
0:04:11.760,0:04:14.640
uh what does it mean to have a long
0:04:13.040,0:04:15.920
running process here i've set this
0:04:14.640,0:04:18.560
display2d
0:04:15.920,0:04:19.440
to be false um which just means that
0:04:18.560,0:04:22.320
things are going to come
0:04:19.440,0:04:23.040
come across in 1d and then i ask it to
0:04:22.320,0:04:25.199
expand
0:04:23.040,0:04:27.280
uh something and i get latex by default
0:04:25.199,0:04:28.639
so so that's what it means is i've sent
0:04:27.280,0:04:30.240
something in and it's going to come
0:04:28.639,0:04:32.080
across in one view which is great
0:04:30.240,0:04:34.560
um maybe you'll also notice that there's
0:04:32.080,0:04:36.720
no semicolon if you're a maxima fan
0:04:34.560,0:04:38.400
um and things are coming across as tech
0:04:36.720,0:04:40.320
so those were some little bonus features
0:04:38.400,0:04:41.040
and i'll show you how that works later
0:04:40.320,0:04:45.440
so
0:04:41.040,0:04:48.880
um the change to obcor is as follows uh
0:04:45.440,0:04:51.520
um we uh actually this should say uh
0:04:48.880,0:04:52.479
instead of stream here it should say um
0:04:51.520,0:04:54.800
servant
0:04:52.479,0:04:56.160
sorry uh we tried an experimental
0:04:54.800,0:04:58.160
version what's called stream so now it's
0:04:56.160,0:05:01.520
called servant but all it does is it
0:04:58.160,0:05:02.639
overrides uh or babel execute laying for
0:05:01.520,0:05:05.919
arbitrary laying
0:05:02.639,0:05:07.840
if you have um a servant
0:05:05.919,0:05:09.759
in your in your params so that's the
0:05:07.840,0:05:11.919
change that hasn't been
0:05:09.759,0:05:13.759
pushed out or sent as a patch to anybody
0:05:11.919,0:05:16.960
but it's a pretty minor change
0:05:13.759,0:05:19.080
um here's an overview without the code
0:05:16.960,0:05:20.720
of just a high level overview of
0:05:19.080,0:05:22.160
observant.el so
0:05:20.720,0:05:24.479
it stores information about these
0:05:22.160,0:05:26.080
processes in a hash table
0:05:24.479,0:05:27.600
it can do pre-processing and
0:05:26.080,0:05:29.759
post-processing
0:05:27.600,0:05:30.720
um it does all these things it stores
0:05:29.759,0:05:32.479
the output
0:05:30.720,0:05:34.080
i mentioned here that in principle we
0:05:32.479,0:05:35.280
could store lots of output and have a
0:05:34.080,0:05:36.560
kind of browsable
0:05:35.280,0:05:38.880
history although we don't do that
0:05:36.560,0:05:40.639
presently um but that's what observant
0:05:38.880,0:05:41.440
does is it does what you might expect
0:05:40.639,0:05:45.440
and here's the
0:05:41.440,0:05:48.160
here's the maxima kind of um
0:05:45.440,0:05:49.600
on-ramp uh to get maxima brought in so
0:05:48.160,0:05:51.360
you have to obviously have a maximum
0:05:49.600,0:05:54.960
process you can call
0:05:51.360,0:05:57.840
um put hash uh this is the preprocessing
0:05:54.960,0:05:58.960
thing i mentioned adding in some tech um
0:05:57.840,0:06:01.520
and adding in
0:05:58.960,0:06:03.120
uh or deleting rather a substring and
0:06:01.520,0:06:03.759
here here is why you delete the
0:06:03.120,0:06:06.960
substring
0:06:03.759,0:06:08.240
is because um maxima thinks it's a good
0:06:06.960,0:06:10.080
idea to tell you false
0:06:08.240,0:06:11.759
once you once you run check on things
0:06:10.080,0:06:13.680
you've got to delete that back out to
0:06:11.759,0:06:16.960
get something kind of coherent out of it
0:06:13.680,0:06:19.280
but so this is how to set up maximal um
0:06:16.960,0:06:20.000
that's enough really of the demo is not
0:06:19.280,0:06:21.919
really a demos for
0:06:20.000,0:06:23.600
show and tell but uh this is an
0:06:21.919,0:06:25.440
experience report i wanted to talk about
0:06:23.600,0:06:28.080
the experience of doing this
0:06:25.440,0:06:30.160
so some some negatives like we tried to
0:06:28.080,0:06:33.199
get emacs jupiter working
0:06:30.160,0:06:36.000
prior to prior to working on the um
0:06:33.199,0:06:37.919
observant and we couldn't get it doing
0:06:36.000,0:06:40.160
everything we wanted despite a bit of
0:06:37.919,0:06:41.840
heavy lifting and debugging and stuff so
0:06:40.160,0:06:42.880
that's not not finished that was a bit
0:06:41.840,0:06:45.360
difficult
0:06:42.880,0:06:47.759
um on the other hand working on
0:06:45.360,0:06:48.479
observing was fun and pretty lightweight
0:06:47.759,0:06:50.400
and easy
0:06:48.479,0:06:52.400
um we got some experience co-editing
0:06:50.400,0:06:54.479
things with these real-time tools
0:06:52.400,0:06:55.919
obviously the stack is somewhat work in
0:06:54.479,0:06:58.000
progress um
0:06:55.919,0:07:00.800
so i just wanted to give a shout out to
0:06:58.000,0:07:03.919
crdt which was really fun
0:07:00.800,0:07:07.039
and champion was making bug fiction
0:07:03.919,0:07:08.960
bug fixes for that as we go similarly
0:07:07.039,0:07:10.160
for fern and log seek the maintainers
0:07:08.960,0:07:12.960
were really responsive
0:07:10.160,0:07:14.560
um and so that was nice we did try to
0:07:12.960,0:07:15.840
get emacs running in the browser
0:07:14.560,0:07:16.880
thinking it would be really nice for
0:07:15.840,0:07:19.520
people who
0:07:16.880,0:07:21.120
um didn't want to install it to get a
0:07:19.520,0:07:23.120
chance to just try it
0:07:21.120,0:07:24.479
uh but actually browsers capture things
0:07:23.120,0:07:27.120
like uh control n
0:07:24.479,0:07:28.479
and so that's that was a bit annoying
0:07:27.120,0:07:29.759
but we did get lots of great feedback
0:07:28.479,0:07:31.599
and interaction with people including
0:07:29.759,0:07:33.759
around this conference so thank you
0:07:31.599,0:07:35.599
to those who we've had discussions with
0:07:33.759,0:07:37.680
um
0:07:35.599,0:07:38.639
so future work okay so maybe you
0:07:37.680,0:07:41.039
remember i didn't
0:07:38.639,0:07:42.400
give a talk a few years back on arcana
0:07:41.039,0:07:44.479
so what what
0:07:42.400,0:07:45.919
might this have to do with org mode and
0:07:44.479,0:07:47.039
that's always the question one asks
0:07:45.919,0:07:48.720
about arcata
0:07:47.039,0:07:50.319
well so arcana one of the things it does
0:07:48.720,0:07:51.680
is transclusions and so that could be
0:07:50.319,0:07:54.000
actually very helpful
0:07:51.680,0:07:55.520
in connection with this combined notes
0:07:54.000,0:07:56.479
and write-up workflow so you might have
0:07:55.520,0:07:58.400
a
0:07:56.479,0:08:00.800
forward mode uh and some of these
0:07:58.400,0:08:03.199
results we got back as raw results
0:08:00.800,0:08:03.919
could go right into your write up in a
0:08:03.199,0:08:07.520
kind of
0:08:03.919,0:08:09.039
convenient way at a level above um
0:08:07.520,0:08:11.280
transparently a level above the notebook
0:08:09.039,0:08:13.440
so you'd have the notebook alongside the
0:08:11.280,0:08:15.599
write-up in that case
0:08:13.440,0:08:16.800
which is a sort of variation on the
0:08:15.599,0:08:19.120
literate programming
0:08:16.800,0:08:20.000
workflow this is speculative who knows
0:08:19.120,0:08:22.080
the other thought is
0:08:20.000,0:08:23.520
it just relates to the idea of network
0:08:22.080,0:08:24.800
programming so we can imagine these
0:08:23.520,0:08:26.639
networks of
0:08:24.800,0:08:28.800
computational nodes maybe sitting inside
0:08:26.639,0:08:31.199
of orgrome calling each other
0:08:28.800,0:08:32.560
um and you would want to maintain some
0:08:31.199,0:08:34.959
kind of model of that
0:08:32.560,0:08:36.640
process and a sort of general question
0:08:34.959,0:08:38.320
is how do we have a remote control for
0:08:36.640,0:08:40.240
long-running processes you could do that
0:08:38.320,0:08:42.479
in lisp or closure but
0:08:40.240,0:08:44.080
maybe we could have something a little
0:08:42.479,0:08:45.839
bit like that here
0:08:44.080,0:08:48.080
um so conclusions what have we actually
0:08:45.839,0:08:50.080
addressed well uh we addressed uh
0:08:48.080,0:08:51.600
accessing any long-running process with
0:08:50.080,0:08:52.959
a simple or mode interface
0:08:51.600,0:08:54.880
obviously we're not the only people to
0:08:52.959,0:08:56.880
think about notebooks but we think that
0:08:54.880,0:08:58.880
emacs has some advantages
0:08:56.880,0:09:01.120
related to reproducible research and
0:08:58.880,0:09:01.680
interdisciplinary collaboration so let's
0:09:01.120,0:09:03.120
just
0:09:01.680,0:09:04.880
say that we think something is
0:09:03.120,0:09:07.200
reproducible if it's actually teachable
0:09:04.880,0:09:09.279
to someone new and they can do it
0:09:07.200,0:09:11.680
org mode seems very useful for that many
0:09:09.279,0:09:14.399
of the other talks have touched on this
0:09:11.680,0:09:16.000
interdisciplinary collaboration is great
0:09:14.399,0:09:17.839
this was an interdisciplinary
0:09:16.000,0:09:19.680
collaboration on some level but
0:09:17.839,0:09:21.600
what about you know future work for
0:09:19.680,0:09:22.320
bringing in scenario planners simulation
0:09:21.600,0:09:24.480
scientists
0:09:22.320,0:09:25.519
and local farmers and building something
0:09:24.480,0:09:27.200
that they can all
0:09:25.519,0:09:29.200
use that's more than the sum of the
0:09:27.200,0:09:30.720
parts so a little
0:09:29.200,0:09:32.160
future work for everybody else here so
0:09:30.720,0:09:33.120
we think science should be widely
0:09:32.160,0:09:34.560
teachable
0:09:33.120,0:09:36.720
shareable semi-automated
0:09:34.560,0:09:40.399
transdisciplinary and real time
0:09:36.720,0:09:43.279
like emacs conf so you can get in touch
0:09:40.399,0:09:44.560
uh via these methods and the code which
0:09:43.279,0:09:45.200
is very much early stage work in
0:09:44.560,0:09:46.640
progress
0:09:45.200,0:09:48.800
as this was meant to be an experience
0:09:46.640,0:09:49.920
report not a it's all done here it is
0:09:48.800,0:09:51.680
polished report
0:09:49.920,0:09:53.360
um it's also online if you'd like to
0:09:51.680,0:09:54.560
have a look so that's the end of the
0:09:53.360,0:09:55.279
talk and i don't know if there's time
0:09:54.560,0:09:57.440
for questions
0:09:55.279,0:10:00.240
or not but um i'm at your disposal now
0:09:57.440,0:10:04.079
thank you you are now unmuted
0:10:00.240,0:10:07.120
um many thanks for the tough job
0:10:04.079,0:10:08.880
uh let's see we have about i think four
0:10:07.120,0:10:10.880
minutes four questions
0:10:08.880,0:10:12.320
um and we have a couple of questions on
0:10:10.880,0:10:14.240
the pad would you like to read them
0:10:12.320,0:10:16.560
yourself or should i read them to you
0:10:14.240,0:10:18.079
uh just for sake of easy management why
0:10:16.560,0:10:21.279
don't you read them out if that's okay
0:10:18.079,0:10:22.240
yeah sure uh so they ask have you looked
0:10:21.279,0:10:25.440
into trying
0:10:22.240,0:10:33.760
sage math i've long uh long wanted
0:10:25.440,0:10:36.959
nothing org files
0:10:33.760,0:10:39.279
right and i and i wrote the answer that
0:10:36.959,0:10:42.079
we should be possible because one can
0:10:39.279,0:10:42.079
call it from a command
0:10:44.839,0:10:48.640
okay
0:10:46.399,0:10:50.079
and i see there's another sagemath
0:10:48.640,0:10:52.100
question that you seem to have answered
0:10:50.079,0:10:54.880
as well so i guess i won't repeat that
0:10:52.100,0:10:56.959
[Music]
0:10:54.880,0:11:00.640
there's let's not forget about embedded
0:10:56.959,0:11:03.680
calc in emacs
0:11:00.640,0:11:05.040
so so the first demos actually were with
0:11:03.680,0:11:06.640
calc so that's that's
0:11:05.040,0:11:07.839
useful uh although i think it was a
0:11:06.640,0:11:08.240
different kind of a different command
0:11:07.839,0:11:11.839
line
0:11:08.240,0:11:13.839
well that was the next calc yeah
0:11:11.839,0:11:15.680
so sure there is there is calc so that
0:11:13.839,0:11:19.120
would be calculus and
0:11:15.680,0:11:21.839
calc is already in org mode yeah
0:11:19.120,0:11:21.839
yeah yeah
0:11:25.680,0:11:29.839
um still looking for questions
0:11:39.760,0:11:44.320
okay i think that's about it i don't see
0:11:41.760,0:11:47.440
any questions on the ether pad
0:11:44.320,0:11:50.480
and let's see
0:11:47.440,0:11:50.480
anything on irc
0:11:53.040,0:11:56.720
um nothing but phrases and everyone
0:11:55.760,0:11:59.120
thanking you
0:11:56.720,0:12:00.240
thank you all right you're welcome
0:11:59.120,0:12:02.880
thanks a lot uh
0:12:00.240,0:12:06.800
we'll see you guys around then cheers
0:12:02.880,0:12:06.800
and see you around
