path: root/2022/captions/emacsconf-2022-jupyter--edit-live-jupyter-notebook-cells-with-emacs--blaine-mooers--main.vtt

WEBVTT captioned by sachac

NOTE Introduction

00:00:00.000 --> 00:00:03.839
Hi, my name is Blaine Mooers.

00:00:03.840 --> 00:00:05.399
I'm an associate professor of biochemistry

00:00:05.400 --> 00:00:08.199
at the University of Oklahoma Health Sciences Center

00:00:08.200 --> 00:00:09.759
in Oklahoma City.

00:00:09.760 --> 00:00:11.999
I'm going to talk about the use of Emacs

00:00:12.000 --> 00:00:16.239
to edit live Jupyter notebook cells

00:00:16.240 --> 00:00:20.519
as well as text areas on web pages.

00:00:20.520 --> 00:00:22.519
So like a lot of technical workers,

00:00:22.520 --> 00:00:24.879
I find myself having to write prose

00:00:24.880 --> 00:00:28.679
in text areas on web pages,

00:00:28.680 --> 00:00:31.719
as well as working with code

00:00:31.720 --> 00:00:36.999
in Jupyter notebooks and Colab notebooks,

00:00:37.000 --> 00:00:39.319
and often I have wished for

00:00:39.320 --> 00:00:43.479
the full power of Emacs while doing so.

NOTE GhostText and Atomic Chrome

00:00:43.480 --> 00:00:45.319
Well, now that is possible.

00:00:45.320 --> 00:00:46.799
Actually, there are several solutions

00:00:46.800 --> 00:00:49.039
that have been available for some time.

00:00:49.040 --> 00:00:50.399
I'm going to talk about one solution

00:00:50.400 --> 00:00:55.319
that I'm familiar with and has worked out for me.

00:00:55.320 --> 00:00:58.119
So this requires the use of two software packages,

00:00:58.120 --> 00:01:01.599
GhostText and Atomic Chrome.

00:01:01.600 --> 00:01:04.239
GhostText is an extension for the web browser,

00:01:04.240 --> 00:01:07.279
whereas Atomic Chrome is a package for Emacs.

00:01:07.280 --> 00:01:10.879
You have to have both of these.

00:01:10.880 --> 00:01:13.399
So Chrome is for the editor side

00:01:13.400 --> 00:01:18.919
and GhostText handles the browser side.

NOTE GhostText

00:01:18.920 --> 00:01:22.479
The GhostText extension is available

00:01:22.480 --> 00:01:26.799
in the Chrome web store.

00:01:26.800 --> 00:01:29.519
And GhostText is represented by this icon,

00:01:29.520 --> 00:01:31.519
which has a ghost in front of the capital letter T.

00:01:31.520 --> 00:01:36.399
It is being developed by Federico Brigante.

00:01:36.400 --> 00:01:41.239
He is a very prolific JavaScript developer.

00:01:41.240 --> 00:01:44.279
He has a web page committed to GhostText,

00:01:44.280 --> 00:01:47.199
as well as a GitHub site.

00:01:47.200 --> 00:01:50.159
So here's an example of GhostText.

00:01:50.160 --> 00:01:54.839
This is a snapshot from a session

00:01:54.840 --> 00:02:01.279
that I had while editing LaTeX on the Overleaf website.

00:02:01.280 --> 00:02:03.759
Overleaf is this web service

00:02:03.760 --> 00:02:09.719
that empowers the editing of LaTeX documents on the web.

00:02:09.720 --> 00:02:14.039
So I have clicked on this GhostText icon in the toolbar.

00:02:14.040 --> 00:02:17.039
I had already opened up Emacs,

00:02:17.040 --> 00:02:19.879
and I had the Atomic Chrome server running.

00:02:19.880 --> 00:02:21.799
So a connection was established,

00:02:21.800 --> 00:02:28.599
as indicated by this blue border around this text area.

00:02:28.600 --> 00:02:30.519
And as soon as that appeared,

00:02:30.520 --> 00:02:35.319
the text appeared in a buffer inside of Emacs.

00:02:35.320 --> 00:02:40.159
So I have overlaid the area where normally the compiled PDF

00:02:40.160 --> 00:02:43.919
would appear in an Overleaf session.

00:02:43.920 --> 00:02:47.679
So I'm using a configuration for LaTeX that I developed,

00:02:47.680 --> 00:02:52.159
which is available through the MooersLab GitHub site.

00:02:52.160 --> 00:02:55.919
I also gave a talk about how I use LaTeX in Emacs

00:02:55.920 --> 00:02:58.839
at the Berlin Emacs meetup in August.

00:02:58.840 --> 00:03:00.039
This talk was not recorded,

00:03:00.040 --> 00:03:05.119
but the slides are available on this website.

NOTE Live coding demo

00:03:05.120 --> 00:03:09.519
So I would like to now switch to a little live coding

00:03:09.520 --> 00:03:12.359
to make this a little more interesting.

00:03:12.360 --> 00:03:22.439
So I start my day at this other website called 750Words.

00:03:22.440 --> 00:03:25.279
This site just takes plain text,

00:03:25.280 --> 00:03:28.559
but I like to write in LaTeX.

00:03:28.560 --> 00:03:31.519
So GhostText came to my rescue

00:03:31.520 --> 00:03:34.479
when I started using this everyday last May.

00:03:34.480 --> 00:03:37.239
So I clicked on the GhostText icon.

00:03:37.240 --> 00:03:39.759
It highlighted that area in blue.

00:03:39.760 --> 00:03:41.839
There's some boilerplate

00:03:41.840 --> 00:03:45.919
that I like to start my day with.

00:03:45.920 --> 00:03:50.599
I like to get a list of my deadlines

00:03:50.600 --> 00:03:54.239
that are coming up, as shown here,

00:03:54.240 --> 00:03:55.839
for the next several months.

00:03:55.840 --> 00:03:59.399
And then I have landed at this tab stop.

00:03:59.400 --> 00:04:04.239
And so I had issued a tab trigger

00:04:04.240 --> 00:04:09.519
which inserted this almost 50 lines of text

00:04:09.520 --> 00:04:13.639
from a snippet through Yasnippet.

00:04:13.640 --> 00:04:17.479
And then I'll change this text to whatever.

00:04:17.480 --> 00:04:29.279
And then I can hit TAB to move to the next site.

00:04:29.280 --> 00:04:38.439
I was dead tired last night, so I fell asleep at my desk,

00:04:38.440 --> 00:04:42.439
and whatever. So I just keep on going

00:04:42.440 --> 00:04:47.319
and then hit TAB again and enter my "To Be Done" items.

00:04:47.320 --> 00:04:50.239
And then what I love about Emacs is that

00:04:50.240 --> 00:04:56.119
you can hit C-c C-j to insert a new item and so forth,

00:04:56.120 --> 00:05:03.239
so you can extend the list.

00:05:03.240 --> 00:05:05.079
Initially, I just have 10 items.

00:05:05.080 --> 00:05:08.839
I'm going to have more. And on I go,

00:05:08.840 --> 00:05:12.359
using the full power of LaTeX.

00:05:12.360 --> 00:05:16.119
So I have configured Atomic Chrome

00:05:16.120 --> 00:05:19.999
so it will recognize this website as a –

00:05:20.000 --> 00:05:21.399
it will open up this website –

00:05:21.400 --> 00:05:26.159
the connection to this website with this buffer

00:05:26.160 --> 00:05:36.559
in the LaTeX major mode. To turn this off,

00:05:36.560 --> 00:05:40.319
we can close – simply just close the buffer

00:05:40.320 --> 00:05:42.959
and that will shut things down.

00:05:42.960 --> 00:05:46.399
On the browser side, you can right-click on the icon

00:05:46.400 --> 00:05:49.039
and disconnect GhostText on this page.

NOTE Editing code cells

00:05:49.040 --> 00:05:53.639
Okay, let's go to a different situation.

00:05:53.640 --> 00:05:57.359
This is not a feature that's advertised by the developer,

00:05:57.360 --> 00:06:00.119
but I discovered that you can edit code cells

00:06:00.120 --> 00:06:02.799
(or any kind of cell for that matter)

00:06:02.800 --> 00:06:04.439
in a Jupyter Notebook.

00:06:04.440 --> 00:06:05.719
However, we have a challenge here.

00:06:05.720 --> 00:06:09.319
We have three text areas open – three code cells.

00:06:09.320 --> 00:06:14.759
So if we click on the GhostText icon,

00:06:14.760 --> 00:06:17.479
these three areas will show up in green

00:06:17.480 --> 00:06:19.119
and we'll be prompted to select the one

00:06:19.120 --> 00:06:20.279
that we want to activate.

00:06:20.280 --> 00:06:22.359
We want to activate the one with text.

00:06:22.360 --> 00:06:31.399
So then we can go in here and make edits, of course,

00:06:31.400 --> 00:06:33.359
and you can do this in Emacs

00:06:33.360 --> 00:06:35.639
or we can do it in the browser. It doesn't matter.

00:06:35.640 --> 00:06:38.359
You saw me editing in Emacs,

00:06:38.360 --> 00:06:40.359
but we can also make the edits

00:06:40.360 --> 00:06:44.879
in the text area of the browser

00:06:44.880 --> 00:06:47.519
and they will show up immediately in Emacs.

00:06:47.520 --> 00:06:54.439
So we could change the case of that M and that's going to –

00:06:54.440 --> 00:06:59.279
shows up over here. Okay, we can run this code.

00:06:59.280 --> 00:07:03.439
So this is R, one of the three major

00:07:03.440 --> 00:07:05.159
programming languages for data science.

00:07:05.160 --> 00:07:07.319
At least, Jupyter is supposed to be

00:07:07.320 --> 00:07:13.479
a combination of Julia, Python, and R.

00:07:13.480 --> 00:07:17.559
So we're running mcmc to get the posterior distribution

00:07:17.560 --> 00:07:21.119
and we're going to plot those out with this pyplots package,

00:07:21.120 --> 00:07:24.079
and we have these beautiful plots showing the median

00:07:24.080 --> 00:07:27.039
of the posterior distribution for four variables in –

00:07:27.040 --> 00:07:31.559
four parameters in the CARS data set,

00:07:31.560 --> 00:07:38.039
which is available – built into the R package.

00:07:38.040 --> 00:07:45.559
And then these shaded areas are the 80% interval.

00:07:45.560 --> 00:07:49.159
Okay. Oops.

NOTE Python

00:07:49.160 --> 00:08:07.639
So now for the Python side,

00:08:07.640 --> 00:08:13.359
here's an example in which I'm going to actually

00:08:13.360 --> 00:08:18.679
insert a snippet of that cell

00:08:18.680 --> 00:08:27.279
and then I'm going to enter nvlig for nglview ligand,

00:08:27.280 --> 00:08:43.319
and just hit enter. Oops. Hit TAB, excuse me,

00:08:43.320 --> 00:08:47.999
and we don't need this line of code, so delete that.

00:08:48.000 --> 00:08:52.199
Yep, we want to load up this pdb file

00:08:52.200 --> 00:08:54.279
that's in this subdirectory.

00:08:54.280 --> 00:08:56.919
So the pdb file is a plain text file

00:08:56.920 --> 00:08:58.839
that contains atomic coordinates

00:08:58.840 --> 00:09:00.159
of protein crystal structure.

00:09:00.160 --> 00:09:02.999
This protein happens to be important in cancer

00:09:03.000 --> 00:09:09.079
and we have – we screened by docking 55,000 compounds

00:09:09.080 --> 00:09:10.679
on a supercomputer

00:09:10.680 --> 00:09:12.743
and then we did MD [molecular dynamics] simulations

00:09:12.744 --> 00:09:14.159
of the top 10 [actually 20] leads.

00:09:14.160 --> 00:09:18.319
Twelve of them had the compound remain bound

00:09:18.320 --> 00:09:19.679
during the period of the simulation,

00:09:19.680 --> 00:09:22.559
so those have some potential for –

00:09:22.560 --> 00:09:25.159
and require experimental validation.

00:09:25.160 --> 00:09:28.639
So we'll run this chunk of code,

00:09:28.640 --> 00:09:30.839
and this will give a view of the molecule

00:09:30.840 --> 00:09:36.479
that we can interact with by using the mouse.

00:09:36.480 --> 00:09:39.879
But I want to share this with my colleague.

00:09:39.880 --> 00:09:42.919
My colleague is not set up to use Jupyter,

00:09:42.920 --> 00:09:46.959
but instead we can write this out to a HTML file,

00:09:46.960 --> 00:09:48.319
which I have loaded up already.

00:09:48.320 --> 00:09:55.319
And so we can actually – perhaps.

00:09:55.320 --> 00:10:01.679
We click on these two arrows pointing at each other,

00:10:01.680 --> 00:10:05.159
and we can get a full screen view of this molecule

00:10:05.160 --> 00:10:10.439
and he can identify each atom in this structure.

00:10:10.440 --> 00:10:12.719
Over a thousand atoms present.

00:10:12.720 --> 00:10:14.959
We're just hovering over a specific atom.

00:10:14.960 --> 00:10:27.759
So shown in gray is the ligand that is bound. Okay.

00:10:27.760 --> 00:10:32.079
So we still have this box selected

00:10:32.080 --> 00:10:36.559
and we still have these two different –

00:10:36.560 --> 00:10:40.519
so for each of the – our selected text areas

00:10:40.520 --> 00:11:03.839
we have a separate – we have a separate buffer open. Okay.

NOTE Julia

00:11:03.840 --> 00:11:06.839
To wrap things up here, here's an example of using –

00:11:06.840 --> 00:11:08.919
with evolving Julia code.

00:11:08.920 --> 00:11:12.919
And so this Julia code in this cell is in a Emacs buffer.

00:11:12.920 --> 00:11:19.279
So you've got an idea now, I think.

00:11:19.280 --> 00:11:23.759
So in terms of plain text areas like in Overleaf

00:11:23.760 --> 00:11:27.519
and then these cells in Jupyter Notebooks,

00:11:27.520 --> 00:11:30.959
these are other areas that can be edited

00:11:30.960 --> 00:11:37.919
like in the text areas within Outlook Webmail and Gmail.

00:11:37.920 --> 00:11:42.639
Instead of having to point with the mouse

00:11:42.640 --> 00:11:43.519
or click with the mouse,

00:11:43.520 --> 00:11:48.079
one can also use keybindings or keyboard shortcuts.

00:11:48.080 --> 00:11:51.599
So here are the ones for three major operating systems.

NOTE How does GhostText work?

00:11:51.600 --> 00:11:54.639
So how does GhostText work?

00:11:54.640 --> 00:11:57.199
Main thing is you have to open up Emacs

00:11:57.200 --> 00:11:59.679
and get this Atomic Chrome server running.

00:11:59.680 --> 00:12:03.119
And then with it up and going,

00:12:03.120 --> 00:12:06.679
GhostText will be able to – has to be activated

00:12:06.680 --> 00:12:09.639
and it will find the GhostText server

00:12:09.640 --> 00:12:15.079
through the localhost port 4001.

00:12:15.080 --> 00:12:19.039
Put that into the web browser.

00:12:19.040 --> 00:12:21.639
If you navigate to that port,

00:12:21.640 --> 00:12:23.719
you'll get output that looks like this

00:12:23.720 --> 00:12:25.639
if everything's working well.

00:12:25.640 --> 00:12:27.919
Otherwise, you'll get a error message

00:12:27.920 --> 00:12:30.519
and it should have a port socket –

00:12:30.520 --> 00:12:34.319
a web socket port number.

00:12:34.320 --> 00:12:36.319
It will not be the same every time.

NOTE Supported web browsers

00:12:36.320 --> 00:12:42.319
So these are the supported web browsers

00:12:42.320 --> 00:12:45.359
in addition to Chrome. These are supported

00:12:45.360 --> 00:12:47.279
and likewise anything in these –

00:12:47.280 --> 00:12:49.879
any browser related to these

00:12:49.880 --> 00:12:53.079
can probably use these extensions.

00:12:53.080 --> 00:12:56.359
For example, the Brave browser will use Chrome extension

00:12:56.360 --> 00:13:01.679
and the Firefox browser extension works with WaterFox.

00:13:01.680 --> 00:13:05.239
These are the supported editors.

00:13:05.240 --> 00:13:10.199
Each editor has its own extension

00:13:10.200 --> 00:13:15.719
and this GhostText was initially developed for SublimeText.

00:13:15.720 --> 00:13:17.919
So if you have SublimeText,

00:13:17.920 --> 00:13:21.959
then you can use its smooth operation as positive control

00:13:21.960 --> 00:13:24.999
when things go wrong with Emacs.

NOTE Atomic Chrome configuration

00:13:25.000 --> 00:13:29.479
This is Atomic. This is a GitHub site for Atomic Chrome.

00:13:29.480 --> 00:13:33.159
Atomic Chrome is available for installation through Melpa.

00:13:33.160 --> 00:13:40.319
This is my configuration for Atomic Chrome.

00:13:40.320 --> 00:13:45.519
So I have this setup so the server starts whenever I log in,

00:13:45.520 --> 00:13:50.319
and I have it set up so that default major mode is Python

00:13:50.320 --> 00:13:55.559
to deal with the Jupyter notebooks and Colab notebooks.

00:13:55.560 --> 00:13:57.719
And then I have major modes

00:13:57.720 --> 00:13:59.839
for these other websites defined below.

00:13:59.840 --> 00:14:06.079
This is a testing site so the developer has made

00:14:06.080 --> 00:14:08.119
to help with troubleshooting.

00:14:08.120 --> 00:14:11.159
He also has a protocol on his website

00:14:11.160 --> 00:14:13.559
to follow during troubleshooting.

NOTE Precautions

00:14:13.560 --> 00:14:16.639
So here are some precautions.

00:14:16.640 --> 00:14:19.079
You'll find that GhostText doesn't work with Pluto.

00:14:19.080 --> 00:14:21.239
Pluto is a new computational notebook

00:14:21.240 --> 00:14:23.479
for working with Julia.

00:14:23.480 --> 00:14:27.039
My suggestion would be just to run IJulia in Jupyter.

00:14:27.040 --> 00:14:31.999
It also doesn't work, of course, with RStudio.

00:14:32.000 --> 00:14:35.199
Even though RStudio sort of resembles

00:14:35.200 --> 00:14:38.999
a web page, web browser, it's not.

00:14:39.000 --> 00:14:41.359
Of course, you can always run R,

00:14:41.360 --> 00:14:48.239
as you've just seen, using the IPy kernel.

00:14:48.240 --> 00:14:52.239
I will also caution you that if you use the Emacs server,

00:14:52.240 --> 00:14:53.519
you may run into issues

00:14:53.520 --> 00:14:57.999
with the server competing with the port 4001.

00:14:58.000 --> 00:15:01.919
So instead, you should probably configure the Emacs server

00:15:01.920 --> 00:15:04.999
to use a specific port.

00:15:05.000 --> 00:15:07.639
So far – although I haven't done that myself –

00:15:07.640 --> 00:15:10.999
so far, I haven't found any conflicts

00:15:11.000 --> 00:15:13.479
with the Org Roam user interface.

NOTE Conclusions

00:15:13.480 --> 00:15:20.919
So my conclusions are: GhostText allows you to edit prose

00:15:20.920 --> 00:15:24.439
with your favorite major mode

00:15:24.440 --> 00:15:28.119
in the text areas of web pages

00:15:28.120 --> 00:15:31.079
and in the cells of Jupyter notebooks.

00:15:31.080 --> 00:15:34.359
This allows you to tap into snippets

00:15:34.360 --> 00:15:37.519
and thereby save time as you have – probably have –

00:15:37.520 --> 00:15:39.719
hopefully got an idea of.

NOTE Thanks

00:15:39.720 --> 00:15:44.039
I'd like to thank my friends and mentors

00:15:44.040 --> 00:15:46.599
who've helped me out during my second year

00:15:46.600 --> 00:15:49.359
in my Emacs learning spiral.

00:15:49.360 --> 00:15:55.839
These include my local colleagues.

00:15:55.840 --> 00:15:58.679
We meet once a month in the Oklahoma Data Science Workshop.

00:15:58.680 --> 00:16:01.719
Last July, I gave a presentation about GhostText.

00:16:01.720 --> 00:16:08.359
And then also my friends at Berlin and Austin Emacs meetups

00:16:08.360 --> 00:16:12.959
and in the UK research software engineer

00:16:12.960 --> 00:16:17.599
Emacs research Slack channel.

00:16:17.600 --> 00:16:18.919
So I don't attend these every month,

00:16:18.920 --> 00:16:21.439
but I try to make the meetings as often as I can.

00:16:21.440 --> 00:16:25.399
Then I'm supported by the following grants,

00:16:25.400 --> 00:16:28.359
which allow me to spend

00:16:28.360 --> 00:16:30.679
at least some time each day in Emacs.

00:16:30.680 --> 00:16:34.960
I'll be happy to take any questions.