summaryrefslogblamecommitdiffstats
path: root/2022/captions/emacsconf-2022-jupyter--edit-live-jupyter-notebook-cells-with-emacs--blaine-mooers--main.vtt
blob: e51ef23bc78108722dfc46979827b6f6fe8b9ef7 (plain) (tree)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792






















































































































































































































































































































































































































































































































































































































































































































































































































                                                             
WEBVTT captioned by sachac

NOTE Introduction

00:00:00.000 --> 00:00:03.839
Hi, my name is Blaine Mooers.

00:00:03.840 --> 00:00:05.399
I'm an associate professor of biochemistry

00:00:05.400 --> 00:00:08.199
at the University of Oklahoma Health Sciences Center

00:00:08.200 --> 00:00:09.759
in Oklahoma City.

00:00:09.760 --> 00:00:11.999
I'm going to talk about the use of Emacs

00:00:12.000 --> 00:00:16.239
to edit live Jupyter notebook cells

00:00:16.240 --> 00:00:20.519
as well as text areas on web pages.

00:00:20.520 --> 00:00:22.519
So like a lot of technical workers,

00:00:22.520 --> 00:00:24.879
I find myself having to write prose

00:00:24.880 --> 00:00:28.679
in text areas on web pages,

00:00:28.680 --> 00:00:31.719
as well as working with code

00:00:31.720 --> 00:00:36.999
in Jupyter notebooks and Colab notebooks,

00:00:37.000 --> 00:00:39.319
and often I have wished for

00:00:39.320 --> 00:00:43.479
the full power of Emacs while doing so.

NOTE GhostText and Atomic Chrome

00:00:43.480 --> 00:00:45.319
Well, now that is possible.

00:00:45.320 --> 00:00:46.799
Actually, there are several solutions

00:00:46.800 --> 00:00:49.039
that have been available for some time.

00:00:49.040 --> 00:00:50.399
I'm going to talk about one solution

00:00:50.400 --> 00:00:55.319
that I'm familiar with and has worked out for me.

00:00:55.320 --> 00:00:58.119
So this requires the use of two software packages,

00:00:58.120 --> 00:01:01.599
GhostText and Atomic Chrome.

00:01:01.600 --> 00:01:04.239
GhostText is an extension for the web browser,

00:01:04.240 --> 00:01:07.279
whereas Atomic Chrome is a package for Emacs.

00:01:07.280 --> 00:01:10.879
You have to have both of these.

00:01:10.880 --> 00:01:13.399
So Chrome is for the editor side

00:01:13.400 --> 00:01:18.919
and GhostText handles the browser side.

NOTE GhostText

00:01:18.920 --> 00:01:22.479
The GhostText extension is available

00:01:22.480 --> 00:01:26.799
in the Chrome web store.

00:01:26.800 --> 00:01:29.519
And GhostText is represented by this icon,

00:01:29.520 --> 00:01:31.519
which has a ghost in front of the capital letter T.

00:01:31.520 --> 00:01:36.399
It is being developed by Federico Brigante.

00:01:36.400 --> 00:01:41.239
He is a very prolific JavaScript developer.

00:01:41.240 --> 00:01:44.279
He has a web page committed to GhostText,

00:01:44.280 --> 00:01:47.199
as well as a GitHub site.

00:01:47.200 --> 00:01:50.159
So here's an example of GhostText.

00:01:50.160 --> 00:01:54.839
This is a snapshot from a session

00:01:54.840 --> 00:02:01.279
that I had while editing LaTeX on the Overleaf website.

00:02:01.280 --> 00:02:03.759
Overleaf is this web service

00:02:03.760 --> 00:02:09.719
that empowers the editing of LaTeX documents on the web.

00:02:09.720 --> 00:02:14.039
So I have clicked on this GhostText icon in the toolbar.

00:02:14.040 --> 00:02:17.039
I had already opened up Emacs,

00:02:17.040 --> 00:02:19.879
and I had the Atomic Chrome server running.

00:02:19.880 --> 00:02:21.799
So a connection was established,

00:02:21.800 --> 00:02:28.599
as indicated by this blue border around this text area.

00:02:28.600 --> 00:02:30.519
And as soon as that appeared,

00:02:30.520 --> 00:02:35.319
the text appeared in a buffer inside of Emacs.

00:02:35.320 --> 00:02:40.159
So I have overlaid the area where normally the compiled PDF

00:02:40.160 --> 00:02:43.919
would appear in an Overleaf session.

00:02:43.920 --> 00:02:47.679
So I'm using a configuration for LaTeX that I developed,

00:02:47.680 --> 00:02:52.159
which is available through the MooersLab GitHub site.

00:02:52.160 --> 00:02:55.919
I also gave a talk about how I use LaTeX in Emacs

00:02:55.920 --> 00:02:58.839
at the Berlin Emacs meetup in August.

00:02:58.840 --> 00:03:00.039
This talk was not recorded,

00:03:00.040 --> 00:03:05.119
but the slides are available on this website.

NOTE Live coding demo

00:03:05.120 --> 00:03:09.519
So I would like to now switch to a little live coding

00:03:09.520 --> 00:03:12.359
to make this a little more interesting.

00:03:12.360 --> 00:03:22.439
So I start my day at this other website called 750Words.

00:03:22.440 --> 00:03:25.279
This site just takes plain text,

00:03:25.280 --> 00:03:28.559
but I like to write in LaTeX.

00:03:28.560 --> 00:03:31.519
So GhostText came to my rescue

00:03:31.520 --> 00:03:34.479
when I started using this everyday last May.

00:03:34.480 --> 00:03:37.239
So I clicked on the GhostText icon.

00:03:37.240 --> 00:03:39.759
It highlighted that area in blue.

00:03:39.760 --> 00:03:41.839
There's some boilerplate

00:03:41.840 --> 00:03:45.919
that I like to start my day with.

00:03:45.920 --> 00:03:50.599
I like to get a list of my deadlines

00:03:50.600 --> 00:03:54.239
that are coming up, as shown here,

00:03:54.240 --> 00:03:55.839
for the next several months.

00:03:55.840 --> 00:03:59.399
And then I have landed at this tab stop.

00:03:59.400 --> 00:04:04.239
And so I had issued a tab trigger

00:04:04.240 --> 00:04:09.519
which inserted this almost 50 lines of text

00:04:09.520 --> 00:04:13.639
from a snippet through Yasnippet.

00:04:13.640 --> 00:04:17.479
And then I'll change this text to whatever.

00:04:17.480 --> 00:04:29.279
And then I can hit TAB to move to the next site.

00:04:29.280 --> 00:04:38.439
I was dead tired last night, so I fell asleep at my desk,

00:04:38.440 --> 00:04:42.439
and whatever. So I just keep on going

00:04:42.440 --> 00:04:47.319
and then hit TAB again and enter my "To Be Done" items.

00:04:47.320 --> 00:04:50.239
And then what I love about Emacs is that

00:04:50.240 --> 00:04:56.119
you can hit C-c C-j to insert a new item and so forth,

00:04:56.120 --> 00:05:03.239
so you can extend the list.

00:05:03.240 --> 00:05:05.079
Initially, I just have 10 items.

00:05:05.080 --> 00:05:08.839
I'm going to have more. And on I go,

00:05:08.840 --> 00:05:12.359
using the full power of LaTeX.

00:05:12.360 --> 00:05:16.119
So I have configured Atomic Chrome

00:05:16.120 --> 00:05:19.999
so it will recognize this website as a –

00:05:20.000 --> 00:05:21.399
it will open up this website –

00:05:21.400 --> 00:05:26.159
the connection to this website with this buffer

00:05:26.160 --> 00:05:36.559
in the LaTeX major mode. To turn this off,

00:05:36.560 --> 00:05:40.319
we can close – simply just close the buffer

00:05:40.320 --> 00:05:42.959
and that will shut things down.

00:05:42.960 --> 00:05:46.399
On the browser side, you can right-click on the icon

00:05:46.400 --> 00:05:49.039
and disconnect GhostText on this page.

NOTE Editing code cells

00:05:49.040 --> 00:05:53.639
Okay, let's go to a different situation.

00:05:53.640 --> 00:05:57.359
This is not a feature that's advertised by the developer,

00:05:57.360 --> 00:06:00.119
but I discovered that you can edit code cells

00:06:00.120 --> 00:06:02.799
(or any kind of cell for that matter)

00:06:02.800 --> 00:06:04.439
in a Jupyter Notebook.

00:06:04.440 --> 00:06:05.719
However, we have a challenge here.

00:06:05.720 --> 00:06:09.319
We have three text areas open – three code cells.

00:06:09.320 --> 00:06:14.759
So if we click on the GhostText icon,

00:06:14.760 --> 00:06:17.479
these three areas will show up in green

00:06:17.480 --> 00:06:19.119
and we'll be prompted to select the one

00:06:19.120 --> 00:06:20.279
that we want to activate.

00:06:20.280 --> 00:06:22.359
We want to activate the one with text.

00:06:22.360 --> 00:06:31.399
So then we can go in here and make edits, of course,

00:06:31.400 --> 00:06:33.359
and you can do this in Emacs

00:06:33.360 --> 00:06:35.639
or we can do it in the browser. It doesn't matter.

00:06:35.640 --> 00:06:38.359
You saw me editing in Emacs,

00:06:38.360 --> 00:06:40.359
but we can also make the edits

00:06:40.360 --> 00:06:44.879
in the text area of the browser

00:06:44.880 --> 00:06:47.519
and they will show up immediately in Emacs.

00:06:47.520 --> 00:06:54.439
So we could change the case of that M and that's going to –

00:06:54.440 --> 00:06:59.279
shows up over here. Okay, we can run this code.

00:06:59.280 --> 00:07:03.439
So this is R, one of the three major

00:07:03.440 --> 00:07:05.159
programming languages for data science.

00:07:05.160 --> 00:07:07.319
At least, Jupyter is supposed to be

00:07:07.320 --> 00:07:13.479
a combination of Julia, Python, and R.

00:07:13.480 --> 00:07:17.559
So we're running mcmc to get the posterior distribution

00:07:17.560 --> 00:07:21.119
and we're going to plot those out with this pyplots package,

00:07:21.120 --> 00:07:24.079
and we have these beautiful plots showing the median

00:07:24.080 --> 00:07:27.039
of the posterior distribution for four variables in –

00:07:27.040 --> 00:07:31.559
four parameters in the CARS data set,

00:07:31.560 --> 00:07:38.039
which is available – built into the R package.

00:07:38.040 --> 00:07:45.559
And then these shaded areas are the 80% interval.

00:07:45.560 --> 00:07:49.159
Okay. Oops.

NOTE Python

00:07:49.160 --> 00:08:07.639
So now for the Python side,

00:08:07.640 --> 00:08:13.359
here's an example in which I'm going to actually

00:08:13.360 --> 00:08:18.679
insert a snippet of that cell

00:08:18.680 --> 00:08:27.279
and then I'm going to enter nvlig for nglview ligand,

00:08:27.280 --> 00:08:43.319
and just hit enter. Oops. Hit TAB, excuse me,

00:08:43.320 --> 00:08:47.999
and we don't need this line of code, so delete that.

00:08:48.000 --> 00:08:52.199
Yep, we want to load up this pdb file

00:08:52.200 --> 00:08:54.279
that's in this subdirectory.

00:08:54.280 --> 00:08:56.919
So the pdb file is a plain text file

00:08:56.920 --> 00:08:58.839
that contains atomic coordinates

00:08:58.840 --> 00:09:00.159
of protein crystal structure.

00:09:00.160 --> 00:09:02.999
This protein happens to be important in cancer

00:09:03.000 --> 00:09:09.079
and we have – we screened by docking 55,000 compounds

00:09:09.080 --> 00:09:10.679
on a supercomputer

00:09:10.680 --> 00:09:12.743
and then we did MD [molecular dynamics] simulations

00:09:12.744 --> 00:09:14.159
of the top 10 [actually 20] leads.

00:09:14.160 --> 00:09:18.319
Twelve of them had the compound remain bound

00:09:18.320 --> 00:09:19.679
during the period of the simulation,

00:09:19.680 --> 00:09:22.559
so those have some potential for –

00:09:22.560 --> 00:09:25.159
and require experimental validation.

00:09:25.160 --> 00:09:28.639
So we'll run this chunk of code,

00:09:28.640 --> 00:09:30.839
and this will give a view of the molecule

00:09:30.840 --> 00:09:36.479
that we can interact with by using the mouse.

00:09:36.480 --> 00:09:39.879
But I want to share this with my colleague.

00:09:39.880 --> 00:09:42.919
My colleague is not set up to use Jupyter,

00:09:42.920 --> 00:09:46.959
but instead we can write this out to a HTML file,

00:09:46.960 --> 00:09:48.319
which I have loaded up already.

00:09:48.320 --> 00:09:55.319
And so we can actually – perhaps.

00:09:55.320 --> 00:10:01.679
We click on these two arrows pointing at each other,

00:10:01.680 --> 00:10:05.159
and we can get a full screen view of this molecule

00:10:05.160 --> 00:10:10.439
and he can identify each atom in this structure.

00:10:10.440 --> 00:10:12.719
Over a thousand atoms present.

00:10:12.720 --> 00:10:14.959
We're just hovering over a specific atom.

00:10:14.960 --> 00:10:27.759
So shown in gray is the ligand that is bound. Okay.

00:10:27.760 --> 00:10:32.079
So we still have this box selected

00:10:32.080 --> 00:10:36.559
and we still have these two different –

00:10:36.560 --> 00:10:40.519
so for each of the – our selected text areas

00:10:40.520 --> 00:11:03.839
we have a separate – we have a separate buffer open. Okay.

NOTE Julia

00:11:03.840 --> 00:11:06.839
To wrap things up here, here's an example of using –

00:11:06.840 --> 00:11:08.919
with evolving Julia code.

00:11:08.920 --> 00:11:12.919
And so this Julia code in this cell is in a Emacs buffer.

00:11:12.920 --> 00:11:19.279
So you've got an idea now, I think.

00:11:19.280 --> 00:11:23.759
So in terms of plain text areas like in Overleaf

00:11:23.760 --> 00:11:27.519
and then these cells in Jupyter Notebooks,

00:11:27.520 --> 00:11:30.959
these are other areas that can be edited

00:11:30.960 --> 00:11:37.919
like in the text areas within Outlook Webmail and Gmail.

00:11:37.920 --> 00:11:42.639
Instead of having to point with the mouse

00:11:42.640 --> 00:11:43.519
or click with the mouse,

00:11:43.520 --> 00:11:48.079
one can also use keybindings or keyboard shortcuts.

00:11:48.080 --> 00:11:51.599
So here are the ones for three major operating systems.

NOTE How does GhostText work?

00:11:51.600 --> 00:11:54.639
So how does GhostText work?

00:11:54.640 --> 00:11:57.199
Main thing is you have to open up Emacs

00:11:57.200 --> 00:11:59.679
and get this Atomic Chrome server running.

00:11:59.680 --> 00:12:03.119
And then with it up and going,

00:12:03.120 --> 00:12:06.679
GhostText will be able to – has to be activated

00:12:06.680 --> 00:12:09.639
and it will find the GhostText server

00:12:09.640 --> 00:12:15.079
through the localhost port 4001.

00:12:15.080 --> 00:12:19.039
Put that into the web browser.

00:12:19.040 --> 00:12:21.639
If you navigate to that port,

00:12:21.640 --> 00:12:23.719
you'll get output that looks like this

00:12:23.720 --> 00:12:25.639
if everything's working well.

00:12:25.640 --> 00:12:27.919
Otherwise, you'll get a error message

00:12:27.920 --> 00:12:30.519
and it should have a port socket –

00:12:30.520 --> 00:12:34.319
a web socket port number.

00:12:34.320 --> 00:12:36.319
It will not be the same every time.

NOTE Supported web browsers

00:12:36.320 --> 00:12:42.319
So these are the supported web browsers

00:12:42.320 --> 00:12:45.359
in addition to Chrome. These are supported

00:12:45.360 --> 00:12:47.279
and likewise anything in these –

00:12:47.280 --> 00:12:49.879
any browser related to these

00:12:49.880 --> 00:12:53.079
can probably use these extensions.

00:12:53.080 --> 00:12:56.359
For example, the Brave browser will use Chrome extension

00:12:56.360 --> 00:13:01.679
and the Firefox browser extension works with WaterFox.

00:13:01.680 --> 00:13:05.239
These are the supported editors.

00:13:05.240 --> 00:13:10.199
Each editor has its own extension

00:13:10.200 --> 00:13:15.719
and this GhostText was initially developed for SublimeText.

00:13:15.720 --> 00:13:17.919
So if you have SublimeText,

00:13:17.920 --> 00:13:21.959
then you can use its smooth operation as positive control

00:13:21.960 --> 00:13:24.999
when things go wrong with Emacs.

NOTE Atomic Chrome configuration

00:13:25.000 --> 00:13:29.479
This is Atomic. This is a GitHub site for Atomic Chrome.

00:13:29.480 --> 00:13:33.159
Atomic Chrome is available for installation through Melpa.

00:13:33.160 --> 00:13:40.319
This is my configuration for Atomic Chrome.

00:13:40.320 --> 00:13:45.519
So I have this setup so the server starts whenever I log in,

00:13:45.520 --> 00:13:50.319
and I have it set up so that default major mode is Python

00:13:50.320 --> 00:13:55.559
to deal with the Jupyter notebooks and Colab notebooks.

00:13:55.560 --> 00:13:57.719
And then I have major modes

00:13:57.720 --> 00:13:59.839
for these other websites defined below.

00:13:59.840 --> 00:14:06.079
This is a testing site so the developer has made

00:14:06.080 --> 00:14:08.119
to help with troubleshooting.

00:14:08.120 --> 00:14:11.159
He also has a protocol on his website

00:14:11.160 --> 00:14:13.559
to follow during troubleshooting.

NOTE Precautions

00:14:13.560 --> 00:14:16.639
So here are some precautions.

00:14:16.640 --> 00:14:19.079
You'll find that GhostText doesn't work with Pluto.

00:14:19.080 --> 00:14:21.239
Pluto is a new computational notebook

00:14:21.240 --> 00:14:23.479
for working with Julia.

00:14:23.480 --> 00:14:27.039
My suggestion would be just to run IJulia in Jupyter.

00:14:27.040 --> 00:14:31.999
It also doesn't work, of course, with RStudio.

00:14:32.000 --> 00:14:35.199
Even though RStudio sort of resembles

00:14:35.200 --> 00:14:38.999
a web page, web browser, it's not.

00:14:39.000 --> 00:14:41.359
Of course, you can always run R,

00:14:41.360 --> 00:14:48.239
as you've just seen, using the IPy kernel.

00:14:48.240 --> 00:14:52.239
I will also caution you that if you use the Emacs server,

00:14:52.240 --> 00:14:53.519
you may run into issues

00:14:53.520 --> 00:14:57.999
with the server competing with the port 4001.

00:14:58.000 --> 00:15:01.919
So instead, you should probably configure the Emacs server

00:15:01.920 --> 00:15:04.999
to use a specific port.

00:15:05.000 --> 00:15:07.639
So far – although I haven't done that myself –

00:15:07.640 --> 00:15:10.999
so far, I haven't found any conflicts

00:15:11.000 --> 00:15:13.479
with the Org Roam user interface.

NOTE Conclusions

00:15:13.480 --> 00:15:20.919
So my conclusions are: GhostText allows you to edit prose

00:15:20.920 --> 00:15:24.439
with your favorite major mode

00:15:24.440 --> 00:15:28.119
in the text areas of web pages

00:15:28.120 --> 00:15:31.079
and in the cells of Jupyter notebooks.

00:15:31.080 --> 00:15:34.359
This allows you to tap into snippets

00:15:34.360 --> 00:15:37.519
and thereby save time as you have – probably have –

00:15:37.520 --> 00:15:39.719
hopefully got an idea of.

NOTE Thanks

00:15:39.720 --> 00:15:44.039
I'd like to thank my friends and mentors

00:15:44.040 --> 00:15:46.599
who've helped me out during my second year

00:15:46.600 --> 00:15:49.359
in my Emacs learning spiral.

00:15:49.360 --> 00:15:55.839
These include my local colleagues.

00:15:55.840 --> 00:15:58.679
We meet once a month in the Oklahoma Data Science Workshop.

00:15:58.680 --> 00:16:01.719
Last July, I gave a presentation about GhostText.

00:16:01.720 --> 00:16:08.359
And then also my friends at Berlin and Austin Emacs meetups

00:16:08.360 --> 00:16:12.959
and in the UK research software engineer

00:16:12.960 --> 00:16:17.599
Emacs research Slack channel.

00:16:17.600 --> 00:16:18.919
So I don't attend these every month,

00:16:18.920 --> 00:16:21.439
but I try to make the meetings as often as I can.

00:16:21.440 --> 00:16:25.399
Then I'm supported by the following grants,

00:16:25.400 --> 00:16:28.359
which allow me to spend

00:16:28.360 --> 00:16:30.679
at least some time each day in Emacs.

00:16:30.680 --> 00:16:34.960
I'll be happy to take any questions.