summaryrefslogblamecommitdiffstats
path: root/2021/captions/emacsconf-2021-imaginary--imaginary-programming--shane-mulligan--main.vtt
blob: 5d822eae7bf0933ae8d06e7cc460c532ffd8a63a (plain) (tree)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
























































































































































































































































































































































































































































































































































































































































































































                                           
WEBVTT

00:01.920 --> 00:03.280
Hi everyone!

00:03.280 --> 00:04.400
My name is Shane,

00:04.400 --> 00:09.679
and it's truly a real honor to be

00:09.679 --> 00:14.480
able to speak at EmacsConf.

00:14.480 --> 00:16.880
I love Emacs. I love Emacs Lisp,

00:16.880 --> 00:19.279
and this is also another topic

00:19.279 --> 00:22.233
that really excites me ever since

00:22.233 --> 00:28.160
I got my hands on OpenAI's GPT-3.

00:28.160 --> 00:29.920
Six months ago, I knew that it was

00:29.920 --> 00:33.920
important to research this for Emacs,

00:33.920 --> 00:35.980
you know, asides from being somebody

00:35.980 --> 00:39.040
who never wants to leave Emacs myself,

00:39.040 --> 00:41.440
and I hope that….

00:41.440 --> 00:43.120
Being one of the few Emacs users,

00:43.120 --> 00:45.360
possibly the only one that had

00:45.360 --> 00:48.160
access to GPT-3 for this long,

00:48.160 --> 00:52.559
I hope that I can contribute.

00:52.559 --> 00:55.199
So, Emacs, sorry.

00:55.199 --> 00:58.960
Imaginary programming is…,

00:58.960 --> 01:01.280
it's based on prompt engineering currently,

01:01.280 --> 01:02.559
but it's an abstraction over

01:02.559 --> 01:03.359
prompt engineering.

01:03.359 --> 01:06.479
You can read about prompt engineering

01:06.479 --> 01:08.560
and language models,

01:08.560 --> 01:13.040
that's quite easy to google for.

01:13.040 --> 01:15.520
And imaginary programming is a subfield

01:15.520 --> 01:17.360
of imaginary computing,

01:17.360 --> 01:19.424
which is just the larger domain

01:19.424 --> 01:21.420
of employing of computing

01:21.420 --> 01:24.240
that's based on imagination,

01:24.240 --> 01:25.600
artificial imagination,

01:25.600 --> 01:29.920
so, basically, dreaming up user interfaces

01:29.920 --> 01:32.159
instead of…, or at least partially

01:32.159 --> 01:34.079
dreaming them up.

01:34.079 --> 01:41.439
I suppose that it's a fringe paradigm,

01:41.439 --> 01:43.759
but it's extremely useful.

01:43.759 --> 01:47.759
Imaginary computing,

01:47.759 --> 01:49.200
if you're willing to call it that,

01:49.200 --> 01:51.920
would be what's used to

01:51.920 --> 01:53.413
imagine simulations

01:53.413 --> 01:54.320
for self-driving cars,

01:54.320 --> 01:57.840
as they're trained for example.

01:57.840 --> 01:58.984
But this technology finds

01:58.984 --> 02:00.960
its way to the public,

02:00.960 --> 02:04.880
and it should be in the public domain.

02:04.880 --> 02:06.799
So, Demis Hassabis

02:06.799 --> 02:08.959
he's the founder of DeepMind,

02:08.959 --> 02:12.400
and he did his Ph.D. in human imagination,

02:12.400 --> 02:16.900
now he's working on artificial imagination.

02:16.900 --> 02:20.239
So, just a couple of days ago

02:20.239 --> 02:25.040
I took AlephAlpha's world model API

02:25.040 --> 02:28.080
and I plugged it into Emacs's eww browser,

02:28.080 --> 02:30.800
and now I have a way of generating

02:30.800 --> 02:33.840
text for the images instead of actually…,

02:33.840 --> 02:38.072
so I can stay in my text only Emacs,

02:38.072 --> 02:42.560
which is the way it should be.

02:42.560 --> 02:45.636
Intelligent NFTs, I'll leave this

02:45.636 --> 02:47.360
for you guys to look at.

02:47.360 --> 02:49.280
Information bubbles.

02:49.280 --> 02:55.360
So, there's some potential bad outcomes

02:55.360 --> 03:05.680
from the runaway empowering of these

03:05.680 --> 03:08.959
large language models and other models

03:08.959 --> 03:10.000
in commercial hands.

03:10.000 --> 03:13.680
It's causing information bubbles and

03:13.680 --> 03:15.680
ways of controlling people.

03:15.680 --> 03:19.440
So, for example,

03:19.440 --> 03:21.680
micro tasks and stuff that…,

03:21.680 --> 03:23.650
furthermore just automating away,

03:23.650 --> 03:26.000
or abstracting away the role

03:26.000 --> 03:27.680
of a programmer,

03:27.680 --> 03:29.360
and the automating more and more

03:29.360 --> 03:32.319
increasingly abstract tasks,

03:32.319 --> 03:33.659
and I think the solution is to

03:33.659 --> 03:36.319
decentralize and break up these tasks.

03:36.319 --> 03:39.120
I have a potential way of doing that,

03:39.120 --> 03:40.693
but firstly I'll talk about

03:40.693 --> 03:42.400
the imaginary web briefly

03:42.400 --> 03:44.720
because the thing about these

03:44.720 --> 03:48.000
language models is,

03:48.000 --> 03:49.280
they can replace basically

03:49.280 --> 03:52.159
everything on the internet.

03:52.159 --> 03:55.280
So, like, replace your Wikipedia,

03:55.280 --> 03:57.680
or Stack Overflow,

03:57.680 --> 04:01.519
replace conversation if you want it with,

04:01.519 --> 04:05.920
from real people to chatbots instead,

04:05.920 --> 04:13.120
replace basically anything there's

04:13.120 --> 04:14.959
a website for.

04:14.959 --> 04:17.280
And that means that rich media has gone

04:17.280 --> 04:20.639
from becoming images and video

04:20.639 --> 04:22.560
and even from paywalls

04:22.560 --> 04:26.400
now into intelligent and truthful,

04:26.400 --> 04:29.520
because generating fictional websites

04:29.520 --> 04:31.919
is going to become a very easy thing to do,

04:31.919 --> 04:35.120
and actually the best way to do it,

04:35.120 --> 04:37.440
and the most useful way to do it,

04:37.440 --> 04:40.560
so then you need a source of truth.

04:40.560 --> 04:43.759
The imaginary web is a thing,

04:43.759 --> 04:46.000
imaginary interpreters are a thing,

04:46.000 --> 04:50.000
so you imagine your interpreter,

04:50.000 --> 04:53.759
or you overlay prompting on top of

04:53.759 --> 04:55.520
a real interpreter to see

04:55.520 --> 04:57.440
what you might want to do

04:57.440 --> 04:58.800
in an interpreter,

04:58.800 --> 05:01.039
and what you might want to say to somebody

05:01.039 --> 05:01.919
if you're talking to them.

05:01.919 --> 05:03.199
If you're inside, say,

05:03.199 --> 05:05.840
ERC in Emacs, an IRC client,

05:05.840 --> 05:07.440
and you prompt after somebody

05:07.440 --> 05:08.479
says something,

05:08.479 --> 05:10.400
then the prompt will probably suggest,

05:10.400 --> 05:13.120
what you might say in return,

05:13.120 --> 05:15.199
and then you can prompt like a multiverse,

05:15.199 --> 05:18.800
and you can pick from them.

05:18.800 --> 05:23.912
There's a bunch of crazy utilities

05:23.912 --> 05:26.560
for these language models.

05:26.560 --> 05:28.960
Paracosm vs Metaverse.

05:28.960 --> 05:30.828
So, Mark Zuck wants you to live in

05:30.828 --> 05:34.320
his virtual reality as defined by him,

05:34.320 --> 05:36.160
and he's going to do it

05:36.160 --> 05:40.539
by using these models to make you

05:40.539 --> 05:42.639
a fictional world that you can live in.

05:42.639 --> 05:43.280
You can do that,

05:43.280 --> 05:46.400
or you can use them yourself,

05:46.400 --> 05:50.000
and you build your own Paracosm.

05:50.000 --> 05:51.440
And I think that's an important

05:51.440 --> 05:53.440
ability to be able to have,

05:53.440 --> 05:56.880
otherwise we will be like the borg,

05:56.880 --> 06:01.520
and we'll be connected to Mark Zuckerberg.

06:01.520 --> 06:06.400
Truth is a hot topic.

06:06.400 --> 06:08.880
So, the way that I think we should do this

06:08.880 --> 06:10.720
to decentralize the language models is,

06:10.720 --> 06:13.199
to use Structuralism.

06:13.199 --> 06:19.600
Universal grammar,

06:19.600 --> 06:21.680
template metaprogramming, and GPT-3

06:21.680 --> 06:23.919
what do they have in common?

06:23.919 --> 06:27.759
Well, you have some kind of basis,

06:27.759 --> 06:31.408
like, you train your GPT-3,

06:31.408 --> 06:32.663
and then you do all your prompting

06:32.663 --> 06:33.360
on top of it.

06:33.360 --> 06:35.840
Like a person is born with this grammar,

06:35.840 --> 06:37.759
and then they quickly learn language,

06:37.759 --> 06:40.240
and like with C++ templates,

06:40.240 --> 06:42.080
you pre-process and then

06:42.080 --> 06:44.800
the run time runs on that.

06:44.800 --> 06:48.000
So, anyway, that was a slide.

06:48.000 --> 06:50.479
Structuralism.

06:50.479 --> 06:52.080
I think you can decompile…,

06:52.080 --> 06:52.880
think you can break,

06:52.880 --> 06:53.840
so you can decompose

06:53.840 --> 06:55.840
the language models into units,

06:55.840 --> 06:57.759
but those units won't look like neurons,

06:57.759 --> 06:59.440
they would look like these,

06:59.440 --> 07:01.759
and you put them onto a blockchain.

07:01.759 --> 07:02.744
But you can look at that later

07:02.744 --> 07:03.840
if you want anyway.

07:03.840 --> 07:07.199
I'm going to skip straight to ilambda,

07:07.199 --> 07:09.919
running out of time.

07:09.919 --> 07:12.240
I'll just quickly show you

07:12.240 --> 07:15.199
the ilambda primitive in ilambda.

07:15.199 --> 07:19.199
So, it evaluates instead of run,

07:19.199 --> 07:21.680
for example, here's the reduced function,

07:21.680 --> 07:22.632
and you've defined your

07:22.632 --> 07:25.520
imaginary lambda here.

07:25.520 --> 07:26.560
And it doesn't have a body,

07:26.560 --> 07:31.599
it's just got the comment

07:31.599 --> 07:33.120
and the parameters,

07:33.120 --> 07:36.560
and that's enough for….

07:36.560 --> 07:38.720
Once you have that ilambda

07:38.720 --> 07:40.080
that runs now as a function,

07:40.080 --> 07:40.800
you can stick it into

07:40.800 --> 07:42.720
a reduced function, for example,

07:42.720 --> 07:46.240
and it'll reduce this list.

07:46.240 --> 07:50.560
You could even remove…, yeah,

07:50.560 --> 07:51.759
so you kind of need the comment

07:51.759 --> 07:56.048
otherwise it's too hard to imagine

07:56.048 --> 07:57.360
what would happen next,

07:57.360 --> 08:01.520
but for a function you can literally

08:01.520 --> 08:03.360
have an idefun even without

08:03.360 --> 08:04.639
the argument list.

08:04.639 --> 08:06.319
It was just like generate-fib-sequence,

08:06.319 --> 08:09.919
and most likely when you run that defun,

08:09.919 --> 08:13.759
it would work the way you want it.

08:13.759 --> 08:16.000
The more information you give the idefun,

08:16.000 --> 08:17.840
the imaginary defun,

08:17.840 --> 08:20.879
the better it would capture the task

08:20.879 --> 08:22.000
which you're trying to do,

08:22.000 --> 08:23.039
in this case you want to generate

08:23.039 --> 08:24.080
a Fibonacci sequence.

08:24.080 --> 08:28.319
And yeah, you can define functions

08:28.319 --> 08:30.560
without having a body,

08:30.560 --> 08:32.640
and they run an inference instead.

08:32.640 --> 08:34.159
Here's a way of overriding

08:34.159 --> 08:36.000
the language model that's used,

08:36.000 --> 08:40.640
for example, using dynamic scope.

08:40.640 --> 08:41.863
So, under the hood,

08:41.863 --> 08:47.040
idefun just uses an ilambda.

08:47.040 --> 08:50.959
This function here just doubles things.

08:50.959 --> 08:53.752
So, here's a function that gets you

08:53.752 --> 08:59.279
a hexadecimal color just from the name.

08:59.279 --> 09:00.160
And you can create

09:00.160 --> 09:02.160
arbitrary functions like this,

09:02.160 --> 09:04.268
so, what we need is like a library

09:04.268 --> 09:07.200
of imaginary functions, I think,

09:07.200 --> 09:12.800
that match a language model.

09:12.800 --> 09:15.760
Macros on the other hand,

09:15.760 --> 09:18.160
as different from functions,

09:18.160 --> 09:20.000
they actually macro expand

09:20.000 --> 09:21.360
and generate code.

09:21.360 --> 09:23.279
So, when you macro expand this,

09:23.279 --> 09:26.320
you'll get this,

09:26.320 --> 09:30.000
and that's because this has an arity of 3,

09:30.000 --> 09:31.839
and then when you macro expand

09:31.839 --> 09:32.959
that underlying macro,

09:32.959 --> 09:35.120
it generates the actual source code.

09:35.120 --> 09:37.839
You can actually run these macros,

09:37.839 --> 09:41.279
and it will cache the output…,

09:41.279 --> 09:42.480
it will catch the source code,

09:42.480 --> 09:44.240
so the macro runs the same

09:44.240 --> 09:45.839
every single time,

09:45.839 --> 09:47.519
or generates the same code,

09:47.519 --> 09:48.560
but you can just use it

09:48.560 --> 09:49.920
to generate code really easily

09:49.920 --> 09:52.000
while you're programming.

09:52.000 --> 09:55.920
I hope that this has been informative,

09:55.920 --> 09:57.600
it wasn't too much time,

09:57.600 --> 10:00.312
but there's plenty of material

10:00.312 --> 10:01.680
for you to dig into it more

10:01.680 --> 10:04.000
if you're interested.

10:04.000 --> 10:04.959
Thank you very much

10:04.959 --> 10:09.519
for letting me talk today.

10:09.519 --> 10:15.680
Peace out!

10:15.680 --> 10:16.680
[captions by bhavin192 (Bhavin Gandhi)]