path: root/2021/captions/emacsconf-2021-imaginary--imaginary-programming--shane-mulligan--main.vtt

WEBVTT

00:01.920 --> 00:03.280
Hi everyone!

00:03.280 --> 00:04.400
My name is Shane,

00:04.400 --> 00:09.679
and it's truly a real honor to be

00:09.679 --> 00:14.480
able to speak at EmacsConf.

00:14.480 --> 00:16.880
I love Emacs. I love Emacs Lisp,

00:16.880 --> 00:19.279
and this is also another topic

00:19.279 --> 00:22.233
that really excites me ever since

00:22.233 --> 00:28.160
I got my hands on OpenAI's GPT-3.

00:28.160 --> 00:29.920
Six months ago, I knew that it was

00:29.920 --> 00:33.920
important to research this for Emacs,

00:33.920 --> 00:35.980
you know, asides from being somebody

00:35.980 --> 00:39.040
who never wants to leave Emacs myself,

00:39.040 --> 00:41.440
and I hope that….

00:41.440 --> 00:43.120
Being one of the few Emacs users,

00:43.120 --> 00:45.360
possibly the only one that had

00:45.360 --> 00:48.160
access to GPT-3 for this long,

00:48.160 --> 00:52.559
I hope that I can contribute.

00:52.559 --> 00:55.199
So, Emacs, sorry.

00:55.199 --> 00:58.960
Imaginary programming is…,

00:58.960 --> 01:01.280
it's based on prompt engineering currently,

01:01.280 --> 01:02.559
but it's an abstraction over

01:02.559 --> 01:03.359
prompt engineering.

01:03.359 --> 01:06.479
You can read about prompt engineering

01:06.479 --> 01:08.560
and language models,

01:08.560 --> 01:13.040
that's quite easy to google for.

01:13.040 --> 01:15.520
And imaginary programming is a subfield

01:15.520 --> 01:17.360
of imaginary computing,

01:17.360 --> 01:19.424
which is just the larger domain

01:19.424 --> 01:21.420
of employing of computing

01:21.420 --> 01:24.240
that's based on imagination,

01:24.240 --> 01:25.600
artificial imagination,

01:25.600 --> 01:29.920
so, basically, dreaming up user interfaces

01:29.920 --> 01:32.159
instead of…, or at least partially

01:32.159 --> 01:34.079
dreaming them up.

01:34.079 --> 01:41.439
I suppose that it's a fringe paradigm,

01:41.439 --> 01:43.759
but it's extremely useful.

01:43.759 --> 01:47.759
Imaginary computing,

01:47.759 --> 01:49.200
if you're willing to call it that,

01:49.200 --> 01:51.920
would be what's used to

01:51.920 --> 01:53.413
imagine simulations

01:53.413 --> 01:54.320
for self-driving cars,

01:54.320 --> 01:57.840
as they're trained for example.

01:57.840 --> 01:58.984
But this technology finds

01:58.984 --> 02:00.960
its way to the public,

02:00.960 --> 02:04.880
and it should be in the public domain.

02:04.880 --> 02:06.799
So, Demis Hassabis

02:06.799 --> 02:08.959
he's the founder of DeepMind,

02:08.959 --> 02:12.400
and he did his Ph.D. in human imagination,

02:12.400 --> 02:16.900
now he's working on artificial imagination.

02:16.900 --> 02:20.239
So, just a couple of days ago

02:20.239 --> 02:25.040
I took AlephAlpha's world model API

02:25.040 --> 02:28.080
and I plugged it into Emacs's eww browser,

02:28.080 --> 02:30.800
and now I have a way of generating

02:30.800 --> 02:33.840
text for the images instead of actually…,

02:33.840 --> 02:38.072
so I can stay in my text only Emacs,

02:38.072 --> 02:42.560
which is the way it should be.

02:42.560 --> 02:45.636
Intelligent NFTs, I'll leave this

02:45.636 --> 02:47.360
for you guys to look at.

02:47.360 --> 02:49.280
Information bubbles.

02:49.280 --> 02:55.360
So, there's some potential bad outcomes

02:55.360 --> 03:05.680
from the runaway empowering of these

03:05.680 --> 03:08.959
large language models and other models

03:08.959 --> 03:10.000
in commercial hands.

03:10.000 --> 03:13.680
It's causing information bubbles and

03:13.680 --> 03:15.680
ways of controlling people.

03:15.680 --> 03:19.440
So, for example,

03:19.440 --> 03:21.680
micro tasks and stuff that…,

03:21.680 --> 03:23.650
furthermore just automating away,

03:23.650 --> 03:26.000
or abstracting away the role

03:26.000 --> 03:27.680
of a programmer,

03:27.680 --> 03:29.360
and the automating more and more

03:29.360 --> 03:32.319
increasingly abstract tasks,

03:32.319 --> 03:33.659
and I think the solution is to

03:33.659 --> 03:36.319
decentralize and break up these tasks.

03:36.319 --> 03:39.120
I have a potential way of doing that,

03:39.120 --> 03:40.693
but firstly I'll talk about

03:40.693 --> 03:42.400
the imaginary web briefly

03:42.400 --> 03:44.720
because the thing about these

03:44.720 --> 03:48.000
language models is,

03:48.000 --> 03:49.280
they can replace basically

03:49.280 --> 03:52.159
everything on the internet.

03:52.159 --> 03:55.280
So, like, replace your Wikipedia,

03:55.280 --> 03:57.680
or Stack Overflow,

03:57.680 --> 04:01.519
replace conversation if you want it with,

04:01.519 --> 04:05.920
from real people to chatbots instead,

04:05.920 --> 04:13.120
replace basically anything there's

04:13.120 --> 04:14.959
a website for.

04:14.959 --> 04:17.280
And that means that rich media has gone

04:17.280 --> 04:20.639
from becoming images and video

04:20.639 --> 04:22.560
and even from paywalls

04:22.560 --> 04:26.400
now into intelligent and truthful,

04:26.400 --> 04:29.520
because generating fictional websites

04:29.520 --> 04:31.919
is going to become a very easy thing to do,

04:31.919 --> 04:35.120
and actually the best way to do it,

04:35.120 --> 04:37.440
and the most useful way to do it,

04:37.440 --> 04:40.560
so then you need a source of truth.

04:40.560 --> 04:43.759
The imaginary web is a thing,

04:43.759 --> 04:46.000
imaginary interpreters are a thing,

04:46.000 --> 04:50.000
so you imagine your interpreter,

04:50.000 --> 04:53.759
or you overlay prompting on top of

04:53.759 --> 04:55.520
a real interpreter to see

04:55.520 --> 04:57.440
what you might want to do

04:57.440 --> 04:58.800
in an interpreter,

04:58.800 --> 05:01.039
and what you might want to say to somebody

05:01.039 --> 05:01.919
if you're talking to them.

05:01.919 --> 05:03.199
If you're inside, say,

05:03.199 --> 05:05.840
ERC in Emacs, an IRC client,

05:05.840 --> 05:07.440
and you prompt after somebody

05:07.440 --> 05:08.479
says something,

05:08.479 --> 05:10.400
then the prompt will probably suggest,

05:10.400 --> 05:13.120
what you might say in return,

05:13.120 --> 05:15.199
and then you can prompt like a multiverse,

05:15.199 --> 05:18.800
and you can pick from them.

05:18.800 --> 05:23.912
There's a bunch of crazy utilities

05:23.912 --> 05:26.560
for these language models.

05:26.560 --> 05:28.960
Paracosm vs Metaverse.

05:28.960 --> 05:30.828
So, Mark Zuck wants you to live in

05:30.828 --> 05:34.320
his virtual reality as defined by him,

05:34.320 --> 05:36.160
and he's going to do it

05:36.160 --> 05:40.539
by using these models to make you

05:40.539 --> 05:42.639
a fictional world that you can live in.

05:42.639 --> 05:43.280
You can do that,

05:43.280 --> 05:46.400
or you can use them yourself,

05:46.400 --> 05:50.000
and you build your own Paracosm.

05:50.000 --> 05:51.440
And I think that's an important

05:51.440 --> 05:53.440
ability to be able to have,

05:53.440 --> 05:56.880
otherwise we will be like the borg,

05:56.880 --> 06:01.520
and we'll be connected to Mark Zuckerberg.

06:01.520 --> 06:06.400
Truth is a hot topic.

06:06.400 --> 06:08.880
So, the way that I think we should do this

06:08.880 --> 06:10.720
to decentralize the language models is,

06:10.720 --> 06:13.199
to use Structuralism.

06:13.199 --> 06:19.600
Universal grammar,

06:19.600 --> 06:21.680
template metaprogramming, and GPT-3

06:21.680 --> 06:23.919
what do they have in common?

06:23.919 --> 06:27.759
Well, you have some kind of basis,

06:27.759 --> 06:31.408
like, you train your GPT-3,

06:31.408 --> 06:32.663
and then you do all your prompting

06:32.663 --> 06:33.360
on top of it.

06:33.360 --> 06:35.840
Like a person is born with this grammar,

06:35.840 --> 06:37.759
and then they quickly learn language,

06:37.759 --> 06:40.240
and like with C++ templates,

06:40.240 --> 06:42.080
you pre-process and then

06:42.080 --> 06:44.800
the run time runs on that.

06:44.800 --> 06:48.000
So, anyway, that was a slide.

06:48.000 --> 06:50.479
Structuralism.

06:50.479 --> 06:52.080
I think you can decompile…,

06:52.080 --> 06:52.880
think you can break,

06:52.880 --> 06:53.840
so you can decompose

06:53.840 --> 06:55.840
the language models into units,

06:55.840 --> 06:57.759
but those units won't look like neurons,

06:57.759 --> 06:59.440
they would look like these,

06:59.440 --> 07:01.759
and you put them onto a blockchain.

07:01.759 --> 07:02.744
But you can look at that later

07:02.744 --> 07:03.840
if you want anyway.

07:03.840 --> 07:07.199
I'm going to skip straight to ilambda,

07:07.199 --> 07:09.919
running out of time.

07:09.919 --> 07:12.240
I'll just quickly show you

07:12.240 --> 07:15.199
the ilambda primitive in ilambda.

07:15.199 --> 07:19.199
So, it evaluates instead of run,

07:19.199 --> 07:21.680
for example, here's the reduced function,

07:21.680 --> 07:22.632
and you've defined your

07:22.632 --> 07:25.520
imaginary lambda here.

07:25.520 --> 07:26.560
And it doesn't have a body,

07:26.560 --> 07:31.599
it's just got the comment

07:31.599 --> 07:33.120
and the parameters,

07:33.120 --> 07:36.560
and that's enough for….

07:36.560 --> 07:38.720
Once you have that ilambda

07:38.720 --> 07:40.080
that runs now as a function,

07:40.080 --> 07:40.800
you can stick it into

07:40.800 --> 07:42.720
a reduced function, for example,

07:42.720 --> 07:46.240
and it'll reduce this list.

07:46.240 --> 07:50.560
You could even remove…, yeah,

07:50.560 --> 07:51.759
so you kind of need the comment

07:51.759 --> 07:56.048
otherwise it's too hard to imagine

07:56.048 --> 07:57.360
what would happen next,

07:57.360 --> 08:01.520
but for a function you can literally

08:01.520 --> 08:03.360
have an idefun even without

08:03.360 --> 08:04.639
the argument list.

08:04.639 --> 08:06.319
It was just like generate-fib-sequence,

08:06.319 --> 08:09.919
and most likely when you run that defun,

08:09.919 --> 08:13.759
it would work the way you want it.

08:13.759 --> 08:16.000
The more information you give the idefun,

08:16.000 --> 08:17.840
the imaginary defun,

08:17.840 --> 08:20.879
the better it would capture the task

08:20.879 --> 08:22.000
which you're trying to do,

08:22.000 --> 08:23.039
in this case you want to generate

08:23.039 --> 08:24.080
a Fibonacci sequence.

08:24.080 --> 08:28.319
And yeah, you can define functions

08:28.319 --> 08:30.560
without having a body,

08:30.560 --> 08:32.640
and they run an inference instead.

08:32.640 --> 08:34.159
Here's a way of overriding

08:34.159 --> 08:36.000
the language model that's used,

08:36.000 --> 08:40.640
for example, using dynamic scope.

08:40.640 --> 08:41.863
So, under the hood,

08:41.863 --> 08:47.040
idefun just uses an ilambda.

08:47.040 --> 08:50.959
This function here just doubles things.

08:50.959 --> 08:53.752
So, here's a function that gets you

08:53.752 --> 08:59.279
a hexadecimal color just from the name.

08:59.279 --> 09:00.160
And you can create

09:00.160 --> 09:02.160
arbitrary functions like this,

09:02.160 --> 09:04.268
so, what we need is like a library

09:04.268 --> 09:07.200
of imaginary functions, I think,

09:07.200 --> 09:12.800
that match a language model.

09:12.800 --> 09:15.760
Macros on the other hand,

09:15.760 --> 09:18.160
as different from functions,

09:18.160 --> 09:20.000
they actually macro expand

09:20.000 --> 09:21.360
and generate code.

09:21.360 --> 09:23.279
So, when you macro expand this,

09:23.279 --> 09:26.320
you'll get this,

09:26.320 --> 09:30.000
and that's because this has an arity of 3,

09:30.000 --> 09:31.839
and then when you macro expand

09:31.839 --> 09:32.959
that underlying macro,

09:32.959 --> 09:35.120
it generates the actual source code.

09:35.120 --> 09:37.839
You can actually run these macros,

09:37.839 --> 09:41.279
and it will cache the output…,

09:41.279 --> 09:42.480
it will catch the source code,

09:42.480 --> 09:44.240
so the macro runs the same

09:44.240 --> 09:45.839
every single time,

09:45.839 --> 09:47.519
or generates the same code,

09:47.519 --> 09:48.560
but you can just use it

09:48.560 --> 09:49.920
to generate code really easily

09:49.920 --> 09:52.000
while you're programming.

09:52.000 --> 09:55.920
I hope that this has been informative,

09:55.920 --> 09:57.600
it wasn't too much time,

09:57.600 --> 10:00.312
but there's plenty of material

10:00.312 --> 10:01.680
for you to dig into it more

10:01.680 --> 10:04.000
if you're interested.

10:04.000 --> 10:04.959
Thank you very much

10:04.959 --> 10:09.519
for letting me talk today.

10:09.519 --> 10:15.680
Peace out!

10:15.680 --> 10:16.680
[captions by bhavin192 (Bhavin Gandhi)]