path: root/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt

WEBVTT captioned by bala, checked by sachac

NOTE Intro to the Talk

00:00:00.000 --> 00:00:04.159
Hello, I'm Andrew Hyatt and I'm going to talk to you

00:00:04.160 --> 00:00:06.439
about large language models and how

00:00:06.440 --> 00:00:11.079
they relate to Emacs.

00:00:11.080 --> 00:00:14.919
And I'm going to talk to you about the technology

00:00:14.920 --> 00:00:18.279
and how we're going to use it in Emacs.

00:00:18.280 --> 00:00:21.159
There'll be demos and there'll be talks about,

00:00:21.160 --> 00:00:22.879
I'll finish up by kind of talking about where

00:00:22.880 --> 00:00:25.079
I think this should go in the future.

NOTE What are LLMs?

00:00:25.080 --> 00:00:28.239
So to start off with, let's just talk like,

00:00:28.240 --> 00:00:29.759
I just want to make sure everyone's on the same page.

00:00:29.760 --> 00:00:30.919
What are large language models?

00:00:30.920 --> 00:00:34.639
Not everyone may be caught up on this.

00:00:34.640 --> 00:00:38.999
Large language models are a way... Basically,

00:00:39.000 --> 00:00:42.999
the current versions of large language models

00:00:43.000 --> 00:00:44.479
are all based on the similar architecture

00:00:44.480 --> 00:00:45.279
called the transformer.

00:00:45.280 --> 00:00:48.719
It's just an efficient way to train and produce output.

00:00:48.720 --> 00:00:51.919
So these things are basically models

00:00:51.920 --> 00:00:58.079
that predict the next word or something like that.

00:00:58.080 --> 00:01:02.119
And they're trained on an enormous corpus of information

00:01:02.120 --> 00:01:04.319
and they get extremely good

00:01:04.320 --> 00:01:06.079
at predicting the next word.

00:01:06.080 --> 00:01:09.679
And from that basic ability, you can train

00:01:09.680 --> 00:01:12.439
through further tuning from human input,

00:01:12.440 --> 00:01:13.959
human ratings and things like that.

00:01:13.960 --> 00:01:17.479
You can train different models based on that

00:01:17.480 --> 00:01:18.759
that will do question answering.

00:01:18.760 --> 00:01:22.519
And this is how basically ChatGPT works.

00:01:22.520 --> 00:01:25.599
There's a base LLM, like GPT.

00:01:25.600 --> 00:01:27.799
And then you have a chat version of that,

00:01:27.800 --> 00:01:29.959
which is just trained to just... You give

00:01:29.960 --> 00:01:32.199
it a prompt, like what do you want it to do?

00:01:32.200 --> 00:01:37.279
And it gives you an output that does what you told it to do,

00:01:37.280 --> 00:01:39.919
or at least attempts to do it.

00:01:39.920 --> 00:01:42.079
Those are the power of large language models is

00:01:42.080 --> 00:01:45.639
they're extremely, extremely impressive.

00:01:45.640 --> 00:01:47.199
Certainly this is, in AI,

00:01:47.200 --> 00:01:49.079
this has been the biggest thing to happen

00:01:49.080 --> 00:01:51.559
probably in my lifetime,

00:01:51.560 --> 00:01:56.359
or at least my lifetime as my working lifetime.

NOTE Power of LLMs (Magit Demo)

00:01:56.360 --> 00:02:02.559
So let me give you a demonstration of

00:02:02.560 --> 00:02:06.679
what kinds of stuff it could do in Emacs.

00:02:06.680 --> 00:02:09.039
So here I have a Emacs file.

00:02:09.040 --> 00:02:12.479
So this is my Emacs init file.

00:02:12.480 --> 00:02:13.599
I have a change.

00:02:13.600 --> 00:02:16.879
Let's commit that change.

00:02:16.880 --> 00:02:19.439
And, you know, I don't like writing commit messages,

00:02:19.440 --> 00:02:23.039
so I can generate it.

00:02:23.040 --> 00:02:27.479
And it did an actually just looking.

00:02:27.480 --> 00:02:29.759
So all it does is it's looking, it's just reading the diff.

00:02:29.760 --> 00:02:32.479
I'm just feeding it the diff with some instructions.

00:02:32.480 --> 00:02:37.759
And it is this a incredible commit message?

00:02:37.760 --> 00:02:39.399
It's not bad, actually.

00:02:39.400 --> 00:02:42.319
You can see that it actually has really extracted

00:02:42.320 --> 00:02:46.439
the meaning of what I'm doing and has written

00:02:46.440 --> 00:02:48.879
a reasonably good commit message.

00:02:48.880 --> 00:02:53.159
Now I have to edit it because this is not quite correct.

00:02:53.160 --> 00:02:55.159
But it's kind of impressive how good it is.

00:02:55.160 --> 00:03:00.039
And my editing, it's kind of easier for me to edit this

00:03:00.040 --> 00:03:01.879
than just to write a new one.

00:03:01.880 --> 00:03:04.479
And quite often it's good enough to just submit as is.

00:03:04.480 --> 00:03:08.119
So this is kind of, you know, you could say

00:03:08.120 --> 00:03:09.359
this is just commit messages.

00:03:09.360 --> 00:03:10.719
You could respond to emails.

00:03:10.720 --> 00:03:15.319
You could, you know, using your own custom instructions

00:03:15.320 --> 00:03:17.839
about what you want your email to say.

00:03:17.840 --> 00:03:19.039
It'll write the email for you.

00:03:19.040 --> 00:03:19.839
It could do like this

00:03:19.840 --> 00:03:22.519
Emacs is a way to interact with buffers.

00:03:22.520 --> 00:03:24.199
This could basically just output text.

00:03:24.200 --> 00:03:27.759
So it's super useful for

00:03:27.760 --> 00:03:30.319
understanding something and outputting text based on that,

00:03:30.320 --> 00:03:32.239
which is just useful for Emacs.

NOTE Drawbacks of LLMs (regex demo)

00:03:32.240 --> 00:03:39.919
So the drawback is, yeah, it's good,

00:03:39.920 --> 00:03:43.359
but it's not that reliable.

00:03:43.360 --> 00:03:45.679
And you'd think it's very easy to get caught up in like,

00:03:45.680 --> 00:03:47.639
oh my gosh, like this is so powerful.

00:03:47.640 --> 00:03:50.599
I bet it could work this, whatever idea could work.

00:03:50.600 --> 00:03:52.919
And these ideas, like they almost can.

00:03:52.920 --> 00:03:55.639
For example, I was thinking, you know what I could do?

00:03:55.640 --> 00:03:57.239
I don't like writing regexes.

00:03:57.240 --> 00:04:01.199
Why can't I have a regex replace that's powered by LLMs?

00:04:01.200 --> 00:04:03.439
And that way I could give just an instruction

00:04:03.440 --> 00:04:07.399
to regex replace.

00:04:07.400 --> 00:04:12.079
And so for example, I could do Emacs LLM regex replace.

00:04:12.080 --> 00:04:12.879
This is not checked in anywhere.

00:04:12.880 --> 00:04:17.199
These are just my own kind of private functions.

00:04:17.200 --> 00:04:19.239
My description lowercase all the org headings.

00:04:19.240 --> 00:04:20.439
Let's see if it works.

00:04:20.440 --> 00:04:21.039
It might work.

00:04:21.040 --> 00:04:22.959
No, it doesn't work.

00:04:22.960 --> 00:04:26.159
So if I, I'm not going to bother to show you

00:04:26.160 --> 00:04:28.159
what it actually came up with, but it's something,

00:04:28.160 --> 00:04:29.879
if you looked at it, it'd be like, wow,

00:04:29.880 --> 00:04:31.639
this is very close to being...

00:04:31.640 --> 00:04:34.239
It looks like it should work, but it doesn't.

00:04:34.240 --> 00:04:35.839
Okay.

00:04:35.840 --> 00:04:38.719
It's not quite good enough to get it right.

00:04:38.720 --> 00:04:41.599
And it's possible that perhaps by giving it

00:04:41.600 --> 00:04:43.639
a few examples of, or explaining more

00:04:43.640 --> 00:04:46.439
what makes Emacs regexes different.

00:04:46.440 --> 00:04:47.959
It could do a better job

00:04:47.960 --> 00:04:49.279
and maybe could solve these problems,

00:04:49.280 --> 00:04:50.679
but it's always a little bit random.

00:04:50.680 --> 00:04:52.359
You're never quite sure what you're going to get.

00:04:52.360 --> 00:04:54.839
So this is the drawback.

00:04:54.840 --> 00:04:58.479
Like there's a lot of things that look like you could do it,

00:04:58.480 --> 00:05:00.999
but when it actually comes down to trying it,

00:05:01.000 --> 00:05:03.399
it's surprisingly hard.

00:05:03.400 --> 00:05:06.319
And, you know, and whatever you're doing,

00:05:06.320 --> 00:05:08.999
it's surprisingly hard to get something

00:05:09.000 --> 00:05:13.879
that is repeatably, that's, that is always good.

00:05:13.880 --> 00:05:20.119
So yeah, that's currently the problem.

NOTE Embeddings

00:05:20.120 --> 00:05:23.399
So I want to talk about embeddings.

00:05:23.400 --> 00:05:26.919
They're another thing that LLMs offer

00:05:26.920 --> 00:05:28.599
and that are extremely useful.

00:05:28.600 --> 00:05:33.119
They are, what they do is they encode from

00:05:33.120 --> 00:05:38.959
a input text that could be a word, a sentence,

00:05:38.960 --> 00:05:42.159
a small document.

00:05:42.160 --> 00:05:45.399
It encodes a vector about what the meaning,

00:05:45.400 --> 00:05:46.919
the semantic meaning of that is.

00:05:46.920 --> 00:05:51.079
That means you could, something that is,

00:05:51.080 --> 00:05:52.279
uses completely different words,

00:05:52.280 --> 00:05:54.159
but is basically talking about the same thing,

00:05:54.160 --> 00:05:57.839
perhaps in a different language, should be pretty close

00:05:57.840 --> 00:06:01.999
as a vector to the other vector.

00:06:02.000 --> 00:06:05.399
You know, as long as they're similarly semantic things,

00:06:05.400 --> 00:06:12.239
like the words

00:06:12.240 --> 00:06:18.959
highway and Camino are two different words.

00:06:18.960 --> 00:06:19.639
They mean the same thing.

00:06:19.640 --> 00:06:21.319
They should have very similar embeddings.

00:06:21.320 --> 00:06:25.119
So it is a way to kind of encode this

00:06:25.120 --> 00:06:26.199
and then you could use this for search.

00:06:26.200 --> 00:06:28.919
For example, I haven't tried to do this yet,

00:06:28.920 --> 00:06:31.479
but you could probably just make an embedding

00:06:31.480 --> 00:06:33.919
for every paragraph in the Emacs manual

00:06:33.920 --> 00:06:36.239
and the Elisp manual.

00:06:36.240 --> 00:06:39.439
And then, and then there's a very standard technique.

00:06:39.440 --> 00:06:43.439
You just... You find that you have a query,

00:06:43.440 --> 00:06:45.799
oh, how do I do whatever, whatever in Emacs again?

00:06:45.800 --> 00:06:49.479
And you could, you just find that 20 things

00:06:49.480 --> 00:06:50.319
that are closest to whatever you're

00:06:50.320 --> 00:06:51.839
trying to... the embedding of your query.

00:06:51.840 --> 00:06:55.279
You send those things to the LLM, as you know,

00:06:55.280 --> 00:06:57.799
with the original query,

00:06:57.800 --> 00:06:59.919
and you're basically telling the--asking the LLM,

00:06:59.920 --> 00:07:01.279
look, the user is trying to do this.

00:07:01.280 --> 00:07:03.039
Here's what I found in the Emacs manual.

00:07:03.040 --> 00:07:04.639
That's on the Elisp manual.

00:07:04.640 --> 00:07:07.439
That's close to what they're trying to do.

00:07:07.440 --> 00:07:12.159
So can you kind of just tell the user what to do?

00:07:12.160 --> 00:07:14.479
And from this, and you could say,

00:07:14.480 --> 00:07:17.639
just use things from this, you know, that I give you.

00:07:17.640 --> 00:07:20.679
Don't just make up your own idea.

00:07:20.680 --> 00:07:21.839
You know, don't use your own ideas,

00:07:21.840 --> 00:07:23.799
because sometimes it likes to do that

00:07:23.800 --> 00:07:24.359
and those things are wrong.

00:07:24.360 --> 00:07:26.719
So you could try to, you know, do this and you get,

00:07:26.720 --> 00:07:28.719
you could get quite good results using this.

00:07:28.720 --> 00:07:29.999
So no one has done this yet,

00:07:30.000 --> 00:07:32.799
but that should not be hard to do.

NOTE Image Generation

00:07:32.800 --> 00:07:34.879
Image generation is something that's, you know,

00:07:34.880 --> 00:07:38.479
it's not quite an LLM in the sense of...

00:07:38.480 --> 00:07:43.079
These are... It's a different technology,

00:07:43.080 --> 00:07:48.439
but these things are kind of packaged together

00:07:48.440 --> 00:07:49.039
in a sense.

00:07:49.040 --> 00:07:51.639
And you'll see that when I talk about Emacs packages,

00:07:51.640 --> 00:07:54.279
a lot of them bundle image generation

00:07:54.280 --> 00:07:55.439
and large language models.

00:07:55.440 --> 00:07:59.039
You know, the APIs are often bundled together by providers.

00:07:59.040 --> 00:08:02.679
And the general idea is it's kind of similar

00:08:02.680 --> 00:08:04.399
because it's very similar to large, you know,

00:08:04.400 --> 00:08:06.559
doing a chat thing where you, you know,

00:08:06.560 --> 00:08:09.760
the chat is like, you give it a text request,

00:08:09.761 --> 00:08:12.759
like write me a sonnet about, you know,

00:08:12.760 --> 00:08:14.879
the battle between Emacs and vi.

00:08:14.880 --> 00:08:15.839
And it could, it could do it.

00:08:15.840 --> 00:08:17.159
It could do a very good job of that.

00:08:17.160 --> 00:08:22.519
But you could also say, you know,

00:08:22.520 --> 00:08:27.599
draw me a picture of Emacs and vi as boxers,

00:08:27.600 --> 00:08:30.359
as a character-character boxing in a ring,

00:08:30.360 --> 00:08:32.239
like a, you know, political cartoon style.

00:08:32.240 --> 00:08:34.999
And it can do that as well.

00:08:35.000 --> 00:08:37.679
And so you could basically think of this

00:08:37.680 --> 00:08:39.439
as just sort of... it's kind of the

00:08:39.440 --> 00:08:42.399
same thing with what you're doing

00:08:42.400 --> 00:08:43.359
with large language models,

00:08:43.360 --> 00:08:44.799
but instead of outputting a text,

00:08:44.800 --> 00:08:48.479
you're outputting a picture.

NOTE Fine-tuning

00:08:48.480 --> 00:08:51.079
There's also, I want to mention the concept of fine-tuning.

00:08:51.080 --> 00:08:55.199
Fine-tuning is a way to take your--

00:08:55.200 --> 00:08:59.759
take a corpus of inputs and outputs and just from

00:08:59.760 --> 00:09:01.599
a large language model, you're like, okay,

00:09:01.600 --> 00:09:03.599
given this base large language model,

00:09:03.600 --> 00:09:06.679
I want to make sure that when I give you input,

00:09:06.680 --> 00:09:08.479
you give me something like output.

00:09:08.480 --> 00:09:10.119
And this is what I'm just going to

00:09:10.120 --> 00:09:11.799
train you further on these,

00:09:11.800 --> 00:09:14.879
these mappings between input and output.

00:09:14.880 --> 00:09:16.399
And for example, you could do this. Like,

00:09:16.400 --> 00:09:18.039
let's say you wanted to fix that regex demo

00:09:18.040 --> 00:09:20.999
I had to make it good.

00:09:21.000 --> 00:09:23.479
I don't think it, I think it'd be

00:09:23.480 --> 00:09:25.039
relatively effective to train,

00:09:25.040 --> 00:09:27.039
to have regex descriptions

00:09:27.040 --> 00:09:30.119
and regex examples, Emacs regex examples

00:09:30.120 --> 00:09:31.239
as inputs and outputs.

00:09:31.240 --> 00:09:33.999
You could get, you know, maybe a hundred,

00:09:34.000 --> 00:09:35.359
a few hundreds of these things.

00:09:35.360 --> 00:09:38.639
You could train it.

00:09:38.640 --> 00:09:40.759
I think that is a reasonable way to,

00:09:40.760 --> 00:09:43.879
let's just say, I don't know how well it would work,

00:09:43.880 --> 00:09:46.839
but these things definitely work some of the time

00:09:46.840 --> 00:09:47.999
and produce pretty good results.

00:09:48.000 --> 00:09:53.039
And you could do this on your own machine.

00:09:53.040 --> 00:09:58.999
Corporations like OpenAI offer APIs with, you know,

00:09:59.000 --> 00:10:01.519
to build your fine tunes on top of OpenAI.

00:10:01.520 --> 00:10:04.159
And I think, I'm not a hundred percent sure,

00:10:04.160 --> 00:10:05.719
but I think then you can share your model

00:10:05.720 --> 00:10:06.519
with other people.

00:10:06.520 --> 00:10:08.519
But if not, then you just, you know,

00:10:08.520 --> 00:10:10.839
you could use your model for your own specialized purposes.

00:10:10.840 --> 00:10:14.039
But in the world of models that you could run,

00:10:14.040 --> 00:10:16.874
for example, based on Llama, which is like...

00:10:16.875 --> 00:10:22.240
Llama is this model you can run on your own machine from Meta.

00:10:23.580 --> 00:10:26.880
There's many fine-tuned models that you could download

00:10:26.881 --> 00:10:28.960
and you could run on your own.

00:10:28.961 --> 00:10:30.839
They can do very different things too.

00:10:30.840 --> 00:10:33.399
Some output Python programs, for example,

00:10:33.400 --> 00:10:34.279
that you could just run.

00:10:34.280 --> 00:10:37.959
So you just say...

00:10:37.960 --> 00:10:40.639
Tell me how old... Let's just say

00:10:40.640 --> 00:10:41.999
you have a random task, like

00:10:42.000 --> 00:10:48.119
tell me how old these five cities are in minutes,

00:10:48.120 --> 00:10:49.799
based on historical evidence.

00:10:49.800 --> 00:10:53.639
It's kind of a weird query, but it probably can figure,

00:10:53.640 --> 00:10:55.119
it could probably run that for you.

00:10:55.120 --> 00:10:57.239
It'll encode its knowledge into whatever

00:10:57.240 --> 00:10:59.599
the Python program, then use the Python program

00:10:59.600 --> 00:11:01.039
to do the correct calculations.

00:11:01.040 --> 00:11:05.679
So pretty, pretty useful stuff.

NOTE Open Source

00:11:08.160 --> 00:11:10.399
So I also want to mention open source

00:11:10.400 --> 00:11:12.679
and basically free software here.

00:11:12.680 --> 00:11:17.599
These LLMs are mostly not free software.

00:11:17.600 --> 00:11:19.159
They're sometimes open source,

00:11:19.160 --> 00:11:21.959
but they're generally not free

00:11:21.960 --> 00:11:23.799
without restrictions to use.

00:11:23.800 --> 00:11:27.279
Most of these things, even Llama,

00:11:27.280 --> 00:11:28.679
which you can use on your own machine,

00:11:28.680 --> 00:11:31.439
have restrictions that you cannot use it

00:11:31.440 --> 00:11:32.519
to train your own model.

00:11:32.520 --> 00:11:35.119
This is something that, you know,

00:11:35.120 --> 00:11:37.519
it costs millions and millions of dollars

00:11:37.520 --> 00:11:40.759
to train and produce these models.

00:11:40.760 --> 00:11:42.319
And that's just computation costs.

00:11:42.320 --> 00:11:45.519
They do not want you

00:11:45.520 --> 00:11:47.839
stealing all that work by training your own models

00:11:47.840 --> 00:11:48.799
based on their output.

00:11:48.800 --> 00:11:55.359
But there are research LLMs that do, I believe,

00:11:55.360 --> 00:11:57.999
conform to free software principles.

00:11:58.000 --> 00:11:59.519
They're just not as good yet.

00:11:59.520 --> 00:12:02.519
And I think that might change in the future.

NOTE The Future

00:12:02.840 --> 00:12:04.119
So speaking of the future,

00:12:04.120 --> 00:12:07.519
one of the things I'd like to point out

00:12:07.520 --> 00:12:09.639
is that like the demos I showed you are based on,

00:12:09.640 --> 00:12:13.519
I'm using OpenAI 3.5 model.

00:12:13.520 --> 00:12:16.439
That's more than, well, no,

00:12:16.440 --> 00:12:18.199
it's like a year old basically at this point.

00:12:18.200 --> 00:12:21.079
And things are moving fast.

00:12:21.080 --> 00:12:22.039
They came out with 4.0.

00:12:22.040 --> 00:12:23.319
4.0 is significantly better.

00:12:23.320 --> 00:12:24.319
I don't have access to it.

00:12:24.320 --> 00:12:30.839
Even though I'm using the API and I'm paying money for it,

00:12:30.840 --> 00:12:33.639
you only can get access to 4.0

00:12:33.640 --> 00:12:34.439
if you can spend a dollar.

00:12:34.440 --> 00:12:36.319
And I've never been able to spend,

00:12:36.320 --> 00:12:38.199
use so much API use that I've spent a dollar.

00:12:38.200 --> 00:12:44.479
So I have, I don't have 4.0, but I've tried it

00:12:44.480 --> 00:12:46.639
because I do pay for this

00:12:46.640 --> 00:12:48.340
so I could get access to 4.0

00:12:48.341 --> 00:12:49.599
and it is substantially better.

00:12:49.600 --> 00:12:50.519
By all reports, it's,

00:12:50.520 --> 00:12:53.839
the difference is extremely significant.

00:12:53.840 --> 00:12:55.159
I would not be surprised

00:12:55.160 --> 00:12:59.759
if some of the limitations and drawbacks I described

00:12:59.760 --> 00:13:02.039
mostly went away with 4.0.

00:13:02.040 --> 00:13:06.679
We're probably at a stage

00:13:06.680 --> 00:13:09.239
where regexes will work maybe 5% of the time

00:13:09.240 --> 00:13:10.119
if you try them.

00:13:10.120 --> 00:13:13.639
But with 4.0, it could work like 80% of the time.

00:13:13.640 --> 00:13:14.559
Now, is that good enough?

00:13:14.560 --> 00:13:17.279
Probably not, but it's a,

00:13:17.280 --> 00:13:20.319
I wouldn't be surprised if you got results like that.

00:13:20.320 --> 00:13:22.919
And in a year's time, in two years time,

00:13:22.920 --> 00:13:26.679
no one knows how much this is going to play out

00:13:26.680 --> 00:13:27.519
before progress stalls,

00:13:27.520 --> 00:13:32.319
but there are a lot of interesting research.

00:13:32.320 --> 00:13:34.279
I don't think, research wise,

00:13:34.280 --> 00:13:35.759
I don't think things have slowed down.

00:13:35.760 --> 00:13:38.719
You're still seeing a lot of advances.

00:13:38.720 --> 00:13:40.999
You're still seeing a lot of models coming out

00:13:41.000 --> 00:13:41.839
and that will come out.

00:13:41.840 --> 00:13:46.279
That will be each one, one upping the other one

00:13:46.280 --> 00:13:49.959
in terms of quality.

00:13:49.960 --> 00:13:52.759
It'll be really interesting to see how this all plays out.

00:13:52.760 --> 00:13:55.919
I think that message here is that

00:13:55.920 --> 00:13:57.999
we're at the beginning here.

00:13:58.000 --> 00:14:01.239
This is why I think this talk is important.

00:14:01.240 --> 00:14:02.279
I think this is why we should be

00:14:02.280 --> 00:14:04.159
paying attention to this stuff.

NOTE LLMs in Emacs - existing packages

00:14:08.200 --> 00:14:11.039
Let's talk about the existing packages.

00:14:11.040 --> 00:14:13.199
Because there's a lot out there, people have,

00:14:13.200 --> 00:14:17.039
I think people have been integrating with

00:14:17.040 --> 00:14:21.239
these LLMs that often have a relatively easy to use API.

00:14:21.240 --> 00:14:24.039
So it's kind of natural that people

00:14:24.040 --> 00:14:25.679
have already put out a lot of packages.

00:14:25.680 --> 00:14:28.319
Coming off this problem from a lot of different angles,

00:14:28.320 --> 00:14:30.639
I don't have time to go through

00:14:30.640 --> 00:14:31.959
all of these packages.

00:14:31.960 --> 00:14:33.559
These are great packages though.

00:14:33.560 --> 00:14:35.279
If you're not familiar with them,

00:14:35.280 --> 00:14:37.679
please check them out.

00:14:37.680 --> 00:14:40.999
And they all are doing slightly different things.

00:14:41.000 --> 00:14:43.959
Some of these are relatively straightforward.

00:14:43.960 --> 00:14:47.919
Interactions, just a way to

00:14:47.920 --> 00:14:52.679
almost in a comment sort of way to kind of

00:14:52.680 --> 00:14:54.199
have just an interaction,

00:14:54.200 --> 00:14:55.479
long running interaction with an LLM

00:14:55.480 --> 00:14:59.039
where you kind of build off previous responses,

00:14:59.040 --> 00:15:01.799
kind of like the OpenAI's UI.

00:15:01.800 --> 00:15:08.559
Two very more Emacsy things where you can sort of

00:15:08.560 --> 00:15:13.679
embed these LLM responses within a org-mode block

00:15:13.680 --> 00:15:15.239
using the org-mode's context.

00:15:15.240 --> 00:15:20.959
Or GitHub Copilot integration where you can use it

00:15:20.960 --> 00:15:23.319
for auto completion in a very powerful,

00:15:23.320 --> 00:15:27.319
you know, this stuff is very useful if it could figure out

00:15:27.320 --> 00:15:29.199
what you're trying to do based on the context.

00:15:29.200 --> 00:15:31.839
It's quite effective.

00:15:31.840 --> 00:15:36.359
But I want to kind of call out one thing

00:15:36.360 --> 00:15:38.239
that I'd like to see change.

00:15:38.240 --> 00:15:42.599
Which is that users right now,

00:15:42.600 --> 00:15:45.199
not all of these have a choice of,

00:15:45.200 --> 00:15:47.959
first of all, there's a lot of them.

00:15:47.960 --> 00:15:49.639
Each one of them is doing their own calls.

00:15:49.640 --> 00:15:53.999
And each one of them is, so each one of them

00:15:54.000 --> 00:15:55.239
has their own interfaces.

00:15:55.240 --> 00:15:57.719
They're rewriting the interface to OpenAI or wherever.

00:15:57.720 --> 00:16:00.119
And they're not, they don't, most of these

00:16:00.120 --> 00:16:05.119
do not make it that configurable or at all configurable

00:16:05.120 --> 00:16:06.599
what LLM use.

00:16:06.600 --> 00:16:07.239
This is not good.

00:16:07.240 --> 00:16:09.679
It is important that we use,

00:16:09.680 --> 00:16:15.679
we give the user a way to change the LLM they use.

00:16:15.680 --> 00:16:21.079
And that is because you might not be comfortable

00:16:21.080 --> 00:16:24.439
sending your requests over to a private corporation

00:16:24.440 --> 00:16:27.799
where you don't get to see how they use their data.

00:16:27.800 --> 00:16:29.799
Your data, really.

00:16:29.800 --> 00:16:33.319
That's especially true with things like embeddings

00:16:33.320 --> 00:16:35.039
where you might be sending over your documents.

00:16:35.040 --> 00:16:37.519
You're just giving them your documents, basically.

00:16:37.520 --> 00:16:40.759
And, you know, that does happen.

00:16:40.760 --> 00:16:43.599
I don't think really that there's a reason

00:16:43.600 --> 00:16:44.639
to be uncomfortable with this,

00:16:44.640 --> 00:16:51.439
but that, you know, people are uncomfortable and that's okay.

00:16:51.440 --> 00:16:53.239
People might want to use a local machine,

00:16:53.240 --> 00:16:58.359
a local LLM for maximum privacy.

00:16:58.360 --> 00:17:00.639
That's something we should allow.

00:17:00.640 --> 00:17:04.519
People might want to especially use free software.

00:17:04.520 --> 00:17:05.839
That's something we should definitely allow.

00:17:05.840 --> 00:17:07.279
This is Emacs.

00:17:07.280 --> 00:17:08.239
We need to encourage that.

00:17:08.240 --> 00:17:12.159
But right now, as most of these things are written,

00:17:12.160 --> 00:17:13.959
you can't do it.

00:17:13.960 --> 00:17:17.839
And they're spending precious time

00:17:17.840 --> 00:17:18.879
just doing things themselves.

00:17:18.880 --> 00:17:20.839
This is why I wrote LLM, which is...

00:17:20.840 --> 00:17:23.039
it will just make that connection to the LLM for you

00:17:23.040 --> 00:17:26.719
and it will connect to, you know, it has plugins.

00:17:26.720 --> 00:17:30.279
So if you can, the user can configure what plugin

00:17:30.280 --> 00:17:31.359
it actually goes to.

00:17:31.360 --> 00:17:32.399
Does it go to OpenAI?

00:17:32.400 --> 00:17:35.239
Does it go to Google Cloud Vertex?

00:17:35.240 --> 00:17:36.999
Does it go to Llama on your machine?

00:17:37.000 --> 00:17:38.399
We're using Ollama,

00:17:38.400 --> 00:17:40.999
which is just a way to run Llama locally.

00:17:41.000 --> 00:17:47.959
And more things in the future, I hope.

00:17:47.960 --> 00:17:52.079
So this is, I'm hoping that we use this.

00:17:52.080 --> 00:17:54.839
It's designed to be sort of maximally usable.

00:17:54.840 --> 00:17:56.279
You don't need to install anything.

00:17:56.280 --> 00:17:58.359
It's on GNU ELPA.

00:17:58.360 --> 00:17:59.879
So even if you write something

00:17:59.880 --> 00:18:01.079
that you want to contribute to GNU ELPA,

00:18:01.080 --> 00:18:02.879
you can use it because it's on GNU ELPA.

00:18:02.880 --> 00:18:06.439
It's part of the Emacs package, Emacs core packages.

00:18:06.440 --> 00:18:09.879
So, but it has no functionality.

00:18:09.880 --> 00:18:11.719
It's really just there as a library

00:18:11.720 --> 00:18:14.439
to use by other things offering functionality. Okay.

NOTE Abstracting LLM challenges

00:18:15.960 --> 00:18:19.839
And it's a little bit difficult to abstract.

00:18:19.840 --> 00:18:21.159
I want to point this out

00:18:21.160 --> 00:18:23.599
because I think it's an important point

00:18:23.600 --> 00:18:29.519
is that the, it's, some of these LLMs, for example,

00:18:29.520 --> 00:18:30.439
have image generation.

00:18:30.440 --> 00:18:31.279
Some do not.

00:18:31.280 --> 00:18:35.319
Some of them have very large context windows, even for chat.

00:18:35.320 --> 00:18:36.999
You say, okay, all these things can do chat.

00:18:37.000 --> 00:18:37.319
Okay.

00:18:37.320 --> 00:18:38.079
Yeah, kind of.

00:18:38.080 --> 00:18:39.999
Some of these things you could pass a book to,

00:18:40.000 --> 00:18:41.239
like Anthropic's API.

00:18:41.240 --> 00:18:43.039
Most, you cannot.

00:18:43.040 --> 00:18:45.559
So there really are big differences

00:18:45.560 --> 00:18:46.399
in how these things work.

00:18:46.400 --> 00:18:51.539
I hope those differences diminish in the future.

00:18:51.540 --> 00:18:53.800
But it's just one of the challenges

00:18:53.801 --> 00:18:57.520
that I hope we can work through in the LLM library.

00:18:57.521 --> 00:19:02.160
So it's compatible, but there's definitely

00:19:02.161 --> 00:19:04.079
limits to that compatibility.

NOTE Emacs is the ideal interface for LLMs

00:19:04.080 --> 00:19:06.160
I want to point out just to finish off,

00:19:06.161 --> 00:19:12.879
Emacs is the, Emacs has real power here

00:19:12.880 --> 00:19:15.679
that nothing else I think in the industry is offering.

00:19:15.680 --> 00:19:19.279
First of all, people that use Emacs

00:19:19.280 --> 00:19:20.439
tend to do a lot of things in Emacs.

00:19:20.440 --> 00:19:22.159
We have our to-dos in Emacs with the org mode.

00:19:22.160 --> 00:19:22.999
We have mail.

00:19:23.000 --> 00:19:25.719
We, you know, we might read email and we might,

00:19:25.720 --> 00:19:27.679
and respond to email in Emacs.

00:19:27.680 --> 00:19:29.199
We might have notes in Emacs.

00:19:29.200 --> 00:19:31.359
This is very powerful.

00:19:31.360 --> 00:19:34.159
Using... there's not other stuff like that.

00:19:34.160 --> 00:19:35.759
And you could feed this stuff to an LLM.

00:19:35.760 --> 00:19:37.039
You could do interesting things

00:19:37.040 --> 00:19:38.559
using a combination of all this data.

00:19:38.560 --> 00:19:40.399
No one else could do this.

00:19:40.400 --> 00:19:41.759
We need to start thinking about it.

00:19:41.760 --> 00:19:45.039
Secondly, Emacs can execute commands.

00:19:45.040 --> 00:19:46.239
This might be a bad idea.

00:19:46.240 --> 00:19:48.399
This might be how the robots take over,

00:19:48.400 --> 00:19:51.799
but you could have the LLMs respond with Emacs

00:19:51.800 --> 00:19:54.199
commands and run those Emacs commands

00:19:54.200 --> 00:19:57.079
and tell the LLM the response and have it do things

00:19:57.080 --> 00:19:58.679
as your agent in the editor.

00:19:58.680 --> 00:20:01.599
I think we need to explore ideas like this.

NOTE Outro

00:20:01.960 --> 00:20:04.279
And I think we need to share these ideas

00:20:04.280 --> 00:20:07.039
and we need to make sure that we're pushing the

00:20:07.040 --> 00:20:10.519
envelope for Emacs and actually, you know, doing things,

00:20:10.520 --> 00:20:12.959
sharing ideas, sharing progress,

00:20:12.960 --> 00:20:15.199
and kind of seeing how far we can push this stuff.

00:20:15.200 --> 00:20:20.639
Let's really help Emacs out, be sort of,

00:20:20.640 --> 00:20:24.519
take advantage of this super powerful technique.

00:20:24.520 --> 00:20:26.160
Thank you for listening.