summaryrefslogtreecommitdiffstats
path: root/2023/captions
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main--chapters.vtt41
-rw-r--r--2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt1377
2 files changed, 1418 insertions, 0 deletions
diff --git a/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main--chapters.vtt b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main--chapters.vtt
new file mode 100644
index 00000000..858d0fdb
--- /dev/null
+++ b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main--chapters.vtt
@@ -0,0 +1,41 @@
+WEBVTT
+
+
+00:00:00.000 --> 00:00:25.079
+Intro to the Talk
+
+00:00:25.080 --> 00:01:56.359
+What are LLMs?
+
+00:01:56.360 --> 00:03:32.239
+Power of LLMs (Magit Demo)
+
+00:03:32.240 --> 00:05:20.119
+Drawbacks of LLMs (regex demo)
+
+00:05:20.120 --> 00:07:32.799
+Embeddings
+
+00:07:32.800 --> 00:08:48.479
+Image Generation
+
+00:08:48.480 --> 00:11:05.679
+Fine-tuning
+
+00:11:08.160 --> 00:12:02.519
+Open Source
+
+00:12:02.840 --> 00:14:04.159
+The Future
+
+00:14:08.200 --> 00:18:14.439
+LLMs in Emacs - existing packages
+
+00:18:15.960 --> 00:19:04.079
+Abstracting LLM challenges
+
+00:19:04.080 --> 00:20:01.599
+Emacs is the ideal interface for LLMs
+
+00:20:01.960 --> 00:20:26.160
+Outro
diff --git a/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt
new file mode 100644
index 00000000..3ac4b34c
--- /dev/null
+++ b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt
@@ -0,0 +1,1377 @@
+WEBVTT captioned by bala, checked by sachac
+
+NOTE Intro to the Talk
+
+00:00:00.000 --> 00:00:04.159
+Hello, I'm Andrew Hyatt and I'm going to talk to you
+
+00:00:04.160 --> 00:00:06.439
+about large language models and how
+
+00:00:06.440 --> 00:00:11.079
+they relate to Emacs.
+
+00:00:11.080 --> 00:00:14.919
+And I'm going to talk to you about the technology
+
+00:00:14.920 --> 00:00:18.279
+and how we're going to use it in Emacs.
+
+00:00:18.280 --> 00:00:21.159
+There'll be demos and there'll be talks about,
+
+00:00:21.160 --> 00:00:22.879
+I'll finish up by kind of talking about where
+
+00:00:22.880 --> 00:00:25.079
+I think this should go in the future.
+
+NOTE What are LLMs?
+
+00:00:25.080 --> 00:00:28.239
+So to start off with, let's just talk like,
+
+00:00:28.240 --> 00:00:29.759
+I just want to make sure everyone's on the same page.
+
+00:00:29.760 --> 00:00:30.919
+What are large language models?
+
+00:00:30.920 --> 00:00:34.639
+Not everyone may be caught up on this.
+
+00:00:34.640 --> 00:00:38.999
+Large language models are a way... Basically,
+
+00:00:39.000 --> 00:00:42.999
+the current versions of large language models
+
+00:00:43.000 --> 00:00:44.479
+are all based on the similar architecture
+
+00:00:44.480 --> 00:00:45.279
+called the transformer.
+
+00:00:45.280 --> 00:00:48.719
+It's just an efficient way to train and produce output.
+
+00:00:48.720 --> 00:00:51.919
+So these things are basically models
+
+00:00:51.920 --> 00:00:58.079
+that predict the next word or something like that.
+
+00:00:58.080 --> 00:01:02.119
+And they're trained on an enormous corpus of information
+
+00:01:02.120 --> 00:01:04.319
+and they get extremely good
+
+00:01:04.320 --> 00:01:06.079
+at predicting the next word.
+
+00:01:06.080 --> 00:01:09.679
+And from that basic ability, you can train
+
+00:01:09.680 --> 00:01:12.439
+through further tuning from human input,
+
+00:01:12.440 --> 00:01:13.959
+human ratings and things like that.
+
+00:01:13.960 --> 00:01:17.479
+You can train different models based on that
+
+00:01:17.480 --> 00:01:18.759
+that will do question answering.
+
+00:01:18.760 --> 00:01:22.519
+And this is how basically ChatGPT works.
+
+00:01:22.520 --> 00:01:25.599
+There's a base LLM, like GPT.
+
+00:01:25.600 --> 00:01:27.799
+And then you have a chat version of that,
+
+00:01:27.800 --> 00:01:29.959
+which is just trained to just... You give
+
+00:01:29.960 --> 00:01:32.199
+it a prompt, like what do you want it to do?
+
+00:01:32.200 --> 00:01:37.279
+And it gives you an output that does what you told it to do,
+
+00:01:37.280 --> 00:01:39.919
+or at least attempts to do it.
+
+00:01:39.920 --> 00:01:42.079
+Those are the power of large language models is
+
+00:01:42.080 --> 00:01:45.639
+they're extremely, extremely impressive.
+
+00:01:45.640 --> 00:01:47.199
+Certainly this is, in AI,
+
+00:01:47.200 --> 00:01:49.079
+this has been the biggest thing to happen
+
+00:01:49.080 --> 00:01:51.559
+probably in my lifetime,
+
+00:01:51.560 --> 00:01:56.359
+or at least my lifetime as my working lifetime.
+
+NOTE Power of LLMs (Magit Demo)
+
+00:01:56.360 --> 00:02:02.559
+So let me give you a demonstration of
+
+00:02:02.560 --> 00:02:06.679
+what kinds of stuff it could do in Emacs.
+
+00:02:06.680 --> 00:02:09.039
+So here I have a Emacs file.
+
+00:02:09.040 --> 00:02:12.479
+So this is my Emacs init file.
+
+00:02:12.480 --> 00:02:13.599
+I have a change.
+
+00:02:13.600 --> 00:02:16.879
+Let's commit that change.
+
+00:02:16.880 --> 00:02:19.439
+And, you know, I don't like writing commit messages,
+
+00:02:19.440 --> 00:02:23.039
+so I can generate it.
+
+00:02:23.040 --> 00:02:27.479
+And it did an actually just looking.
+
+00:02:27.480 --> 00:02:29.759
+So all it does is it's looking, it's just reading the diff.
+
+00:02:29.760 --> 00:02:32.479
+I'm just feeding it the diff with some instructions.
+
+00:02:32.480 --> 00:02:37.759
+And it is this a incredible commit message?
+
+00:02:37.760 --> 00:02:39.399
+It's not bad, actually.
+
+00:02:39.400 --> 00:02:42.319
+You can see that it actually has really extracted
+
+00:02:42.320 --> 00:02:46.439
+the meaning of what I'm doing and has written
+
+00:02:46.440 --> 00:02:48.879
+a reasonably good commit message.
+
+00:02:48.880 --> 00:02:53.159
+Now I have to edit it because this is not quite correct.
+
+00:02:53.160 --> 00:02:55.159
+But it's kind of impressive how good it is.
+
+00:02:55.160 --> 00:03:00.039
+And my editing, it's kind of easier for me to edit this
+
+00:03:00.040 --> 00:03:01.879
+than just to write a new one.
+
+00:03:01.880 --> 00:03:04.479
+And quite often it's good enough to just submit as is.
+
+00:03:04.480 --> 00:03:08.119
+So this is kind of, you know, you could say
+
+00:03:08.120 --> 00:03:09.359
+this is just commit messages.
+
+00:03:09.360 --> 00:03:10.719
+You could respond to emails.
+
+00:03:10.720 --> 00:03:15.319
+You could, you know, using your own custom instructions
+
+00:03:15.320 --> 00:03:17.839
+about what you want your email to say.
+
+00:03:17.840 --> 00:03:19.039
+It'll write the email for you.
+
+00:03:19.040 --> 00:03:19.839
+It could do like this
+
+00:03:19.840 --> 00:03:22.519
+Emacs is a way to interact with buffers.
+
+00:03:22.520 --> 00:03:24.199
+This could basically just output text.
+
+00:03:24.200 --> 00:03:27.759
+So it's super useful for
+
+00:03:27.760 --> 00:03:30.319
+understanding something and outputting text based on that,
+
+00:03:30.320 --> 00:03:32.239
+which is just useful for Emacs.
+
+NOTE Drawbacks of LLMs (regex demo)
+
+00:03:32.240 --> 00:03:39.919
+So the drawback is, yeah, it's good,
+
+00:03:39.920 --> 00:03:43.359
+but it's not that reliable.
+
+00:03:43.360 --> 00:03:45.679
+And you'd think it's very easy to get caught up in like,
+
+00:03:45.680 --> 00:03:47.639
+oh my gosh, like this is so powerful.
+
+00:03:47.640 --> 00:03:50.599
+I bet it could work this, whatever idea could work.
+
+00:03:50.600 --> 00:03:52.919
+And these ideas, like they almost can.
+
+00:03:52.920 --> 00:03:55.639
+For example, I was thinking, you know what I could do?
+
+00:03:55.640 --> 00:03:57.239
+I don't like writing regexes.
+
+00:03:57.240 --> 00:04:01.199
+Why can't I have a regex replace that's powered by LLMs?
+
+00:04:01.200 --> 00:04:03.439
+And that way I could give just an instruction
+
+00:04:03.440 --> 00:04:07.399
+to regex replace.
+
+00:04:07.400 --> 00:04:12.079
+And so for example, I could do Emacs LLM regex replace.
+
+00:04:12.080 --> 00:04:12.879
+This is not checked in anywhere.
+
+00:04:12.880 --> 00:04:17.199
+These are just my own kind of private functions.
+
+00:04:17.200 --> 00:04:19.239
+My description lowercase all the org headings.
+
+00:04:19.240 --> 00:04:20.439
+Let's see if it works.
+
+00:04:20.440 --> 00:04:21.039
+It might work.
+
+00:04:21.040 --> 00:04:22.959
+No, it doesn't work.
+
+00:04:22.960 --> 00:04:26.159
+So if I, I'm not going to bother to show you
+
+00:04:26.160 --> 00:04:28.159
+what it actually came up with, but it's something,
+
+00:04:28.160 --> 00:04:29.879
+if you looked at it, it'd be like, wow,
+
+00:04:29.880 --> 00:04:31.639
+this is very close to being...
+
+00:04:31.640 --> 00:04:34.239
+It looks like it should work, but it doesn't.
+
+00:04:34.240 --> 00:04:35.839
+Okay.
+
+00:04:35.840 --> 00:04:38.719
+It's not quite good enough to get it right.
+
+00:04:38.720 --> 00:04:41.599
+And it's possible that perhaps by giving it
+
+00:04:41.600 --> 00:04:43.639
+a few examples of, or explaining more
+
+00:04:43.640 --> 00:04:46.439
+what makes Emacs regexes different.
+
+00:04:46.440 --> 00:04:47.959
+It could do a better job
+
+00:04:47.960 --> 00:04:49.279
+and maybe could solve these problems,
+
+00:04:49.280 --> 00:04:50.679
+but it's always a little bit random.
+
+00:04:50.680 --> 00:04:52.359
+You're never quite sure what you're going to get.
+
+00:04:52.360 --> 00:04:54.839
+So this is the drawback.
+
+00:04:54.840 --> 00:04:58.479
+Like there's a lot of things that look like you could do it,
+
+00:04:58.480 --> 00:05:00.999
+but when it actually comes down to trying it,
+
+00:05:01.000 --> 00:05:03.399
+it's surprisingly hard.
+
+00:05:03.400 --> 00:05:06.319
+And, you know, and whatever you're doing,
+
+00:05:06.320 --> 00:05:08.999
+it's surprisingly hard to get something
+
+00:05:09.000 --> 00:05:13.879
+that is repeatably, that's, that is always good.
+
+00:05:13.880 --> 00:05:20.119
+So yeah, that's currently the problem.
+
+NOTE Embeddings
+
+00:05:20.120 --> 00:05:23.399
+So I want to talk about embeddings.
+
+00:05:23.400 --> 00:05:26.919
+They're another thing that LLMs offer
+
+00:05:26.920 --> 00:05:28.599
+and that are extremely useful.
+
+00:05:28.600 --> 00:05:33.119
+They are, what they do is they encode from
+
+00:05:33.120 --> 00:05:38.959
+a input text that could be a word, a sentence,
+
+00:05:38.960 --> 00:05:42.159
+a small document.
+
+00:05:42.160 --> 00:05:45.399
+It encodes a vector about what the meaning,
+
+00:05:45.400 --> 00:05:46.919
+the semantic meaning of that is.
+
+00:05:46.920 --> 00:05:51.079
+That means you could, something that is,
+
+00:05:51.080 --> 00:05:52.279
+uses completely different words,
+
+00:05:52.280 --> 00:05:54.159
+but is basically talking about the same thing,
+
+00:05:54.160 --> 00:05:57.839
+perhaps in a different language, should be pretty close
+
+00:05:57.840 --> 00:06:01.999
+as a vector to the other vector.
+
+00:06:02.000 --> 00:06:05.399
+You know, as long as they're similarly semantic things,
+
+00:06:05.400 --> 00:06:12.239
+like the words
+
+00:06:12.240 --> 00:06:18.959
+highway and Camino are two different words.
+
+00:06:18.960 --> 00:06:19.639
+They mean the same thing.
+
+00:06:19.640 --> 00:06:21.319
+They should have very similar embeddings.
+
+00:06:21.320 --> 00:06:25.119
+So it is a way to kind of encode this
+
+00:06:25.120 --> 00:06:26.199
+and then you could use this for search.
+
+00:06:26.200 --> 00:06:28.919
+For example, I haven't tried to do this yet,
+
+00:06:28.920 --> 00:06:31.479
+but you could probably just make an embedding
+
+00:06:31.480 --> 00:06:33.919
+for every paragraph in the Emacs manual
+
+00:06:33.920 --> 00:06:36.239
+and the Elisp manual.
+
+00:06:36.240 --> 00:06:39.439
+And then, and then there's a very standard technique.
+
+00:06:39.440 --> 00:06:43.439
+You just... You find that you have a query,
+
+00:06:43.440 --> 00:06:45.799
+oh, how do I do whatever, whatever in Emacs again?
+
+00:06:45.800 --> 00:06:49.479
+And you could, you just find that 20 things
+
+00:06:49.480 --> 00:06:50.319
+that are closest to whatever you're
+
+00:06:50.320 --> 00:06:51.839
+trying to... the embedding of your query.
+
+00:06:51.840 --> 00:06:55.279
+You send those things to the LLM, as you know,
+
+00:06:55.280 --> 00:06:57.799
+with the original query,
+
+00:06:57.800 --> 00:06:59.919
+and you're basically telling the--asking the LLM,
+
+00:06:59.920 --> 00:07:01.279
+look, the user is trying to do this.
+
+00:07:01.280 --> 00:07:03.039
+Here's what I found in the Emacs manual.
+
+00:07:03.040 --> 00:07:04.639
+That's on the Elisp manual.
+
+00:07:04.640 --> 00:07:07.439
+That's close to what they're trying to do.
+
+00:07:07.440 --> 00:07:12.159
+So can you kind of just tell the user what to do?
+
+00:07:12.160 --> 00:07:14.479
+And from this, and you could say,
+
+00:07:14.480 --> 00:07:17.639
+just use things from this, you know, that I give you.
+
+00:07:17.640 --> 00:07:20.679
+Don't just make up your own idea.
+
+00:07:20.680 --> 00:07:21.839
+You know, don't use your own ideas,
+
+00:07:21.840 --> 00:07:23.799
+because sometimes it likes to do that
+
+00:07:23.800 --> 00:07:24.359
+and those things are wrong.
+
+00:07:24.360 --> 00:07:26.719
+So you could try to, you know, do this and you get,
+
+00:07:26.720 --> 00:07:28.719
+you could get quite good results using this.
+
+00:07:28.720 --> 00:07:29.999
+So no one has done this yet,
+
+00:07:30.000 --> 00:07:32.799
+but that should not be hard to do.
+
+NOTE Image Generation
+
+00:07:32.800 --> 00:07:34.879
+Image generation is something that's, you know,
+
+00:07:34.880 --> 00:07:38.479
+it's not quite an LLM in the sense of...
+
+00:07:38.480 --> 00:07:43.079
+These are... It's a different technology,
+
+00:07:43.080 --> 00:07:48.439
+but these things are kind of packaged together
+
+00:07:48.440 --> 00:07:49.039
+in a sense.
+
+00:07:49.040 --> 00:07:51.639
+And you'll see that when I talk about Emacs packages,
+
+00:07:51.640 --> 00:07:54.279
+a lot of them bundle image generation
+
+00:07:54.280 --> 00:07:55.439
+and large language models.
+
+00:07:55.440 --> 00:07:59.039
+You know, the APIs are often bundled together by providers.
+
+00:07:59.040 --> 00:08:02.679
+And the general idea is it's kind of similar
+
+00:08:02.680 --> 00:08:04.399
+because it's very similar to large, you know,
+
+00:08:04.400 --> 00:08:06.559
+doing a chat thing where you, you know,
+
+00:08:06.560 --> 00:08:09.760
+the chat is like, you give it a text request,
+
+00:08:09.761 --> 00:08:12.759
+like write me a sonnet about, you know,
+
+00:08:12.760 --> 00:08:14.879
+the battle between Emacs and vi.
+
+00:08:14.880 --> 00:08:15.839
+And it could, it could do it.
+
+00:08:15.840 --> 00:08:17.159
+It could do a very good job of that.
+
+00:08:17.160 --> 00:08:22.519
+But you could also say, you know,
+
+00:08:22.520 --> 00:08:27.599
+draw me a picture of Emacs and vi as boxers,
+
+00:08:27.600 --> 00:08:30.359
+as a character-character boxing in a ring,
+
+00:08:30.360 --> 00:08:32.239
+like a, you know, political cartoon style.
+
+00:08:32.240 --> 00:08:34.999
+And it can do that as well.
+
+00:08:35.000 --> 00:08:37.679
+And so you could basically think of this
+
+00:08:37.680 --> 00:08:39.439
+as just sort of... it's kind of the
+
+00:08:39.440 --> 00:08:42.399
+same thing with what you're doing
+
+00:08:42.400 --> 00:08:43.359
+with large language models,
+
+00:08:43.360 --> 00:08:44.799
+but instead of outputting a text,
+
+00:08:44.800 --> 00:08:48.479
+you're outputting a picture.
+
+NOTE Fine-tuning
+
+00:08:48.480 --> 00:08:51.079
+There's also, I want to mention the concept of fine-tuning.
+
+00:08:51.080 --> 00:08:55.199
+Fine-tuning is a way to take your--
+
+00:08:55.200 --> 00:08:59.759
+take a corpus of inputs and outputs and just from
+
+00:08:59.760 --> 00:09:01.599
+a large language model, you're like, okay,
+
+00:09:01.600 --> 00:09:03.599
+given this base large language model,
+
+00:09:03.600 --> 00:09:06.679
+I want to make sure that when I give you input,
+
+00:09:06.680 --> 00:09:08.479
+you give me something like output.
+
+00:09:08.480 --> 00:09:10.119
+And this is what I'm just going to
+
+00:09:10.120 --> 00:09:11.799
+train you further on these,
+
+00:09:11.800 --> 00:09:14.879
+these mappings between input and output.
+
+00:09:14.880 --> 00:09:16.399
+And for example, you could do this. Like,
+
+00:09:16.400 --> 00:09:18.039
+let's say you wanted to fix that regex demo
+
+00:09:18.040 --> 00:09:20.999
+I had to make it good.
+
+00:09:21.000 --> 00:09:23.479
+I don't think it, I think it'd be
+
+00:09:23.480 --> 00:09:25.039
+relatively effective to train,
+
+00:09:25.040 --> 00:09:27.039
+to have regex descriptions
+
+00:09:27.040 --> 00:09:30.119
+and regex examples, Emacs regex examples
+
+00:09:30.120 --> 00:09:31.239
+as inputs and outputs.
+
+00:09:31.240 --> 00:09:33.999
+You could get, you know, maybe a hundred,
+
+00:09:34.000 --> 00:09:35.359
+a few hundreds of these things.
+
+00:09:35.360 --> 00:09:38.639
+You could train it.
+
+00:09:38.640 --> 00:09:40.759
+I think that is a reasonable way to,
+
+00:09:40.760 --> 00:09:43.879
+let's just say, I don't know how well it would work,
+
+00:09:43.880 --> 00:09:46.839
+but these things definitely work some of the time
+
+00:09:46.840 --> 00:09:47.999
+and produce pretty good results.
+
+00:09:48.000 --> 00:09:53.039
+And you could do this on your own machine.
+
+00:09:53.040 --> 00:09:58.999
+Corporations like OpenAI offer APIs with, you know,
+
+00:09:59.000 --> 00:10:01.519
+to build your fine tunes on top of OpenAI.
+
+00:10:01.520 --> 00:10:04.159
+And I think, I'm not a hundred percent sure,
+
+00:10:04.160 --> 00:10:05.719
+but I think then you can share your model
+
+00:10:05.720 --> 00:10:06.519
+with other people.
+
+00:10:06.520 --> 00:10:08.519
+But if not, then you just, you know,
+
+00:10:08.520 --> 00:10:10.839
+you could use your model for your own specialized purposes.
+
+00:10:10.840 --> 00:10:14.039
+But in the world of models that you could run,
+
+00:10:14.040 --> 00:10:16.874
+for example, based on Llama, which is like...
+
+00:10:16.875 --> 00:10:22.240
+Llama is this model you can run on your own machine from Meta.
+
+00:10:23.580 --> 00:10:26.880
+There's many fine-tuned models that you could download
+
+00:10:26.881 --> 00:10:28.960
+and you could run on your own.
+
+00:10:28.961 --> 00:10:30.839
+They can do very different things too.
+
+00:10:30.840 --> 00:10:33.399
+Some output Python programs, for example,
+
+00:10:33.400 --> 00:10:34.279
+that you could just run.
+
+00:10:34.280 --> 00:10:37.959
+So you just say...
+
+00:10:37.960 --> 00:10:40.639
+Tell me how old... Let's just say
+
+00:10:40.640 --> 00:10:41.999
+you have a random task, like
+
+00:10:42.000 --> 00:10:48.119
+tell me how old these five cities are in minutes,
+
+00:10:48.120 --> 00:10:49.799
+based on historical evidence.
+
+00:10:49.800 --> 00:10:53.639
+It's kind of a weird query, but it probably can figure,
+
+00:10:53.640 --> 00:10:55.119
+it could probably run that for you.
+
+00:10:55.120 --> 00:10:57.239
+It'll encode its knowledge into whatever
+
+00:10:57.240 --> 00:10:59.599
+the Python program, then use the Python program
+
+00:10:59.600 --> 00:11:01.039
+to do the correct calculations.
+
+00:11:01.040 --> 00:11:05.679
+So pretty, pretty useful stuff.
+
+NOTE Open Source
+
+00:11:08.160 --> 00:11:10.399
+So I also want to mention open source
+
+00:11:10.400 --> 00:11:12.679
+and basically free software here.
+
+00:11:12.680 --> 00:11:17.599
+These LLMs are mostly not free software.
+
+00:11:17.600 --> 00:11:19.159
+They're sometimes open source,
+
+00:11:19.160 --> 00:11:21.959
+but they're generally not free
+
+00:11:21.960 --> 00:11:23.799
+without restrictions to use.
+
+00:11:23.800 --> 00:11:27.279
+Most of these things, even Llama,
+
+00:11:27.280 --> 00:11:28.679
+which you can use on your own machine,
+
+00:11:28.680 --> 00:11:31.439
+have restrictions that you cannot use it
+
+00:11:31.440 --> 00:11:32.519
+to train your own model.
+
+00:11:32.520 --> 00:11:35.119
+This is something that, you know,
+
+00:11:35.120 --> 00:11:37.519
+it costs millions and millions of dollars
+
+00:11:37.520 --> 00:11:40.759
+to train and produce these models.
+
+00:11:40.760 --> 00:11:42.319
+And that's just computation costs.
+
+00:11:42.320 --> 00:11:45.519
+They do not want you
+
+00:11:45.520 --> 00:11:47.839
+stealing all that work by training your own models
+
+00:11:47.840 --> 00:11:48.799
+based on their output.
+
+00:11:48.800 --> 00:11:55.359
+But there are research LLMs that do, I believe,
+
+00:11:55.360 --> 00:11:57.999
+conform to free software principles.
+
+00:11:58.000 --> 00:11:59.519
+They're just not as good yet.
+
+00:11:59.520 --> 00:12:02.519
+And I think that might change in the future.
+
+NOTE The Future
+
+00:12:02.840 --> 00:12:04.119
+So speaking of the future,
+
+00:12:04.120 --> 00:12:07.519
+one of the things I'd like to point out
+
+00:12:07.520 --> 00:12:09.639
+is that like the demos I showed you are based on,
+
+00:12:09.640 --> 00:12:13.519
+I'm using OpenAI 3.5 model.
+
+00:12:13.520 --> 00:12:16.439
+That's more than, well, no,
+
+00:12:16.440 --> 00:12:18.199
+it's like a year old basically at this point.
+
+00:12:18.200 --> 00:12:21.079
+And things are moving fast.
+
+00:12:21.080 --> 00:12:22.039
+They came out with 4.0.
+
+00:12:22.040 --> 00:12:23.319
+4.0 is significantly better.
+
+00:12:23.320 --> 00:12:24.319
+I don't have access to it.
+
+00:12:24.320 --> 00:12:30.839
+Even though I'm using the API and I'm paying money for it,
+
+00:12:30.840 --> 00:12:33.639
+you only can get access to 4.0
+
+00:12:33.640 --> 00:12:34.439
+if you can spend a dollar.
+
+00:12:34.440 --> 00:12:36.319
+And I've never been able to spend,
+
+00:12:36.320 --> 00:12:38.199
+use so much API use that I've spent a dollar.
+
+00:12:38.200 --> 00:12:44.479
+So I have, I don't have 4.0, but I've tried it
+
+00:12:44.480 --> 00:12:46.639
+because I do pay for this
+
+00:12:46.640 --> 00:12:48.340
+so I could get access to 4.0
+
+00:12:48.341 --> 00:12:49.599
+and it is substantially better.
+
+00:12:49.600 --> 00:12:50.519
+By all reports, it's,
+
+00:12:50.520 --> 00:12:53.839
+the difference is extremely significant.
+
+00:12:53.840 --> 00:12:55.159
+I would not be surprised
+
+00:12:55.160 --> 00:12:59.759
+if some of the limitations and drawbacks I described
+
+00:12:59.760 --> 00:13:02.039
+mostly went away with 4.0.
+
+00:13:02.040 --> 00:13:06.679
+We're probably at a stage
+
+00:13:06.680 --> 00:13:09.239
+where regexes will work maybe 5% of the time
+
+00:13:09.240 --> 00:13:10.119
+if you try them.
+
+00:13:10.120 --> 00:13:13.639
+But with 4.0, it could work like 80% of the time.
+
+00:13:13.640 --> 00:13:14.559
+Now, is that good enough?
+
+00:13:14.560 --> 00:13:17.279
+Probably not, but it's a,
+
+00:13:17.280 --> 00:13:20.319
+I wouldn't be surprised if you got results like that.
+
+00:13:20.320 --> 00:13:22.919
+And in a year's time, in two years time,
+
+00:13:22.920 --> 00:13:26.679
+no one knows how much this is going to play out
+
+00:13:26.680 --> 00:13:27.519
+before progress stalls,
+
+00:13:27.520 --> 00:13:32.319
+but there are a lot of interesting research.
+
+00:13:32.320 --> 00:13:34.279
+I don't think, research wise,
+
+00:13:34.280 --> 00:13:35.759
+I don't think things have slowed down.
+
+00:13:35.760 --> 00:13:38.719
+You're still seeing a lot of advances.
+
+00:13:38.720 --> 00:13:40.999
+You're still seeing a lot of models coming out
+
+00:13:41.000 --> 00:13:41.839
+and that will come out.
+
+00:13:41.840 --> 00:13:46.279
+That will be each one, one upping the other one
+
+00:13:46.280 --> 00:13:49.959
+in terms of quality.
+
+00:13:49.960 --> 00:13:52.759
+It'll be really interesting to see how this all plays out.
+
+00:13:52.760 --> 00:13:55.919
+I think that message here is that
+
+00:13:55.920 --> 00:13:57.999
+we're at the beginning here.
+
+00:13:58.000 --> 00:14:01.239
+This is why I think this talk is important.
+
+00:14:01.240 --> 00:14:02.279
+I think this is why we should be
+
+00:14:02.280 --> 00:14:04.159
+paying attention to this stuff.
+
+NOTE LLMs in Emacs - existing packages
+
+00:14:08.200 --> 00:14:11.039
+Let's talk about the existing packages.
+
+00:14:11.040 --> 00:14:13.199
+Because there's a lot out there, people have,
+
+00:14:13.200 --> 00:14:17.039
+I think people have been integrating with
+
+00:14:17.040 --> 00:14:21.239
+these LLMs that often have a relatively easy to use API.
+
+00:14:21.240 --> 00:14:24.039
+So it's kind of natural that people
+
+00:14:24.040 --> 00:14:25.679
+have already put out a lot of packages.
+
+00:14:25.680 --> 00:14:28.319
+Coming off this problem from a lot of different angles,
+
+00:14:28.320 --> 00:14:30.639
+I don't have time to go through
+
+00:14:30.640 --> 00:14:31.959
+all of these packages.
+
+00:14:31.960 --> 00:14:33.559
+These are great packages though.
+
+00:14:33.560 --> 00:14:35.279
+If you're not familiar with them,
+
+00:14:35.280 --> 00:14:37.679
+please check them out.
+
+00:14:37.680 --> 00:14:40.999
+And they all are doing slightly different things.
+
+00:14:41.000 --> 00:14:43.959
+Some of these are relatively straightforward.
+
+00:14:43.960 --> 00:14:47.919
+Interactions, just a way to
+
+00:14:47.920 --> 00:14:52.679
+almost in a comment sort of way to kind of
+
+00:14:52.680 --> 00:14:54.199
+have just an interaction,
+
+00:14:54.200 --> 00:14:55.479
+long running interaction with an LLM
+
+00:14:55.480 --> 00:14:59.039
+where you kind of build off previous responses,
+
+00:14:59.040 --> 00:15:01.799
+kind of like the OpenAI's UI.
+
+00:15:01.800 --> 00:15:08.559
+Two very more Emacsy things where you can sort of
+
+00:15:08.560 --> 00:15:13.679
+embed these LLM responses within a org-mode block
+
+00:15:13.680 --> 00:15:15.239
+using the org-mode's context.
+
+00:15:15.240 --> 00:15:20.959
+Or GitHub Copilot integration where you can use it
+
+00:15:20.960 --> 00:15:23.319
+for auto completion in a very powerful,
+
+00:15:23.320 --> 00:15:27.319
+you know, this stuff is very useful if it could figure out
+
+00:15:27.320 --> 00:15:29.199
+what you're trying to do based on the context.
+
+00:15:29.200 --> 00:15:31.839
+It's quite effective.
+
+00:15:31.840 --> 00:15:36.359
+But I want to kind of call out one thing
+
+00:15:36.360 --> 00:15:38.239
+that I'd like to see change.
+
+00:15:38.240 --> 00:15:42.599
+Which is that users right now,
+
+00:15:42.600 --> 00:15:45.199
+not all of these have a choice of,
+
+00:15:45.200 --> 00:15:47.959
+first of all, there's a lot of them.
+
+00:15:47.960 --> 00:15:49.639
+Each one of them is doing their own calls.
+
+00:15:49.640 --> 00:15:53.999
+And each one of them is, so each one of them
+
+00:15:54.000 --> 00:15:55.239
+has their own interfaces.
+
+00:15:55.240 --> 00:15:57.719
+They're rewriting the interface to OpenAI or wherever.
+
+00:15:57.720 --> 00:16:00.119
+And they're not, they don't, most of these
+
+00:16:00.120 --> 00:16:05.119
+do not make it that configurable or at all configurable
+
+00:16:05.120 --> 00:16:06.599
+what LLM use.
+
+00:16:06.600 --> 00:16:07.239
+This is not good.
+
+00:16:07.240 --> 00:16:09.679
+It is important that we use,
+
+00:16:09.680 --> 00:16:15.679
+we give the user a way to change the LLM they use.
+
+00:16:15.680 --> 00:16:21.079
+And that is because you might not be comfortable
+
+00:16:21.080 --> 00:16:24.439
+sending your requests over to a private corporation
+
+00:16:24.440 --> 00:16:27.799
+where you don't get to see how they use their data.
+
+00:16:27.800 --> 00:16:29.799
+Your data, really.
+
+00:16:29.800 --> 00:16:33.319
+That's especially true with things like embeddings
+
+00:16:33.320 --> 00:16:35.039
+where you might be sending over your documents.
+
+00:16:35.040 --> 00:16:37.519
+You're just giving them your documents, basically.
+
+00:16:37.520 --> 00:16:40.759
+And, you know, that does happen.
+
+00:16:40.760 --> 00:16:43.599
+I don't think really that there's a reason
+
+00:16:43.600 --> 00:16:44.639
+to be uncomfortable with this,
+
+00:16:44.640 --> 00:16:51.439
+but that, you know, people are uncomfortable and that's okay.
+
+00:16:51.440 --> 00:16:53.239
+People might want to use a local machine,
+
+00:16:53.240 --> 00:16:58.359
+a local LLM for maximum privacy.
+
+00:16:58.360 --> 00:17:00.639
+That's something we should allow.
+
+00:17:00.640 --> 00:17:04.519
+People might want to especially use free software.
+
+00:17:04.520 --> 00:17:05.839
+That's something we should definitely allow.
+
+00:17:05.840 --> 00:17:07.279
+This is Emacs.
+
+00:17:07.280 --> 00:17:08.239
+We need to encourage that.
+
+00:17:08.240 --> 00:17:12.159
+But right now, as most of these things are written,
+
+00:17:12.160 --> 00:17:13.959
+you can't do it.
+
+00:17:13.960 --> 00:17:17.839
+And they're spending precious time
+
+00:17:17.840 --> 00:17:18.879
+just doing things themselves.
+
+00:17:18.880 --> 00:17:20.839
+This is why I wrote LLM, which is...
+
+00:17:20.840 --> 00:17:23.039
+it will just make that connection to the LLM for you
+
+00:17:23.040 --> 00:17:26.719
+and it will connect to, you know, it has plugins.
+
+00:17:26.720 --> 00:17:30.279
+So if you can, the user can configure what plugin
+
+00:17:30.280 --> 00:17:31.359
+it actually goes to.
+
+00:17:31.360 --> 00:17:32.399
+Does it go to OpenAI?
+
+00:17:32.400 --> 00:17:35.239
+Does it go to Google Cloud Vertex?
+
+00:17:35.240 --> 00:17:36.999
+Does it go to Llama on your machine?
+
+00:17:37.000 --> 00:17:38.399
+We're using Ollama,
+
+00:17:38.400 --> 00:17:40.999
+which is just a way to run Llama locally.
+
+00:17:41.000 --> 00:17:47.959
+And more things in the future, I hope.
+
+00:17:47.960 --> 00:17:52.079
+So this is, I'm hoping that we use this.
+
+00:17:52.080 --> 00:17:54.839
+It's designed to be sort of maximally usable.
+
+00:17:54.840 --> 00:17:56.279
+You don't need to install anything.
+
+00:17:56.280 --> 00:17:58.359
+It's on GNU ELPA.
+
+00:17:58.360 --> 00:17:59.879
+So even if you write something
+
+00:17:59.880 --> 00:18:01.079
+that you want to contribute to GNU ELPA,
+
+00:18:01.080 --> 00:18:02.879
+you can use it because it's on GNU ELPA.
+
+00:18:02.880 --> 00:18:06.439
+It's part of the Emacs package, Emacs core packages.
+
+00:18:06.440 --> 00:18:09.879
+So, but it has no functionality.
+
+00:18:09.880 --> 00:18:11.719
+It's really just there as a library
+
+00:18:11.720 --> 00:18:14.439
+to use by other things offering functionality. Okay.
+
+NOTE Abstracting LLM challenges
+
+00:18:15.960 --> 00:18:19.839
+And it's a little bit difficult to abstract.
+
+00:18:19.840 --> 00:18:21.159
+I want to point this out
+
+00:18:21.160 --> 00:18:23.599
+because I think it's an important point
+
+00:18:23.600 --> 00:18:29.519
+is that the, it's, some of these LLMs, for example,
+
+00:18:29.520 --> 00:18:30.439
+have image generation.
+
+00:18:30.440 --> 00:18:31.279
+Some do not.
+
+00:18:31.280 --> 00:18:35.319
+Some of them have very large context windows, even for chat.
+
+00:18:35.320 --> 00:18:36.999
+You say, okay, all these things can do chat.
+
+00:18:37.000 --> 00:18:37.319
+Okay.
+
+00:18:37.320 --> 00:18:38.079
+Yeah, kind of.
+
+00:18:38.080 --> 00:18:39.999
+Some of these things you could pass a book to,
+
+00:18:40.000 --> 00:18:41.239
+like Anthropic's API.
+
+00:18:41.240 --> 00:18:43.039
+Most, you cannot.
+
+00:18:43.040 --> 00:18:45.559
+So there really are big differences
+
+00:18:45.560 --> 00:18:46.399
+in how these things work.
+
+00:18:46.400 --> 00:18:51.539
+I hope those differences diminish in the future.
+
+00:18:51.540 --> 00:18:53.800
+But it's just one of the challenges
+
+00:18:53.801 --> 00:18:57.520
+that I hope we can work through in the LLM library.
+
+00:18:57.521 --> 00:19:02.160
+So it's compatible, but there's definitely
+
+00:19:02.161 --> 00:19:04.079
+limits to that compatibility.
+
+NOTE Emacs is the ideal interface for LLMs
+
+00:19:04.080 --> 00:19:06.160
+I want to point out just to finish off,
+
+00:19:06.161 --> 00:19:12.879
+Emacs is the, Emacs has real power here
+
+00:19:12.880 --> 00:19:15.679
+that nothing else I think in the industry is offering.
+
+00:19:15.680 --> 00:19:19.279
+First of all, people that use Emacs
+
+00:19:19.280 --> 00:19:20.439
+tend to do a lot of things in Emacs.
+
+00:19:20.440 --> 00:19:22.159
+We have our to-dos in Emacs with the org mode.
+
+00:19:22.160 --> 00:19:22.999
+We have mail.
+
+00:19:23.000 --> 00:19:25.719
+We, you know, we might read email and we might,
+
+00:19:25.720 --> 00:19:27.679
+and respond to email in Emacs.
+
+00:19:27.680 --> 00:19:29.199
+We might have notes in Emacs.
+
+00:19:29.200 --> 00:19:31.359
+This is very powerful.
+
+00:19:31.360 --> 00:19:34.159
+Using... there's not other stuff like that.
+
+00:19:34.160 --> 00:19:35.759
+And you could feed this stuff to an LLM.
+
+00:19:35.760 --> 00:19:37.039
+You could do interesting things
+
+00:19:37.040 --> 00:19:38.559
+using a combination of all this data.
+
+00:19:38.560 --> 00:19:40.399
+No one else could do this.
+
+00:19:40.400 --> 00:19:41.759
+We need to start thinking about it.
+
+00:19:41.760 --> 00:19:45.039
+Secondly, Emacs can execute commands.
+
+00:19:45.040 --> 00:19:46.239
+This might be a bad idea.
+
+00:19:46.240 --> 00:19:48.399
+This might be how the robots take over,
+
+00:19:48.400 --> 00:19:51.799
+but you could have the LLMs respond with Emacs
+
+00:19:51.800 --> 00:19:54.199
+commands and run those Emacs commands
+
+00:19:54.200 --> 00:19:57.079
+and tell the LLM the response and have it do things
+
+00:19:57.080 --> 00:19:58.679
+as your agent in the editor.
+
+00:19:58.680 --> 00:20:01.599
+I think we need to explore ideas like this.
+
+NOTE Outro
+
+00:20:01.960 --> 00:20:04.279
+And I think we need to share these ideas
+
+00:20:04.280 --> 00:20:07.039
+and we need to make sure that we're pushing the
+
+00:20:07.040 --> 00:20:10.519
+envelope for Emacs and actually, you know, doing things,
+
+00:20:10.520 --> 00:20:12.959
+sharing ideas, sharing progress,
+
+00:20:12.960 --> 00:20:15.199
+and kind of seeing how far we can push this stuff.
+
+00:20:15.200 --> 00:20:20.639
+Let's really help Emacs out, be sort of,
+
+00:20:20.640 --> 00:20:24.519
+take advantage of this super powerful technique.
+
+00:20:24.520 --> 00:20:26.160
+Thank you for listening.