diff options
Diffstat (limited to '2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt')
| -rw-r--r-- | 2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt | 1377 |
1 files changed, 1377 insertions, 0 deletions
diff --git a/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt new file mode 100644 index 00000000..3ac4b34c --- /dev/null +++ b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--main.vtt @@ -0,0 +1,1377 @@ +WEBVTT captioned by bala, checked by sachac + +NOTE Intro to the Talk + +00:00:00.000 --> 00:00:04.159 +Hello, I'm Andrew Hyatt and I'm going to talk to you + +00:00:04.160 --> 00:00:06.439 +about large language models and how + +00:00:06.440 --> 00:00:11.079 +they relate to Emacs. + +00:00:11.080 --> 00:00:14.919 +And I'm going to talk to you about the technology + +00:00:14.920 --> 00:00:18.279 +and how we're going to use it in Emacs. + +00:00:18.280 --> 00:00:21.159 +There'll be demos and there'll be talks about, + +00:00:21.160 --> 00:00:22.879 +I'll finish up by kind of talking about where + +00:00:22.880 --> 00:00:25.079 +I think this should go in the future. + +NOTE What are LLMs? + +00:00:25.080 --> 00:00:28.239 +So to start off with, let's just talk like, + +00:00:28.240 --> 00:00:29.759 +I just want to make sure everyone's on the same page. + +00:00:29.760 --> 00:00:30.919 +What are large language models? + +00:00:30.920 --> 00:00:34.639 +Not everyone may be caught up on this. + +00:00:34.640 --> 00:00:38.999 +Large language models are a way... Basically, + +00:00:39.000 --> 00:00:42.999 +the current versions of large language models + +00:00:43.000 --> 00:00:44.479 +are all based on the similar architecture + +00:00:44.480 --> 00:00:45.279 +called the transformer. + +00:00:45.280 --> 00:00:48.719 +It's just an efficient way to train and produce output. + +00:00:48.720 --> 00:00:51.919 +So these things are basically models + +00:00:51.920 --> 00:00:58.079 +that predict the next word or something like that. + +00:00:58.080 --> 00:01:02.119 +And they're trained on an enormous corpus of information + +00:01:02.120 --> 00:01:04.319 +and they get extremely good + +00:01:04.320 --> 00:01:06.079 +at predicting the next word. + +00:01:06.080 --> 00:01:09.679 +And from that basic ability, you can train + +00:01:09.680 --> 00:01:12.439 +through further tuning from human input, + +00:01:12.440 --> 00:01:13.959 +human ratings and things like that. + +00:01:13.960 --> 00:01:17.479 +You can train different models based on that + +00:01:17.480 --> 00:01:18.759 +that will do question answering. + +00:01:18.760 --> 00:01:22.519 +And this is how basically ChatGPT works. + +00:01:22.520 --> 00:01:25.599 +There's a base LLM, like GPT. + +00:01:25.600 --> 00:01:27.799 +And then you have a chat version of that, + +00:01:27.800 --> 00:01:29.959 +which is just trained to just... You give + +00:01:29.960 --> 00:01:32.199 +it a prompt, like what do you want it to do? + +00:01:32.200 --> 00:01:37.279 +And it gives you an output that does what you told it to do, + +00:01:37.280 --> 00:01:39.919 +or at least attempts to do it. + +00:01:39.920 --> 00:01:42.079 +Those are the power of large language models is + +00:01:42.080 --> 00:01:45.639 +they're extremely, extremely impressive. + +00:01:45.640 --> 00:01:47.199 +Certainly this is, in AI, + +00:01:47.200 --> 00:01:49.079 +this has been the biggest thing to happen + +00:01:49.080 --> 00:01:51.559 +probably in my lifetime, + +00:01:51.560 --> 00:01:56.359 +or at least my lifetime as my working lifetime. + +NOTE Power of LLMs (Magit Demo) + +00:01:56.360 --> 00:02:02.559 +So let me give you a demonstration of + +00:02:02.560 --> 00:02:06.679 +what kinds of stuff it could do in Emacs. + +00:02:06.680 --> 00:02:09.039 +So here I have a Emacs file. + +00:02:09.040 --> 00:02:12.479 +So this is my Emacs init file. + +00:02:12.480 --> 00:02:13.599 +I have a change. + +00:02:13.600 --> 00:02:16.879 +Let's commit that change. + +00:02:16.880 --> 00:02:19.439 +And, you know, I don't like writing commit messages, + +00:02:19.440 --> 00:02:23.039 +so I can generate it. + +00:02:23.040 --> 00:02:27.479 +And it did an actually just looking. + +00:02:27.480 --> 00:02:29.759 +So all it does is it's looking, it's just reading the diff. + +00:02:29.760 --> 00:02:32.479 +I'm just feeding it the diff with some instructions. + +00:02:32.480 --> 00:02:37.759 +And it is this a incredible commit message? + +00:02:37.760 --> 00:02:39.399 +It's not bad, actually. + +00:02:39.400 --> 00:02:42.319 +You can see that it actually has really extracted + +00:02:42.320 --> 00:02:46.439 +the meaning of what I'm doing and has written + +00:02:46.440 --> 00:02:48.879 +a reasonably good commit message. + +00:02:48.880 --> 00:02:53.159 +Now I have to edit it because this is not quite correct. + +00:02:53.160 --> 00:02:55.159 +But it's kind of impressive how good it is. + +00:02:55.160 --> 00:03:00.039 +And my editing, it's kind of easier for me to edit this + +00:03:00.040 --> 00:03:01.879 +than just to write a new one. + +00:03:01.880 --> 00:03:04.479 +And quite often it's good enough to just submit as is. + +00:03:04.480 --> 00:03:08.119 +So this is kind of, you know, you could say + +00:03:08.120 --> 00:03:09.359 +this is just commit messages. + +00:03:09.360 --> 00:03:10.719 +You could respond to emails. + +00:03:10.720 --> 00:03:15.319 +You could, you know, using your own custom instructions + +00:03:15.320 --> 00:03:17.839 +about what you want your email to say. + +00:03:17.840 --> 00:03:19.039 +It'll write the email for you. + +00:03:19.040 --> 00:03:19.839 +It could do like this + +00:03:19.840 --> 00:03:22.519 +Emacs is a way to interact with buffers. + +00:03:22.520 --> 00:03:24.199 +This could basically just output text. + +00:03:24.200 --> 00:03:27.759 +So it's super useful for + +00:03:27.760 --> 00:03:30.319 +understanding something and outputting text based on that, + +00:03:30.320 --> 00:03:32.239 +which is just useful for Emacs. + +NOTE Drawbacks of LLMs (regex demo) + +00:03:32.240 --> 00:03:39.919 +So the drawback is, yeah, it's good, + +00:03:39.920 --> 00:03:43.359 +but it's not that reliable. + +00:03:43.360 --> 00:03:45.679 +And you'd think it's very easy to get caught up in like, + +00:03:45.680 --> 00:03:47.639 +oh my gosh, like this is so powerful. + +00:03:47.640 --> 00:03:50.599 +I bet it could work this, whatever idea could work. + +00:03:50.600 --> 00:03:52.919 +And these ideas, like they almost can. + +00:03:52.920 --> 00:03:55.639 +For example, I was thinking, you know what I could do? + +00:03:55.640 --> 00:03:57.239 +I don't like writing regexes. + +00:03:57.240 --> 00:04:01.199 +Why can't I have a regex replace that's powered by LLMs? + +00:04:01.200 --> 00:04:03.439 +And that way I could give just an instruction + +00:04:03.440 --> 00:04:07.399 +to regex replace. + +00:04:07.400 --> 00:04:12.079 +And so for example, I could do Emacs LLM regex replace. + +00:04:12.080 --> 00:04:12.879 +This is not checked in anywhere. + +00:04:12.880 --> 00:04:17.199 +These are just my own kind of private functions. + +00:04:17.200 --> 00:04:19.239 +My description lowercase all the org headings. + +00:04:19.240 --> 00:04:20.439 +Let's see if it works. + +00:04:20.440 --> 00:04:21.039 +It might work. + +00:04:21.040 --> 00:04:22.959 +No, it doesn't work. + +00:04:22.960 --> 00:04:26.159 +So if I, I'm not going to bother to show you + +00:04:26.160 --> 00:04:28.159 +what it actually came up with, but it's something, + +00:04:28.160 --> 00:04:29.879 +if you looked at it, it'd be like, wow, + +00:04:29.880 --> 00:04:31.639 +this is very close to being... + +00:04:31.640 --> 00:04:34.239 +It looks like it should work, but it doesn't. + +00:04:34.240 --> 00:04:35.839 +Okay. + +00:04:35.840 --> 00:04:38.719 +It's not quite good enough to get it right. + +00:04:38.720 --> 00:04:41.599 +And it's possible that perhaps by giving it + +00:04:41.600 --> 00:04:43.639 +a few examples of, or explaining more + +00:04:43.640 --> 00:04:46.439 +what makes Emacs regexes different. + +00:04:46.440 --> 00:04:47.959 +It could do a better job + +00:04:47.960 --> 00:04:49.279 +and maybe could solve these problems, + +00:04:49.280 --> 00:04:50.679 +but it's always a little bit random. + +00:04:50.680 --> 00:04:52.359 +You're never quite sure what you're going to get. + +00:04:52.360 --> 00:04:54.839 +So this is the drawback. + +00:04:54.840 --> 00:04:58.479 +Like there's a lot of things that look like you could do it, + +00:04:58.480 --> 00:05:00.999 +but when it actually comes down to trying it, + +00:05:01.000 --> 00:05:03.399 +it's surprisingly hard. + +00:05:03.400 --> 00:05:06.319 +And, you know, and whatever you're doing, + +00:05:06.320 --> 00:05:08.999 +it's surprisingly hard to get something + +00:05:09.000 --> 00:05:13.879 +that is repeatably, that's, that is always good. + +00:05:13.880 --> 00:05:20.119 +So yeah, that's currently the problem. + +NOTE Embeddings + +00:05:20.120 --> 00:05:23.399 +So I want to talk about embeddings. + +00:05:23.400 --> 00:05:26.919 +They're another thing that LLMs offer + +00:05:26.920 --> 00:05:28.599 +and that are extremely useful. + +00:05:28.600 --> 00:05:33.119 +They are, what they do is they encode from + +00:05:33.120 --> 00:05:38.959 +a input text that could be a word, a sentence, + +00:05:38.960 --> 00:05:42.159 +a small document. + +00:05:42.160 --> 00:05:45.399 +It encodes a vector about what the meaning, + +00:05:45.400 --> 00:05:46.919 +the semantic meaning of that is. + +00:05:46.920 --> 00:05:51.079 +That means you could, something that is, + +00:05:51.080 --> 00:05:52.279 +uses completely different words, + +00:05:52.280 --> 00:05:54.159 +but is basically talking about the same thing, + +00:05:54.160 --> 00:05:57.839 +perhaps in a different language, should be pretty close + +00:05:57.840 --> 00:06:01.999 +as a vector to the other vector. + +00:06:02.000 --> 00:06:05.399 +You know, as long as they're similarly semantic things, + +00:06:05.400 --> 00:06:12.239 +like the words + +00:06:12.240 --> 00:06:18.959 +highway and Camino are two different words. + +00:06:18.960 --> 00:06:19.639 +They mean the same thing. + +00:06:19.640 --> 00:06:21.319 +They should have very similar embeddings. + +00:06:21.320 --> 00:06:25.119 +So it is a way to kind of encode this + +00:06:25.120 --> 00:06:26.199 +and then you could use this for search. + +00:06:26.200 --> 00:06:28.919 +For example, I haven't tried to do this yet, + +00:06:28.920 --> 00:06:31.479 +but you could probably just make an embedding + +00:06:31.480 --> 00:06:33.919 +for every paragraph in the Emacs manual + +00:06:33.920 --> 00:06:36.239 +and the Elisp manual. + +00:06:36.240 --> 00:06:39.439 +And then, and then there's a very standard technique. + +00:06:39.440 --> 00:06:43.439 +You just... You find that you have a query, + +00:06:43.440 --> 00:06:45.799 +oh, how do I do whatever, whatever in Emacs again? + +00:06:45.800 --> 00:06:49.479 +And you could, you just find that 20 things + +00:06:49.480 --> 00:06:50.319 +that are closest to whatever you're + +00:06:50.320 --> 00:06:51.839 +trying to... the embedding of your query. + +00:06:51.840 --> 00:06:55.279 +You send those things to the LLM, as you know, + +00:06:55.280 --> 00:06:57.799 +with the original query, + +00:06:57.800 --> 00:06:59.919 +and you're basically telling the--asking the LLM, + +00:06:59.920 --> 00:07:01.279 +look, the user is trying to do this. + +00:07:01.280 --> 00:07:03.039 +Here's what I found in the Emacs manual. + +00:07:03.040 --> 00:07:04.639 +That's on the Elisp manual. + +00:07:04.640 --> 00:07:07.439 +That's close to what they're trying to do. + +00:07:07.440 --> 00:07:12.159 +So can you kind of just tell the user what to do? + +00:07:12.160 --> 00:07:14.479 +And from this, and you could say, + +00:07:14.480 --> 00:07:17.639 +just use things from this, you know, that I give you. + +00:07:17.640 --> 00:07:20.679 +Don't just make up your own idea. + +00:07:20.680 --> 00:07:21.839 +You know, don't use your own ideas, + +00:07:21.840 --> 00:07:23.799 +because sometimes it likes to do that + +00:07:23.800 --> 00:07:24.359 +and those things are wrong. + +00:07:24.360 --> 00:07:26.719 +So you could try to, you know, do this and you get, + +00:07:26.720 --> 00:07:28.719 +you could get quite good results using this. + +00:07:28.720 --> 00:07:29.999 +So no one has done this yet, + +00:07:30.000 --> 00:07:32.799 +but that should not be hard to do. + +NOTE Image Generation + +00:07:32.800 --> 00:07:34.879 +Image generation is something that's, you know, + +00:07:34.880 --> 00:07:38.479 +it's not quite an LLM in the sense of... + +00:07:38.480 --> 00:07:43.079 +These are... It's a different technology, + +00:07:43.080 --> 00:07:48.439 +but these things are kind of packaged together + +00:07:48.440 --> 00:07:49.039 +in a sense. + +00:07:49.040 --> 00:07:51.639 +And you'll see that when I talk about Emacs packages, + +00:07:51.640 --> 00:07:54.279 +a lot of them bundle image generation + +00:07:54.280 --> 00:07:55.439 +and large language models. + +00:07:55.440 --> 00:07:59.039 +You know, the APIs are often bundled together by providers. + +00:07:59.040 --> 00:08:02.679 +And the general idea is it's kind of similar + +00:08:02.680 --> 00:08:04.399 +because it's very similar to large, you know, + +00:08:04.400 --> 00:08:06.559 +doing a chat thing where you, you know, + +00:08:06.560 --> 00:08:09.760 +the chat is like, you give it a text request, + +00:08:09.761 --> 00:08:12.759 +like write me a sonnet about, you know, + +00:08:12.760 --> 00:08:14.879 +the battle between Emacs and vi. + +00:08:14.880 --> 00:08:15.839 +And it could, it could do it. + +00:08:15.840 --> 00:08:17.159 +It could do a very good job of that. + +00:08:17.160 --> 00:08:22.519 +But you could also say, you know, + +00:08:22.520 --> 00:08:27.599 +draw me a picture of Emacs and vi as boxers, + +00:08:27.600 --> 00:08:30.359 +as a character-character boxing in a ring, + +00:08:30.360 --> 00:08:32.239 +like a, you know, political cartoon style. + +00:08:32.240 --> 00:08:34.999 +And it can do that as well. + +00:08:35.000 --> 00:08:37.679 +And so you could basically think of this + +00:08:37.680 --> 00:08:39.439 +as just sort of... it's kind of the + +00:08:39.440 --> 00:08:42.399 +same thing with what you're doing + +00:08:42.400 --> 00:08:43.359 +with large language models, + +00:08:43.360 --> 00:08:44.799 +but instead of outputting a text, + +00:08:44.800 --> 00:08:48.479 +you're outputting a picture. + +NOTE Fine-tuning + +00:08:48.480 --> 00:08:51.079 +There's also, I want to mention the concept of fine-tuning. + +00:08:51.080 --> 00:08:55.199 +Fine-tuning is a way to take your-- + +00:08:55.200 --> 00:08:59.759 +take a corpus of inputs and outputs and just from + +00:08:59.760 --> 00:09:01.599 +a large language model, you're like, okay, + +00:09:01.600 --> 00:09:03.599 +given this base large language model, + +00:09:03.600 --> 00:09:06.679 +I want to make sure that when I give you input, + +00:09:06.680 --> 00:09:08.479 +you give me something like output. + +00:09:08.480 --> 00:09:10.119 +And this is what I'm just going to + +00:09:10.120 --> 00:09:11.799 +train you further on these, + +00:09:11.800 --> 00:09:14.879 +these mappings between input and output. + +00:09:14.880 --> 00:09:16.399 +And for example, you could do this. Like, + +00:09:16.400 --> 00:09:18.039 +let's say you wanted to fix that regex demo + +00:09:18.040 --> 00:09:20.999 +I had to make it good. + +00:09:21.000 --> 00:09:23.479 +I don't think it, I think it'd be + +00:09:23.480 --> 00:09:25.039 +relatively effective to train, + +00:09:25.040 --> 00:09:27.039 +to have regex descriptions + +00:09:27.040 --> 00:09:30.119 +and regex examples, Emacs regex examples + +00:09:30.120 --> 00:09:31.239 +as inputs and outputs. + +00:09:31.240 --> 00:09:33.999 +You could get, you know, maybe a hundred, + +00:09:34.000 --> 00:09:35.359 +a few hundreds of these things. + +00:09:35.360 --> 00:09:38.639 +You could train it. + +00:09:38.640 --> 00:09:40.759 +I think that is a reasonable way to, + +00:09:40.760 --> 00:09:43.879 +let's just say, I don't know how well it would work, + +00:09:43.880 --> 00:09:46.839 +but these things definitely work some of the time + +00:09:46.840 --> 00:09:47.999 +and produce pretty good results. + +00:09:48.000 --> 00:09:53.039 +And you could do this on your own machine. + +00:09:53.040 --> 00:09:58.999 +Corporations like OpenAI offer APIs with, you know, + +00:09:59.000 --> 00:10:01.519 +to build your fine tunes on top of OpenAI. + +00:10:01.520 --> 00:10:04.159 +And I think, I'm not a hundred percent sure, + +00:10:04.160 --> 00:10:05.719 +but I think then you can share your model + +00:10:05.720 --> 00:10:06.519 +with other people. + +00:10:06.520 --> 00:10:08.519 +But if not, then you just, you know, + +00:10:08.520 --> 00:10:10.839 +you could use your model for your own specialized purposes. + +00:10:10.840 --> 00:10:14.039 +But in the world of models that you could run, + +00:10:14.040 --> 00:10:16.874 +for example, based on Llama, which is like... + +00:10:16.875 --> 00:10:22.240 +Llama is this model you can run on your own machine from Meta. + +00:10:23.580 --> 00:10:26.880 +There's many fine-tuned models that you could download + +00:10:26.881 --> 00:10:28.960 +and you could run on your own. + +00:10:28.961 --> 00:10:30.839 +They can do very different things too. + +00:10:30.840 --> 00:10:33.399 +Some output Python programs, for example, + +00:10:33.400 --> 00:10:34.279 +that you could just run. + +00:10:34.280 --> 00:10:37.959 +So you just say... + +00:10:37.960 --> 00:10:40.639 +Tell me how old... Let's just say + +00:10:40.640 --> 00:10:41.999 +you have a random task, like + +00:10:42.000 --> 00:10:48.119 +tell me how old these five cities are in minutes, + +00:10:48.120 --> 00:10:49.799 +based on historical evidence. + +00:10:49.800 --> 00:10:53.639 +It's kind of a weird query, but it probably can figure, + +00:10:53.640 --> 00:10:55.119 +it could probably run that for you. + +00:10:55.120 --> 00:10:57.239 +It'll encode its knowledge into whatever + +00:10:57.240 --> 00:10:59.599 +the Python program, then use the Python program + +00:10:59.600 --> 00:11:01.039 +to do the correct calculations. + +00:11:01.040 --> 00:11:05.679 +So pretty, pretty useful stuff. + +NOTE Open Source + +00:11:08.160 --> 00:11:10.399 +So I also want to mention open source + +00:11:10.400 --> 00:11:12.679 +and basically free software here. + +00:11:12.680 --> 00:11:17.599 +These LLMs are mostly not free software. + +00:11:17.600 --> 00:11:19.159 +They're sometimes open source, + +00:11:19.160 --> 00:11:21.959 +but they're generally not free + +00:11:21.960 --> 00:11:23.799 +without restrictions to use. + +00:11:23.800 --> 00:11:27.279 +Most of these things, even Llama, + +00:11:27.280 --> 00:11:28.679 +which you can use on your own machine, + +00:11:28.680 --> 00:11:31.439 +have restrictions that you cannot use it + +00:11:31.440 --> 00:11:32.519 +to train your own model. + +00:11:32.520 --> 00:11:35.119 +This is something that, you know, + +00:11:35.120 --> 00:11:37.519 +it costs millions and millions of dollars + +00:11:37.520 --> 00:11:40.759 +to train and produce these models. + +00:11:40.760 --> 00:11:42.319 +And that's just computation costs. + +00:11:42.320 --> 00:11:45.519 +They do not want you + +00:11:45.520 --> 00:11:47.839 +stealing all that work by training your own models + +00:11:47.840 --> 00:11:48.799 +based on their output. + +00:11:48.800 --> 00:11:55.359 +But there are research LLMs that do, I believe, + +00:11:55.360 --> 00:11:57.999 +conform to free software principles. + +00:11:58.000 --> 00:11:59.519 +They're just not as good yet. + +00:11:59.520 --> 00:12:02.519 +And I think that might change in the future. + +NOTE The Future + +00:12:02.840 --> 00:12:04.119 +So speaking of the future, + +00:12:04.120 --> 00:12:07.519 +one of the things I'd like to point out + +00:12:07.520 --> 00:12:09.639 +is that like the demos I showed you are based on, + +00:12:09.640 --> 00:12:13.519 +I'm using OpenAI 3.5 model. + +00:12:13.520 --> 00:12:16.439 +That's more than, well, no, + +00:12:16.440 --> 00:12:18.199 +it's like a year old basically at this point. + +00:12:18.200 --> 00:12:21.079 +And things are moving fast. + +00:12:21.080 --> 00:12:22.039 +They came out with 4.0. + +00:12:22.040 --> 00:12:23.319 +4.0 is significantly better. + +00:12:23.320 --> 00:12:24.319 +I don't have access to it. + +00:12:24.320 --> 00:12:30.839 +Even though I'm using the API and I'm paying money for it, + +00:12:30.840 --> 00:12:33.639 +you only can get access to 4.0 + +00:12:33.640 --> 00:12:34.439 +if you can spend a dollar. + +00:12:34.440 --> 00:12:36.319 +And I've never been able to spend, + +00:12:36.320 --> 00:12:38.199 +use so much API use that I've spent a dollar. + +00:12:38.200 --> 00:12:44.479 +So I have, I don't have 4.0, but I've tried it + +00:12:44.480 --> 00:12:46.639 +because I do pay for this + +00:12:46.640 --> 00:12:48.340 +so I could get access to 4.0 + +00:12:48.341 --> 00:12:49.599 +and it is substantially better. + +00:12:49.600 --> 00:12:50.519 +By all reports, it's, + +00:12:50.520 --> 00:12:53.839 +the difference is extremely significant. + +00:12:53.840 --> 00:12:55.159 +I would not be surprised + +00:12:55.160 --> 00:12:59.759 +if some of the limitations and drawbacks I described + +00:12:59.760 --> 00:13:02.039 +mostly went away with 4.0. + +00:13:02.040 --> 00:13:06.679 +We're probably at a stage + +00:13:06.680 --> 00:13:09.239 +where regexes will work maybe 5% of the time + +00:13:09.240 --> 00:13:10.119 +if you try them. + +00:13:10.120 --> 00:13:13.639 +But with 4.0, it could work like 80% of the time. + +00:13:13.640 --> 00:13:14.559 +Now, is that good enough? + +00:13:14.560 --> 00:13:17.279 +Probably not, but it's a, + +00:13:17.280 --> 00:13:20.319 +I wouldn't be surprised if you got results like that. + +00:13:20.320 --> 00:13:22.919 +And in a year's time, in two years time, + +00:13:22.920 --> 00:13:26.679 +no one knows how much this is going to play out + +00:13:26.680 --> 00:13:27.519 +before progress stalls, + +00:13:27.520 --> 00:13:32.319 +but there are a lot of interesting research. + +00:13:32.320 --> 00:13:34.279 +I don't think, research wise, + +00:13:34.280 --> 00:13:35.759 +I don't think things have slowed down. + +00:13:35.760 --> 00:13:38.719 +You're still seeing a lot of advances. + +00:13:38.720 --> 00:13:40.999 +You're still seeing a lot of models coming out + +00:13:41.000 --> 00:13:41.839 +and that will come out. + +00:13:41.840 --> 00:13:46.279 +That will be each one, one upping the other one + +00:13:46.280 --> 00:13:49.959 +in terms of quality. + +00:13:49.960 --> 00:13:52.759 +It'll be really interesting to see how this all plays out. + +00:13:52.760 --> 00:13:55.919 +I think that message here is that + +00:13:55.920 --> 00:13:57.999 +we're at the beginning here. + +00:13:58.000 --> 00:14:01.239 +This is why I think this talk is important. + +00:14:01.240 --> 00:14:02.279 +I think this is why we should be + +00:14:02.280 --> 00:14:04.159 +paying attention to this stuff. + +NOTE LLMs in Emacs - existing packages + +00:14:08.200 --> 00:14:11.039 +Let's talk about the existing packages. + +00:14:11.040 --> 00:14:13.199 +Because there's a lot out there, people have, + +00:14:13.200 --> 00:14:17.039 +I think people have been integrating with + +00:14:17.040 --> 00:14:21.239 +these LLMs that often have a relatively easy to use API. + +00:14:21.240 --> 00:14:24.039 +So it's kind of natural that people + +00:14:24.040 --> 00:14:25.679 +have already put out a lot of packages. + +00:14:25.680 --> 00:14:28.319 +Coming off this problem from a lot of different angles, + +00:14:28.320 --> 00:14:30.639 +I don't have time to go through + +00:14:30.640 --> 00:14:31.959 +all of these packages. + +00:14:31.960 --> 00:14:33.559 +These are great packages though. + +00:14:33.560 --> 00:14:35.279 +If you're not familiar with them, + +00:14:35.280 --> 00:14:37.679 +please check them out. + +00:14:37.680 --> 00:14:40.999 +And they all are doing slightly different things. + +00:14:41.000 --> 00:14:43.959 +Some of these are relatively straightforward. + +00:14:43.960 --> 00:14:47.919 +Interactions, just a way to + +00:14:47.920 --> 00:14:52.679 +almost in a comment sort of way to kind of + +00:14:52.680 --> 00:14:54.199 +have just an interaction, + +00:14:54.200 --> 00:14:55.479 +long running interaction with an LLM + +00:14:55.480 --> 00:14:59.039 +where you kind of build off previous responses, + +00:14:59.040 --> 00:15:01.799 +kind of like the OpenAI's UI. + +00:15:01.800 --> 00:15:08.559 +Two very more Emacsy things where you can sort of + +00:15:08.560 --> 00:15:13.679 +embed these LLM responses within a org-mode block + +00:15:13.680 --> 00:15:15.239 +using the org-mode's context. + +00:15:15.240 --> 00:15:20.959 +Or GitHub Copilot integration where you can use it + +00:15:20.960 --> 00:15:23.319 +for auto completion in a very powerful, + +00:15:23.320 --> 00:15:27.319 +you know, this stuff is very useful if it could figure out + +00:15:27.320 --> 00:15:29.199 +what you're trying to do based on the context. + +00:15:29.200 --> 00:15:31.839 +It's quite effective. + +00:15:31.840 --> 00:15:36.359 +But I want to kind of call out one thing + +00:15:36.360 --> 00:15:38.239 +that I'd like to see change. + +00:15:38.240 --> 00:15:42.599 +Which is that users right now, + +00:15:42.600 --> 00:15:45.199 +not all of these have a choice of, + +00:15:45.200 --> 00:15:47.959 +first of all, there's a lot of them. + +00:15:47.960 --> 00:15:49.639 +Each one of them is doing their own calls. + +00:15:49.640 --> 00:15:53.999 +And each one of them is, so each one of them + +00:15:54.000 --> 00:15:55.239 +has their own interfaces. + +00:15:55.240 --> 00:15:57.719 +They're rewriting the interface to OpenAI or wherever. + +00:15:57.720 --> 00:16:00.119 +And they're not, they don't, most of these + +00:16:00.120 --> 00:16:05.119 +do not make it that configurable or at all configurable + +00:16:05.120 --> 00:16:06.599 +what LLM use. + +00:16:06.600 --> 00:16:07.239 +This is not good. + +00:16:07.240 --> 00:16:09.679 +It is important that we use, + +00:16:09.680 --> 00:16:15.679 +we give the user a way to change the LLM they use. + +00:16:15.680 --> 00:16:21.079 +And that is because you might not be comfortable + +00:16:21.080 --> 00:16:24.439 +sending your requests over to a private corporation + +00:16:24.440 --> 00:16:27.799 +where you don't get to see how they use their data. + +00:16:27.800 --> 00:16:29.799 +Your data, really. + +00:16:29.800 --> 00:16:33.319 +That's especially true with things like embeddings + +00:16:33.320 --> 00:16:35.039 +where you might be sending over your documents. + +00:16:35.040 --> 00:16:37.519 +You're just giving them your documents, basically. + +00:16:37.520 --> 00:16:40.759 +And, you know, that does happen. + +00:16:40.760 --> 00:16:43.599 +I don't think really that there's a reason + +00:16:43.600 --> 00:16:44.639 +to be uncomfortable with this, + +00:16:44.640 --> 00:16:51.439 +but that, you know, people are uncomfortable and that's okay. + +00:16:51.440 --> 00:16:53.239 +People might want to use a local machine, + +00:16:53.240 --> 00:16:58.359 +a local LLM for maximum privacy. + +00:16:58.360 --> 00:17:00.639 +That's something we should allow. + +00:17:00.640 --> 00:17:04.519 +People might want to especially use free software. + +00:17:04.520 --> 00:17:05.839 +That's something we should definitely allow. + +00:17:05.840 --> 00:17:07.279 +This is Emacs. + +00:17:07.280 --> 00:17:08.239 +We need to encourage that. + +00:17:08.240 --> 00:17:12.159 +But right now, as most of these things are written, + +00:17:12.160 --> 00:17:13.959 +you can't do it. + +00:17:13.960 --> 00:17:17.839 +And they're spending precious time + +00:17:17.840 --> 00:17:18.879 +just doing things themselves. + +00:17:18.880 --> 00:17:20.839 +This is why I wrote LLM, which is... + +00:17:20.840 --> 00:17:23.039 +it will just make that connection to the LLM for you + +00:17:23.040 --> 00:17:26.719 +and it will connect to, you know, it has plugins. + +00:17:26.720 --> 00:17:30.279 +So if you can, the user can configure what plugin + +00:17:30.280 --> 00:17:31.359 +it actually goes to. + +00:17:31.360 --> 00:17:32.399 +Does it go to OpenAI? + +00:17:32.400 --> 00:17:35.239 +Does it go to Google Cloud Vertex? + +00:17:35.240 --> 00:17:36.999 +Does it go to Llama on your machine? + +00:17:37.000 --> 00:17:38.399 +We're using Ollama, + +00:17:38.400 --> 00:17:40.999 +which is just a way to run Llama locally. + +00:17:41.000 --> 00:17:47.959 +And more things in the future, I hope. + +00:17:47.960 --> 00:17:52.079 +So this is, I'm hoping that we use this. + +00:17:52.080 --> 00:17:54.839 +It's designed to be sort of maximally usable. + +00:17:54.840 --> 00:17:56.279 +You don't need to install anything. + +00:17:56.280 --> 00:17:58.359 +It's on GNU ELPA. + +00:17:58.360 --> 00:17:59.879 +So even if you write something + +00:17:59.880 --> 00:18:01.079 +that you want to contribute to GNU ELPA, + +00:18:01.080 --> 00:18:02.879 +you can use it because it's on GNU ELPA. + +00:18:02.880 --> 00:18:06.439 +It's part of the Emacs package, Emacs core packages. + +00:18:06.440 --> 00:18:09.879 +So, but it has no functionality. + +00:18:09.880 --> 00:18:11.719 +It's really just there as a library + +00:18:11.720 --> 00:18:14.439 +to use by other things offering functionality. Okay. + +NOTE Abstracting LLM challenges + +00:18:15.960 --> 00:18:19.839 +And it's a little bit difficult to abstract. + +00:18:19.840 --> 00:18:21.159 +I want to point this out + +00:18:21.160 --> 00:18:23.599 +because I think it's an important point + +00:18:23.600 --> 00:18:29.519 +is that the, it's, some of these LLMs, for example, + +00:18:29.520 --> 00:18:30.439 +have image generation. + +00:18:30.440 --> 00:18:31.279 +Some do not. + +00:18:31.280 --> 00:18:35.319 +Some of them have very large context windows, even for chat. + +00:18:35.320 --> 00:18:36.999 +You say, okay, all these things can do chat. + +00:18:37.000 --> 00:18:37.319 +Okay. + +00:18:37.320 --> 00:18:38.079 +Yeah, kind of. + +00:18:38.080 --> 00:18:39.999 +Some of these things you could pass a book to, + +00:18:40.000 --> 00:18:41.239 +like Anthropic's API. + +00:18:41.240 --> 00:18:43.039 +Most, you cannot. + +00:18:43.040 --> 00:18:45.559 +So there really are big differences + +00:18:45.560 --> 00:18:46.399 +in how these things work. + +00:18:46.400 --> 00:18:51.539 +I hope those differences diminish in the future. + +00:18:51.540 --> 00:18:53.800 +But it's just one of the challenges + +00:18:53.801 --> 00:18:57.520 +that I hope we can work through in the LLM library. + +00:18:57.521 --> 00:19:02.160 +So it's compatible, but there's definitely + +00:19:02.161 --> 00:19:04.079 +limits to that compatibility. + +NOTE Emacs is the ideal interface for LLMs + +00:19:04.080 --> 00:19:06.160 +I want to point out just to finish off, + +00:19:06.161 --> 00:19:12.879 +Emacs is the, Emacs has real power here + +00:19:12.880 --> 00:19:15.679 +that nothing else I think in the industry is offering. + +00:19:15.680 --> 00:19:19.279 +First of all, people that use Emacs + +00:19:19.280 --> 00:19:20.439 +tend to do a lot of things in Emacs. + +00:19:20.440 --> 00:19:22.159 +We have our to-dos in Emacs with the org mode. + +00:19:22.160 --> 00:19:22.999 +We have mail. + +00:19:23.000 --> 00:19:25.719 +We, you know, we might read email and we might, + +00:19:25.720 --> 00:19:27.679 +and respond to email in Emacs. + +00:19:27.680 --> 00:19:29.199 +We might have notes in Emacs. + +00:19:29.200 --> 00:19:31.359 +This is very powerful. + +00:19:31.360 --> 00:19:34.159 +Using... there's not other stuff like that. + +00:19:34.160 --> 00:19:35.759 +And you could feed this stuff to an LLM. + +00:19:35.760 --> 00:19:37.039 +You could do interesting things + +00:19:37.040 --> 00:19:38.559 +using a combination of all this data. + +00:19:38.560 --> 00:19:40.399 +No one else could do this. + +00:19:40.400 --> 00:19:41.759 +We need to start thinking about it. + +00:19:41.760 --> 00:19:45.039 +Secondly, Emacs can execute commands. + +00:19:45.040 --> 00:19:46.239 +This might be a bad idea. + +00:19:46.240 --> 00:19:48.399 +This might be how the robots take over, + +00:19:48.400 --> 00:19:51.799 +but you could have the LLMs respond with Emacs + +00:19:51.800 --> 00:19:54.199 +commands and run those Emacs commands + +00:19:54.200 --> 00:19:57.079 +and tell the LLM the response and have it do things + +00:19:57.080 --> 00:19:58.679 +as your agent in the editor. + +00:19:58.680 --> 00:20:01.599 +I think we need to explore ideas like this. + +NOTE Outro + +00:20:01.960 --> 00:20:04.279 +And I think we need to share these ideas + +00:20:04.280 --> 00:20:07.039 +and we need to make sure that we're pushing the + +00:20:07.040 --> 00:20:10.519 +envelope for Emacs and actually, you know, doing things, + +00:20:10.520 --> 00:20:12.959 +sharing ideas, sharing progress, + +00:20:12.960 --> 00:20:15.199 +and kind of seeing how far we can push this stuff. + +00:20:15.200 --> 00:20:20.639 +Let's really help Emacs out, be sort of, + +00:20:20.640 --> 00:20:24.519 +take advantage of this super powerful technique. + +00:20:24.520 --> 00:20:26.160 +Thank you for listening. |