WEBVTT captioned by bala, checked by sachac NOTE Intro to the Talk 00:00:00.000 --> 00:00:04.159 Hello, I'm Andrew Hyatt and I'm going to talk to you 00:00:04.160 --> 00:00:06.439 about large language models and how 00:00:06.440 --> 00:00:11.079 they relate to Emacs. 00:00:11.080 --> 00:00:14.919 And I'm going to talk to you about the technology 00:00:14.920 --> 00:00:18.279 and how we're going to use it in Emacs. 00:00:18.280 --> 00:00:21.159 There'll be demos and there'll be talks about, 00:00:21.160 --> 00:00:22.879 I'll finish up by kind of talking about where 00:00:22.880 --> 00:00:25.079 I think this should go in the future. NOTE What are LLMs? 00:00:25.080 --> 00:00:28.239 So to start off with, let's just talk like, 00:00:28.240 --> 00:00:29.759 I just want to make sure everyone's on the same page. 00:00:29.760 --> 00:00:30.919 What are large language models? 00:00:30.920 --> 00:00:34.639 Not everyone may be caught up on this. 00:00:34.640 --> 00:00:38.999 Large language models are a way... Basically, 00:00:39.000 --> 00:00:42.999 the current versions of large language models 00:00:43.000 --> 00:00:44.479 are all based on the similar architecture 00:00:44.480 --> 00:00:45.279 called the transformer. 00:00:45.280 --> 00:00:48.719 It's just an efficient way to train and produce output. 00:00:48.720 --> 00:00:51.919 So these things are basically models 00:00:51.920 --> 00:00:58.079 that predict the next word or something like that. 00:00:58.080 --> 00:01:02.119 And they're trained on an enormous corpus of information 00:01:02.120 --> 00:01:04.319 and they get extremely good 00:01:04.320 --> 00:01:06.079 at predicting the next word. 00:01:06.080 --> 00:01:09.679 And from that basic ability, you can train 00:01:09.680 --> 00:01:12.439 through further tuning from human input, 00:01:12.440 --> 00:01:13.959 human ratings and things like that. 00:01:13.960 --> 00:01:17.479 You can train different models based on that 00:01:17.480 --> 00:01:18.759 that will do question answering. 00:01:18.760 --> 00:01:22.519 And this is how basically ChatGPT works. 00:01:22.520 --> 00:01:25.599 There's a base LLM, like GPT. 00:01:25.600 --> 00:01:27.799 And then you have a chat version of that, 00:01:27.800 --> 00:01:29.959 which is just trained to just... You give 00:01:29.960 --> 00:01:32.199 it a prompt, like what do you want it to do? 00:01:32.200 --> 00:01:37.279 And it gives you an output that does what you told it to do, 00:01:37.280 --> 00:01:39.919 or at least attempts to do it. 00:01:39.920 --> 00:01:42.079 Those are the power of large language models is 00:01:42.080 --> 00:01:45.639 they're extremely, extremely impressive. 00:01:45.640 --> 00:01:47.199 Certainly this is, in AI, 00:01:47.200 --> 00:01:49.079 this has been the biggest thing to happen 00:01:49.080 --> 00:01:51.559 probably in my lifetime, 00:01:51.560 --> 00:01:56.359 or at least my lifetime as my working lifetime. NOTE Power of LLMs (Magit Demo) 00:01:56.360 --> 00:02:02.559 So let me give you a demonstration of 00:02:02.560 --> 00:02:06.679 what kinds of stuff it could do in Emacs. 00:02:06.680 --> 00:02:09.039 So here I have a Emacs file. 00:02:09.040 --> 00:02:12.479 So this is my Emacs init file. 00:02:12.480 --> 00:02:13.599 I have a change. 00:02:13.600 --> 00:02:16.879 Let's commit that change. 00:02:16.880 --> 00:02:19.439 And, you know, I don't like writing commit messages, 00:02:19.440 --> 00:02:23.039 so I can generate it. 00:02:23.040 --> 00:02:27.479 And it did an actually just looking. 00:02:27.480 --> 00:02:29.759 So all it does is it's looking, it's just reading the diff. 00:02:29.760 --> 00:02:32.479 I'm just feeding it the diff with some instructions. 00:02:32.480 --> 00:02:37.759 And it is this a incredible commit message? 00:02:37.760 --> 00:02:39.399 It's not bad, actually. 00:02:39.400 --> 00:02:42.319 You can see that it actually has really extracted 00:02:42.320 --> 00:02:46.439 the meaning of what I'm doing and has written 00:02:46.440 --> 00:02:48.879 a reasonably good commit message. 00:02:48.880 --> 00:02:53.159 Now I have to edit it because this is not quite correct. 00:02:53.160 --> 00:02:55.159 But it's kind of impressive how good it is. 00:02:55.160 --> 00:03:00.039 And my editing, it's kind of easier for me to edit this 00:03:00.040 --> 00:03:01.879 than just to write a new one. 00:03:01.880 --> 00:03:04.479 And quite often it's good enough to just submit as is. 00:03:04.480 --> 00:03:08.119 So this is kind of, you know, you could say 00:03:08.120 --> 00:03:09.359 this is just commit messages. 00:03:09.360 --> 00:03:10.719 You could respond to emails. 00:03:10.720 --> 00:03:15.319 You could, you know, using your own custom instructions 00:03:15.320 --> 00:03:17.839 about what you want your email to say. 00:03:17.840 --> 00:03:19.039 It'll write the email for you. 00:03:19.040 --> 00:03:19.839 It could do like this 00:03:19.840 --> 00:03:22.519 Emacs is a way to interact with buffers. 00:03:22.520 --> 00:03:24.199 This could basically just output text. 00:03:24.200 --> 00:03:27.759 So it's super useful for 00:03:27.760 --> 00:03:30.319 understanding something and outputting text based on that, 00:03:30.320 --> 00:03:32.239 which is just useful for Emacs. NOTE Drawbacks of LLMs (regex demo) 00:03:32.240 --> 00:03:39.919 So the drawback is, yeah, it's good, 00:03:39.920 --> 00:03:43.359 but it's not that reliable. 00:03:43.360 --> 00:03:45.679 And you'd think it's very easy to get caught up in like, 00:03:45.680 --> 00:03:47.639 oh my gosh, like this is so powerful. 00:03:47.640 --> 00:03:50.599 I bet it could work this, whatever idea could work. 00:03:50.600 --> 00:03:52.919 And these ideas, like they almost can. 00:03:52.920 --> 00:03:55.639 For example, I was thinking, you know what I could do? 00:03:55.640 --> 00:03:57.239 I don't like writing regexes. 00:03:57.240 --> 00:04:01.199 Why can't I have a regex replace that's powered by LLMs? 00:04:01.200 --> 00:04:03.439 And that way I could give just an instruction 00:04:03.440 --> 00:04:07.399 to regex replace. 00:04:07.400 --> 00:04:12.079 And so for example, I could do Emacs LLM regex replace. 00:04:12.080 --> 00:04:12.879 This is not checked in anywhere. 00:04:12.880 --> 00:04:17.199 These are just my own kind of private functions. 00:04:17.200 --> 00:04:19.239 My description lowercase all the org headings. 00:04:19.240 --> 00:04:20.439 Let's see if it works. 00:04:20.440 --> 00:04:21.039 It might work. 00:04:21.040 --> 00:04:22.959 No, it doesn't work. 00:04:22.960 --> 00:04:26.159 So if I, I'm not going to bother to show you 00:04:26.160 --> 00:04:28.159 what it actually came up with, but it's something, 00:04:28.160 --> 00:04:29.879 if you looked at it, it'd be like, wow, 00:04:29.880 --> 00:04:31.639 this is very close to being... 00:04:31.640 --> 00:04:34.239 It looks like it should work, but it doesn't. 00:04:34.240 --> 00:04:35.839 Okay. 00:04:35.840 --> 00:04:38.719 It's not quite good enough to get it right. 00:04:38.720 --> 00:04:41.599 And it's possible that perhaps by giving it 00:04:41.600 --> 00:04:43.639 a few examples of, or explaining more 00:04:43.640 --> 00:04:46.439 what makes Emacs regexes different. 00:04:46.440 --> 00:04:47.959 It could do a better job 00:04:47.960 --> 00:04:49.279 and maybe could solve these problems, 00:04:49.280 --> 00:04:50.679 but it's always a little bit random. 00:04:50.680 --> 00:04:52.359 You're never quite sure what you're going to get. 00:04:52.360 --> 00:04:54.839 So this is the drawback. 00:04:54.840 --> 00:04:58.479 Like there's a lot of things that look like you could do it, 00:04:58.480 --> 00:05:00.999 but when it actually comes down to trying it, 00:05:01.000 --> 00:05:03.399 it's surprisingly hard. 00:05:03.400 --> 00:05:06.319 And, you know, and whatever you're doing, 00:05:06.320 --> 00:05:08.999 it's surprisingly hard to get something 00:05:09.000 --> 00:05:13.879 that is repeatably, that's, that is always good. 00:05:13.880 --> 00:05:20.119 So yeah, that's currently the problem. NOTE Embeddings 00:05:20.120 --> 00:05:23.399 So I want to talk about embeddings. 00:05:23.400 --> 00:05:26.919 They're another thing that LLMs offer 00:05:26.920 --> 00:05:28.599 and that are extremely useful. 00:05:28.600 --> 00:05:33.119 They are, what they do is they encode from 00:05:33.120 --> 00:05:38.959 a input text that could be a word, a sentence, 00:05:38.960 --> 00:05:42.159 a small document. 00:05:42.160 --> 00:05:45.399 It encodes a vector about what the meaning, 00:05:45.400 --> 00:05:46.919 the semantic meaning of that is. 00:05:46.920 --> 00:05:51.079 That means you could, something that is, 00:05:51.080 --> 00:05:52.279 uses completely different words, 00:05:52.280 --> 00:05:54.159 but is basically talking about the same thing, 00:05:54.160 --> 00:05:57.839 perhaps in a different language, should be pretty close 00:05:57.840 --> 00:06:01.999 as a vector to the other vector. 00:06:02.000 --> 00:06:05.399 You know, as long as they're similarly semantic things, 00:06:05.400 --> 00:06:12.239 like the words 00:06:12.240 --> 00:06:18.959 highway and Camino are two different words. 00:06:18.960 --> 00:06:19.639 They mean the same thing. 00:06:19.640 --> 00:06:21.319 They should have very similar embeddings. 00:06:21.320 --> 00:06:25.119 So it is a way to kind of encode this 00:06:25.120 --> 00:06:26.199 and then you could use this for search. 00:06:26.200 --> 00:06:28.919 For example, I haven't tried to do this yet, 00:06:28.920 --> 00:06:31.479 but you could probably just make an embedding 00:06:31.480 --> 00:06:33.919 for every paragraph in the Emacs manual 00:06:33.920 --> 00:06:36.239 and the Elisp manual. 00:06:36.240 --> 00:06:39.439 And then, and then there's a very standard technique. 00:06:39.440 --> 00:06:43.439 You just... You find that you have a query, 00:06:43.440 --> 00:06:45.799 oh, how do I do whatever, whatever in Emacs again? 00:06:45.800 --> 00:06:49.479 And you could, you just find that 20 things 00:06:49.480 --> 00:06:50.319 that are closest to whatever you're 00:06:50.320 --> 00:06:51.839 trying to... the embedding of your query. 00:06:51.840 --> 00:06:55.279 You send those things to the LLM, as you know, 00:06:55.280 --> 00:06:57.799 with the original query, 00:06:57.800 --> 00:06:59.919 and you're basically telling the--asking the LLM, 00:06:59.920 --> 00:07:01.279 look, the user is trying to do this. 00:07:01.280 --> 00:07:03.039 Here's what I found in the Emacs manual. 00:07:03.040 --> 00:07:04.639 That's on the Elisp manual. 00:07:04.640 --> 00:07:07.439 That's close to what they're trying to do. 00:07:07.440 --> 00:07:12.159 So can you kind of just tell the user what to do? 00:07:12.160 --> 00:07:14.479 And from this, and you could say, 00:07:14.480 --> 00:07:17.639 just use things from this, you know, that I give you. 00:07:17.640 --> 00:07:20.679 Don't just make up your own idea. 00:07:20.680 --> 00:07:21.839 You know, don't use your own ideas, 00:07:21.840 --> 00:07:23.799 because sometimes it likes to do that 00:07:23.800 --> 00:07:24.359 and those things are wrong. 00:07:24.360 --> 00:07:26.719 So you could try to, you know, do this and you get, 00:07:26.720 --> 00:07:28.719 you could get quite good results using this. 00:07:28.720 --> 00:07:29.999 So no one has done this yet, 00:07:30.000 --> 00:07:32.799 but that should not be hard to do. NOTE Image Generation 00:07:32.800 --> 00:07:34.879 Image generation is something that's, you know, 00:07:34.880 --> 00:07:38.479 it's not quite an LLM in the sense of... 00:07:38.480 --> 00:07:43.079 These are... It's a different technology, 00:07:43.080 --> 00:07:48.439 but these things are kind of packaged together 00:07:48.440 --> 00:07:49.039 in a sense. 00:07:49.040 --> 00:07:51.639 And you'll see that when I talk about Emacs packages, 00:07:51.640 --> 00:07:54.279 a lot of them bundle image generation 00:07:54.280 --> 00:07:55.439 and large language models. 00:07:55.440 --> 00:07:59.039 You know, the APIs are often bundled together by providers. 00:07:59.040 --> 00:08:02.679 And the general idea is it's kind of similar 00:08:02.680 --> 00:08:04.399 because it's very similar to large, you know, 00:08:04.400 --> 00:08:06.559 doing a chat thing where you, you know, 00:08:06.560 --> 00:08:09.760 the chat is like, you give it a text request, 00:08:09.761 --> 00:08:12.759 like write me a sonnet about, you know, 00:08:12.760 --> 00:08:14.879 the battle between Emacs and vi. 00:08:14.880 --> 00:08:15.839 And it could, it could do it. 00:08:15.840 --> 00:08:17.159 It could do a very good job of that. 00:08:17.160 --> 00:08:22.519 But you could also say, you know, 00:08:22.520 --> 00:08:27.599 draw me a picture of Emacs and vi as boxers, 00:08:27.600 --> 00:08:30.359 as a character-character boxing in a ring, 00:08:30.360 --> 00:08:32.239 like a, you know, political cartoon style. 00:08:32.240 --> 00:08:34.999 And it can do that as well. 00:08:35.000 --> 00:08:37.679 And so you could basically think of this 00:08:37.680 --> 00:08:39.439 as just sort of... it's kind of the 00:08:39.440 --> 00:08:42.399 same thing with what you're doing 00:08:42.400 --> 00:08:43.359 with large language models, 00:08:43.360 --> 00:08:44.799 but instead of outputting a text, 00:08:44.800 --> 00:08:48.479 you're outputting a picture. NOTE Fine-tuning 00:08:48.480 --> 00:08:51.079 There's also, I want to mention the concept of fine-tuning. 00:08:51.080 --> 00:08:55.199 Fine-tuning is a way to take your-- 00:08:55.200 --> 00:08:59.759 take a corpus of inputs and outputs and just from 00:08:59.760 --> 00:09:01.599 a large language model, you're like, okay, 00:09:01.600 --> 00:09:03.599 given this base large language model, 00:09:03.600 --> 00:09:06.679 I want to make sure that when I give you input, 00:09:06.680 --> 00:09:08.479 you give me something like output. 00:09:08.480 --> 00:09:10.119 And this is what I'm just going to 00:09:10.120 --> 00:09:11.799 train you further on these, 00:09:11.800 --> 00:09:14.879 these mappings between input and output. 00:09:14.880 --> 00:09:16.399 And for example, you could do this. Like, 00:09:16.400 --> 00:09:18.039 let's say you wanted to fix that regex demo 00:09:18.040 --> 00:09:20.999 I had to make it good. 00:09:21.000 --> 00:09:23.479 I don't think it, I think it'd be 00:09:23.480 --> 00:09:25.039 relatively effective to train, 00:09:25.040 --> 00:09:27.039 to have regex descriptions 00:09:27.040 --> 00:09:30.119 and regex examples, Emacs regex examples 00:09:30.120 --> 00:09:31.239 as inputs and outputs. 00:09:31.240 --> 00:09:33.999 You could get, you know, maybe a hundred, 00:09:34.000 --> 00:09:35.359 a few hundreds of these things. 00:09:35.360 --> 00:09:38.639 You could train it. 00:09:38.640 --> 00:09:40.759 I think that is a reasonable way to, 00:09:40.760 --> 00:09:43.879 let's just say, I don't know how well it would work, 00:09:43.880 --> 00:09:46.839 but these things definitely work some of the time 00:09:46.840 --> 00:09:47.999 and produce pretty good results. 00:09:48.000 --> 00:09:53.039 And you could do this on your own machine. 00:09:53.040 --> 00:09:58.999 Corporations like OpenAI offer APIs with, you know, 00:09:59.000 --> 00:10:01.519 to build your fine tunes on top of OpenAI. 00:10:01.520 --> 00:10:04.159 And I think, I'm not a hundred percent sure, 00:10:04.160 --> 00:10:05.719 but I think then you can share your model 00:10:05.720 --> 00:10:06.519 with other people. 00:10:06.520 --> 00:10:08.519 But if not, then you just, you know, 00:10:08.520 --> 00:10:10.839 you could use your model for your own specialized purposes. 00:10:10.840 --> 00:10:14.039 But in the world of models that you could run, 00:10:14.040 --> 00:10:16.874 for example, based on Llama, which is like... 00:10:16.875 --> 00:10:22.240 Llama is this model you can run on your own machine from Meta. 00:10:23.580 --> 00:10:26.880 There's many fine-tuned models that you could download 00:10:26.881 --> 00:10:28.960 and you could run on your own. 00:10:28.961 --> 00:10:30.839 They can do very different things too. 00:10:30.840 --> 00:10:33.399 Some output Python programs, for example, 00:10:33.400 --> 00:10:34.279 that you could just run. 00:10:34.280 --> 00:10:37.959 So you just say... 00:10:37.960 --> 00:10:40.639 Tell me how old... Let's just say 00:10:40.640 --> 00:10:41.999 you have a random task, like 00:10:42.000 --> 00:10:48.119 tell me how old these five cities are in minutes, 00:10:48.120 --> 00:10:49.799 based on historical evidence. 00:10:49.800 --> 00:10:53.639 It's kind of a weird query, but it probably can figure, 00:10:53.640 --> 00:10:55.119 it could probably run that for you. 00:10:55.120 --> 00:10:57.239 It'll encode its knowledge into whatever 00:10:57.240 --> 00:10:59.599 the Python program, then use the Python program 00:10:59.600 --> 00:11:01.039 to do the correct calculations. 00:11:01.040 --> 00:11:05.679 So pretty, pretty useful stuff. NOTE Open Source 00:11:08.160 --> 00:11:10.399 So I also want to mention open source 00:11:10.400 --> 00:11:12.679 and basically free software here. 00:11:12.680 --> 00:11:17.599 These LLMs are mostly not free software. 00:11:17.600 --> 00:11:19.159 They're sometimes open source, 00:11:19.160 --> 00:11:21.959 but they're generally not free 00:11:21.960 --> 00:11:23.799 without restrictions to use. 00:11:23.800 --> 00:11:27.279 Most of these things, even Llama, 00:11:27.280 --> 00:11:28.679 which you can use on your own machine, 00:11:28.680 --> 00:11:31.439 have restrictions that you cannot use it 00:11:31.440 --> 00:11:32.519 to train your own model. 00:11:32.520 --> 00:11:35.119 This is something that, you know, 00:11:35.120 --> 00:11:37.519 it costs millions and millions of dollars 00:11:37.520 --> 00:11:40.759 to train and produce these models. 00:11:40.760 --> 00:11:42.319 And that's just computation costs. 00:11:42.320 --> 00:11:45.519 They do not want you 00:11:45.520 --> 00:11:47.839 stealing all that work by training your own models 00:11:47.840 --> 00:11:48.799 based on their output. 00:11:48.800 --> 00:11:55.359 But there are research LLMs that do, I believe, 00:11:55.360 --> 00:11:57.999 conform to free software principles. 00:11:58.000 --> 00:11:59.519 They're just not as good yet. 00:11:59.520 --> 00:12:02.519 And I think that might change in the future. NOTE The Future 00:12:02.840 --> 00:12:04.119 So speaking of the future, 00:12:04.120 --> 00:12:07.519 one of the things I'd like to point out 00:12:07.520 --> 00:12:09.639 is that like the demos I showed you are based on, 00:12:09.640 --> 00:12:13.519 I'm using OpenAI 3.5 model. 00:12:13.520 --> 00:12:16.439 That's more than, well, no, 00:12:16.440 --> 00:12:18.199 it's like a year old basically at this point. 00:12:18.200 --> 00:12:21.079 And things are moving fast. 00:12:21.080 --> 00:12:22.039 They came out with 4.0. 00:12:22.040 --> 00:12:23.319 4.0 is significantly better. 00:12:23.320 --> 00:12:24.319 I don't have access to it. 00:12:24.320 --> 00:12:30.839 Even though I'm using the API and I'm paying money for it, 00:12:30.840 --> 00:12:33.639 you only can get access to 4.0 00:12:33.640 --> 00:12:34.439 if you can spend a dollar. 00:12:34.440 --> 00:12:36.319 And I've never been able to spend, 00:12:36.320 --> 00:12:38.199 use so much API use that I've spent a dollar. 00:12:38.200 --> 00:12:44.479 So I have, I don't have 4.0, but I've tried it 00:12:44.480 --> 00:12:46.639 because I do pay for this 00:12:46.640 --> 00:12:48.340 so I could get access to 4.0 00:12:48.341 --> 00:12:49.599 and it is substantially better. 00:12:49.600 --> 00:12:50.519 By all reports, it's, 00:12:50.520 --> 00:12:53.839 the difference is extremely significant. 00:12:53.840 --> 00:12:55.159 I would not be surprised 00:12:55.160 --> 00:12:59.759 if some of the limitations and drawbacks I described 00:12:59.760 --> 00:13:02.039 mostly went away with 4.0. 00:13:02.040 --> 00:13:06.679 We're probably at a stage 00:13:06.680 --> 00:13:09.239 where regexes will work maybe 5% of the time 00:13:09.240 --> 00:13:10.119 if you try them. 00:13:10.120 --> 00:13:13.639 But with 4.0, it could work like 80% of the time. 00:13:13.640 --> 00:13:14.559 Now, is that good enough? 00:13:14.560 --> 00:13:17.279 Probably not, but it's a, 00:13:17.280 --> 00:13:20.319 I wouldn't be surprised if you got results like that. 00:13:20.320 --> 00:13:22.919 And in a year's time, in two years time, 00:13:22.920 --> 00:13:26.679 no one knows how much this is going to play out 00:13:26.680 --> 00:13:27.519 before progress stalls, 00:13:27.520 --> 00:13:32.319 but there are a lot of interesting research. 00:13:32.320 --> 00:13:34.279 I don't think, research wise, 00:13:34.280 --> 00:13:35.759 I don't think things have slowed down. 00:13:35.760 --> 00:13:38.719 You're still seeing a lot of advances. 00:13:38.720 --> 00:13:40.999 You're still seeing a lot of models coming out 00:13:41.000 --> 00:13:41.839 and that will come out. 00:13:41.840 --> 00:13:46.279 That will be each one, one upping the other one 00:13:46.280 --> 00:13:49.959 in terms of quality. 00:13:49.960 --> 00:13:52.759 It'll be really interesting to see how this all plays out. 00:13:52.760 --> 00:13:55.919 I think that message here is that 00:13:55.920 --> 00:13:57.999 we're at the beginning here. 00:13:58.000 --> 00:14:01.239 This is why I think this talk is important. 00:14:01.240 --> 00:14:02.279 I think this is why we should be 00:14:02.280 --> 00:14:04.159 paying attention to this stuff. NOTE LLMs in Emacs - existing packages 00:14:08.200 --> 00:14:11.039 Let's talk about the existing packages. 00:14:11.040 --> 00:14:13.199 Because there's a lot out there, people have, 00:14:13.200 --> 00:14:17.039 I think people have been integrating with 00:14:17.040 --> 00:14:21.239 these LLMs that often have a relatively easy to use API. 00:14:21.240 --> 00:14:24.039 So it's kind of natural that people 00:14:24.040 --> 00:14:25.679 have already put out a lot of packages. 00:14:25.680 --> 00:14:28.319 Coming off this problem from a lot of different angles, 00:14:28.320 --> 00:14:30.639 I don't have time to go through 00:14:30.640 --> 00:14:31.959 all of these packages. 00:14:31.960 --> 00:14:33.559 These are great packages though. 00:14:33.560 --> 00:14:35.279 If you're not familiar with them, 00:14:35.280 --> 00:14:37.679 please check them out. 00:14:37.680 --> 00:14:40.999 And they all are doing slightly different things. 00:14:41.000 --> 00:14:43.959 Some of these are relatively straightforward. 00:14:43.960 --> 00:14:47.919 Interactions, just a way to 00:14:47.920 --> 00:14:52.679 almost in a comment sort of way to kind of 00:14:52.680 --> 00:14:54.199 have just an interaction, 00:14:54.200 --> 00:14:55.479 long running interaction with an LLM 00:14:55.480 --> 00:14:59.039 where you kind of build off previous responses, 00:14:59.040 --> 00:15:01.799 kind of like the OpenAI's UI. 00:15:01.800 --> 00:15:08.559 Two very more Emacsy things where you can sort of 00:15:08.560 --> 00:15:13.679 embed these LLM responses within a org-mode block 00:15:13.680 --> 00:15:15.239 using the org-mode's context. 00:15:15.240 --> 00:15:20.959 Or GitHub Copilot integration where you can use it 00:15:20.960 --> 00:15:23.319 for auto completion in a very powerful, 00:15:23.320 --> 00:15:27.319 you know, this stuff is very useful if it could figure out 00:15:27.320 --> 00:15:29.199 what you're trying to do based on the context. 00:15:29.200 --> 00:15:31.839 It's quite effective. 00:15:31.840 --> 00:15:36.359 But I want to kind of call out one thing 00:15:36.360 --> 00:15:38.239 that I'd like to see change. 00:15:38.240 --> 00:15:42.599 Which is that users right now, 00:15:42.600 --> 00:15:45.199 not all of these have a choice of, 00:15:45.200 --> 00:15:47.959 first of all, there's a lot of them. 00:15:47.960 --> 00:15:49.639 Each one of them is doing their own calls. 00:15:49.640 --> 00:15:53.999 And each one of them is, so each one of them 00:15:54.000 --> 00:15:55.239 has their own interfaces. 00:15:55.240 --> 00:15:57.719 They're rewriting the interface to OpenAI or wherever. 00:15:57.720 --> 00:16:00.119 And they're not, they don't, most of these 00:16:00.120 --> 00:16:05.119 do not make it that configurable or at all configurable 00:16:05.120 --> 00:16:06.599 what LLM use. 00:16:06.600 --> 00:16:07.239 This is not good. 00:16:07.240 --> 00:16:09.679 It is important that we use, 00:16:09.680 --> 00:16:15.679 we give the user a way to change the LLM they use. 00:16:15.680 --> 00:16:21.079 And that is because you might not be comfortable 00:16:21.080 --> 00:16:24.439 sending your requests over to a private corporation 00:16:24.440 --> 00:16:27.799 where you don't get to see how they use their data. 00:16:27.800 --> 00:16:29.799 Your data, really. 00:16:29.800 --> 00:16:33.319 That's especially true with things like embeddings 00:16:33.320 --> 00:16:35.039 where you might be sending over your documents. 00:16:35.040 --> 00:16:37.519 You're just giving them your documents, basically. 00:16:37.520 --> 00:16:40.759 And, you know, that does happen. 00:16:40.760 --> 00:16:43.599 I don't think really that there's a reason 00:16:43.600 --> 00:16:44.639 to be uncomfortable with this, 00:16:44.640 --> 00:16:51.439 but that, you know, people are uncomfortable and that's okay. 00:16:51.440 --> 00:16:53.239 People might want to use a local machine, 00:16:53.240 --> 00:16:58.359 a local LLM for maximum privacy. 00:16:58.360 --> 00:17:00.639 That's something we should allow. 00:17:00.640 --> 00:17:04.519 People might want to especially use free software. 00:17:04.520 --> 00:17:05.839 That's something we should definitely allow. 00:17:05.840 --> 00:17:07.279 This is Emacs. 00:17:07.280 --> 00:17:08.239 We need to encourage that. 00:17:08.240 --> 00:17:12.159 But right now, as most of these things are written, 00:17:12.160 --> 00:17:13.959 you can't do it. 00:17:13.960 --> 00:17:17.839 And they're spending precious time 00:17:17.840 --> 00:17:18.879 just doing things themselves. 00:17:18.880 --> 00:17:20.839 This is why I wrote LLM, which is... 00:17:20.840 --> 00:17:23.039 it will just make that connection to the LLM for you 00:17:23.040 --> 00:17:26.719 and it will connect to, you know, it has plugins. 00:17:26.720 --> 00:17:30.279 So if you can, the user can configure what plugin 00:17:30.280 --> 00:17:31.359 it actually goes to. 00:17:31.360 --> 00:17:32.399 Does it go to OpenAI? 00:17:32.400 --> 00:17:35.239 Does it go to Google Cloud Vertex? 00:17:35.240 --> 00:17:36.999 Does it go to Llama on your machine? 00:17:37.000 --> 00:17:38.399 We're using Ollama, 00:17:38.400 --> 00:17:40.999 which is just a way to run Llama locally. 00:17:41.000 --> 00:17:47.959 And more things in the future, I hope. 00:17:47.960 --> 00:17:52.079 So this is, I'm hoping that we use this. 00:17:52.080 --> 00:17:54.839 It's designed to be sort of maximally usable. 00:17:54.840 --> 00:17:56.279 You don't need to install anything. 00:17:56.280 --> 00:17:58.359 It's on GNU ELPA. 00:17:58.360 --> 00:17:59.879 So even if you write something 00:17:59.880 --> 00:18:01.079 that you want to contribute to GNU ELPA, 00:18:01.080 --> 00:18:02.879 you can use it because it's on GNU ELPA. 00:18:02.880 --> 00:18:06.439 It's part of the Emacs package, Emacs core packages. 00:18:06.440 --> 00:18:09.879 So, but it has no functionality. 00:18:09.880 --> 00:18:11.719 It's really just there as a library 00:18:11.720 --> 00:18:14.439 to use by other things offering functionality. Okay. NOTE Abstracting LLM challenges 00:18:15.960 --> 00:18:19.839 And it's a little bit difficult to abstract. 00:18:19.840 --> 00:18:21.159 I want to point this out 00:18:21.160 --> 00:18:23.599 because I think it's an important point 00:18:23.600 --> 00:18:29.519 is that the, it's, some of these LLMs, for example, 00:18:29.520 --> 00:18:30.439 have image generation. 00:18:30.440 --> 00:18:31.279 Some do not. 00:18:31.280 --> 00:18:35.319 Some of them have very large context windows, even for chat. 00:18:35.320 --> 00:18:36.999 You say, okay, all these things can do chat. 00:18:37.000 --> 00:18:37.319 Okay. 00:18:37.320 --> 00:18:38.079 Yeah, kind of. 00:18:38.080 --> 00:18:39.999 Some of these things you could pass a book to, 00:18:40.000 --> 00:18:41.239 like Anthropic's API. 00:18:41.240 --> 00:18:43.039 Most, you cannot. 00:18:43.040 --> 00:18:45.559 So there really are big differences 00:18:45.560 --> 00:18:46.399 in how these things work. 00:18:46.400 --> 00:18:51.539 I hope those differences diminish in the future. 00:18:51.540 --> 00:18:53.800 But it's just one of the challenges 00:18:53.801 --> 00:18:57.520 that I hope we can work through in the LLM library. 00:18:57.521 --> 00:19:02.160 So it's compatible, but there's definitely 00:19:02.161 --> 00:19:04.079 limits to that compatibility. NOTE Emacs is the ideal interface for LLMs 00:19:04.080 --> 00:19:06.160 I want to point out just to finish off, 00:19:06.161 --> 00:19:12.879 Emacs is the, Emacs has real power here 00:19:12.880 --> 00:19:15.679 that nothing else I think in the industry is offering. 00:19:15.680 --> 00:19:19.279 First of all, people that use Emacs 00:19:19.280 --> 00:19:20.439 tend to do a lot of things in Emacs. 00:19:20.440 --> 00:19:22.159 We have our to-dos in Emacs with the org mode. 00:19:22.160 --> 00:19:22.999 We have mail. 00:19:23.000 --> 00:19:25.719 We, you know, we might read email and we might, 00:19:25.720 --> 00:19:27.679 and respond to email in Emacs. 00:19:27.680 --> 00:19:29.199 We might have notes in Emacs. 00:19:29.200 --> 00:19:31.359 This is very powerful. 00:19:31.360 --> 00:19:34.159 Using... there's not other stuff like that. 00:19:34.160 --> 00:19:35.759 And you could feed this stuff to an LLM. 00:19:35.760 --> 00:19:37.039 You could do interesting things 00:19:37.040 --> 00:19:38.559 using a combination of all this data. 00:19:38.560 --> 00:19:40.399 No one else could do this. 00:19:40.400 --> 00:19:41.759 We need to start thinking about it. 00:19:41.760 --> 00:19:45.039 Secondly, Emacs can execute commands. 00:19:45.040 --> 00:19:46.239 This might be a bad idea. 00:19:46.240 --> 00:19:48.399 This might be how the robots take over, 00:19:48.400 --> 00:19:51.799 but you could have the LLMs respond with Emacs 00:19:51.800 --> 00:19:54.199 commands and run those Emacs commands 00:19:54.200 --> 00:19:57.079 and tell the LLM the response and have it do things 00:19:57.080 --> 00:19:58.679 as your agent in the editor. 00:19:58.680 --> 00:20:01.599 I think we need to explore ideas like this. NOTE Outro 00:20:01.960 --> 00:20:04.279 And I think we need to share these ideas 00:20:04.280 --> 00:20:07.039 and we need to make sure that we're pushing the 00:20:07.040 --> 00:20:10.519 envelope for Emacs and actually, you know, doing things, 00:20:10.520 --> 00:20:12.959 sharing ideas, sharing progress, 00:20:12.960 --> 00:20:15.199 and kind of seeing how far we can push this stuff. 00:20:15.200 --> 00:20:20.639 Let's really help Emacs out, be sort of, 00:20:20.640 --> 00:20:24.519 take advantage of this super powerful technique. 00:20:24.520 --> 00:20:26.160 Thank you for listening.