diff options
Diffstat (limited to '2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--answers.vtt')
| -rw-r--r-- | 2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--answers.vtt | 1910 |
1 files changed, 1910 insertions, 0 deletions
diff --git a/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--answers.vtt b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--answers.vtt new file mode 100644 index 00000000..d48cc06d --- /dev/null +++ b/2023/captions/emacsconf-2023-llm--llm-clients-in-emacs-functionality-and-standardization--andrew-hyatt--answers.vtt @@ -0,0 +1,1910 @@ +WEBVTT + + +00:00:13.099 --> 00:00:13.599 +[Speaker 0]: Okay. Hello, everyone. + +00:00:16.560 --> 00:00:17.060 +I think this is the start of the Q&A session. + +00:00:25.119 --> 00:00:25.599 +So people can just ask me questions here. + +00:00:28.259 --> 00:00:28.380 +Or I think maybe these questions are going to + +00:00:30.560 --> 00:00:30.980 +be read by someone. Yes, + +00:00:34.680 --> 00:00:35.060 +thank you. Should I start doing that? + +00:00:39.280 --> 00:00:39.400 +I also know that there's questions in the + +00:00:41.320 --> 00:00:41.480 +either pad room, so I could start out + +00:00:42.280 --> 00:00:42.780 +answering those as well. + +00:00:45.020 --> 00:00:45.300 +[Speaker 1]: Right, sure. Whichever way you prefer. + +00:00:46.860 --> 00:00:47.220 +If you prefer to read the questions yourself, + +00:00:48.940 --> 00:00:49.080 +by all means, or if you would prefer me to + +00:00:50.080 --> 00:00:50.280 +read them to you, that also works. + +00:00:50.580 --> 00:00:50.920 +[Speaker 2]: Oh, I see. + +00:00:51.600 --> 00:00:51.760 +[Speaker 0]: Why don't you read them to me? + +00:00:53.260 --> 00:00:53.760 +I think it'll just be more interesting then. + +00:00:56.820 --> 00:00:57.160 +[Speaker 1]: Sure. OK, let's see. The first question is, + +00:00:58.360 --> 00:00:58.860 +what is your use case for embedding, + +00:01:00.060 --> 00:01:00.560 +mainly for searching? + +00:01:06.180 --> 00:01:06.340 +[Speaker 0]: Yeah, I mean, I think the use case really is + +00:01:12.100 --> 00:01:12.320 +searching. And I think it is very useful when + +00:01:15.060 --> 00:01:15.320 +you're searching for something in a vague + +00:01:18.280 --> 00:01:18.780 +way. Just to give you an example, + +00:01:23.860 --> 00:01:24.360 +I have a note system called EKG. + +00:01:25.760 --> 00:01:26.260 +I type all my notes on it. + +00:01:28.620 --> 00:01:29.120 +You can find it on GitHub and Melba. + +00:01:34.140 --> 00:01:34.400 +But I wrote something at some point a year + +00:01:35.840 --> 00:01:36.020 +ago or something. I wrote something that I + +00:01:36.600 --> 00:01:36.980 +just vaguely remembered. + +00:01:38.800 --> 00:01:38.940 +Oh, this was about a certain kind of + +00:01:41.580 --> 00:01:41.760 +communication. I wanted communicating to + +00:01:43.280 --> 00:01:43.479 +large audiences. There's some interesting tip + +00:01:44.700 --> 00:01:45.060 +that I wrote down that was really cool. + +00:01:49.920 --> 00:01:50.080 +And I was like, well, I need to find it. + +00:01:52.260 --> 00:01:52.640 +So I did an embedding search for something + +00:01:55.479 --> 00:01:55.979 +like, you know, tips for communicating. + +00:01:58.979 --> 00:01:59.100 +Like those words may not have been in what I + +00:02:00.020 --> 00:02:00.520 +was trying to find at all, + +00:02:02.680 --> 00:02:03.180 +But it was able to find it. + +00:02:05.840 --> 00:02:06.260 +And that is something that's very hard to do + +00:02:07.200 --> 00:02:07.360 +in other ways. Like, you know, + +00:02:08.520 --> 00:02:08.720 +if you had to do this with normal search, + +00:02:09.199 --> 00:02:09.660 +you have to do synonyms. + +00:02:10.940 --> 00:02:11.200 +And like maybe those synonyms wouldn't cover + +00:02:11.960 --> 00:02:12.340 +it. Like with embedding, + +00:02:13.940 --> 00:02:14.160 +you can basically get at like the vague + +00:02:14.960 --> 00:02:15.460 +sentiment. You're like, + +00:02:17.320 --> 00:02:17.560 +you know, you're, you know, + +00:02:19.520 --> 00:02:19.700 +you can really query on like what things are + +00:02:21.760 --> 00:02:22.260 +about as opposed to what words they have. + +00:02:25.600 --> 00:02:26.100 +Also, it's super good for similarity search. + +00:02:27.720 --> 00:02:27.900 +So you could say, look, + +00:02:30.040 --> 00:02:30.240 +I have a bunch of things that are encoded + +00:02:31.400 --> 00:02:31.900 +with embeddings that I want to show. + +00:02:34.120 --> 00:02:34.280 +For example, you can make an embedding for + +00:02:35.220 --> 00:02:35.380 +every buffer. You'd be like, + +00:02:37.060 --> 00:02:37.200 +well, show me buffers that are similar to + +00:02:38.740 --> 00:02:38.980 +this buffer. That doesn't sound super useful, + +00:02:40.440 --> 00:02:40.940 +but this is the kind of thing you could do. + +00:02:45.300 --> 00:02:45.480 +And so if you have a bunch of notes or + +00:02:46.720 --> 00:02:46.920 +something else that you want to search on, + +00:02:48.240 --> 00:02:48.740 +you'd be like, what's similar to this buffer? + +00:02:51.500 --> 00:02:51.760 +Or what notes are similar to each other? + +00:02:53.040 --> 00:02:53.540 +What buffers are similar to each other? + +00:02:55.380 --> 00:02:55.880 +It's super good for this sort of thing. + +00:03:00.780 --> 00:03:01.280 +And it's also good for this kind of retrieval + +00:03:03.080 --> 00:03:03.520 +augmented generation, where you sort of, + +00:03:05.080 --> 00:03:05.280 +you retrieve things and the purpose is not + +00:03:06.600 --> 00:03:06.880 +for you to see them, but then you pass that + +00:03:12.040 --> 00:03:12.180 +to the LLM. And then it's able to be a little + +00:03:14.340 --> 00:03:14.800 +bit more accurate because it has the actual + +00:03:15.760 --> 00:03:16.260 +text that you're trying to, + +00:03:18.960 --> 00:03:19.180 +that is relevant, and it can cite from and + +00:03:20.720 --> 00:03:20.820 +things like that. And then it could give you + +00:03:22.260 --> 00:03:22.660 +a much better answer that's kind of, + +00:03:25.520 --> 00:03:25.680 +you know, not just from its own little neural + +00:03:26.320 --> 00:03:26.820 +nets and memory. + +00:03:31.920 --> 00:03:32.120 +[Speaker 1]: Cool, thanks. Let's see, + +00:03:35.740 --> 00:03:36.100 +next question. What do you think about embed + +00:03:40.160 --> 00:03:40.660 +Emacs manual versus GPT's Emacs manual? + +00:03:45.480 --> 00:03:45.660 +[Speaker 0]: I'm not exactly sure what this question is + +00:03:46.980 --> 00:03:47.480 +trying to say. So I mean, + +00:03:51.000 --> 00:03:51.140 +if someone wrote that and wants to expand on + +00:03:55.080 --> 00:03:55.320 +it a little bit, but I think that maybe + +00:03:58.420 --> 00:03:58.920 +you're saying like you could embed, + +00:04:00.280 --> 00:04:00.780 +have embeddings for like various, + +00:04:02.520 --> 00:04:02.720 +like every paragraph or something of the + +00:04:04.540 --> 00:04:04.680 +Emacs manual. But it's also the case that + +00:04:06.500 --> 00:04:07.000 +like GPT is already for sure already read it, + +00:04:09.760 --> 00:04:09.960 +right? And so you could ask questions that + +00:04:13.460 --> 00:04:13.780 +are about Emacs and our ELISP or whatever + +00:04:15.200 --> 00:04:15.700 +part of the manual you want to find. + +00:04:19.760 --> 00:04:20.240 +And it will do a reasonably good job, + +00:04:22.280 --> 00:04:22.780 +especially the better models will do a + +00:04:24.620 --> 00:04:24.940 +reasonably good job of saying you something + +00:04:26.040 --> 00:04:26.540 +that is vaguely accurate. + +00:04:29.440 --> 00:04:29.860 +But if you do this retrieval augmented + +00:04:30.580 --> 00:04:31.080 +generation with embeddings, + +00:04:32.640 --> 00:04:33.140 +you can get something that is very accurate. + +00:04:36.700 --> 00:04:36.960 +At least I think. I haven't tried it, + +00:04:38.760 --> 00:04:39.020 +but this is a technique that works in other + +00:04:43.040 --> 00:04:43.260 +similar cases. So you can also imagine like, + +00:04:44.320 --> 00:04:44.500 +oh, this whole thing I said, + +00:04:47.860 --> 00:04:48.120 +like, oh, you can query for vague things and + +00:04:49.140 --> 00:04:49.600 +get parts of the manual, + +00:04:52.680 --> 00:04:53.000 +perhaps. I'm not exactly sure if that would + +00:04:55.120 --> 00:04:55.520 +be useful, but maybe. Usually when I'm + +00:04:57.040 --> 00:04:57.180 +looking things up in the Emacs manual or + +00:04:58.320 --> 00:04:58.780 +Elist manual, I have something extremely + +00:05:00.020 --> 00:05:00.300 +specific and I kind of know where to look. + +00:05:02.960 --> 00:05:03.080 +But having other ways to get at this + +00:05:04.000 --> 00:05:04.500 +information is always good. + +00:05:10.240 --> 00:05:10.740 +[Speaker 1]: Right. Looks like they added a clarification + +00:05:12.280 --> 00:05:12.720 +if you would like to read that yourself, + +00:05:14.180 --> 00:05:14.680 +or would you like me to read it for you? + +00:05:17.640 --> 00:05:18.140 +[Speaker 0]: Yeah. Yes, OK. It says, + +00:05:20.460 --> 00:05:20.740 +I've never tried. Yeah, + +00:05:21.500 --> 00:05:21.820 +the question is like OK, + +00:05:23.100 --> 00:05:23.240 +there is a difference between the kind of + +00:05:23.860 --> 00:05:24.360 +thing as I just described. + +00:05:26.200 --> 00:05:26.600 +I have not tried the difference with the EMAX + +00:05:31.560 --> 00:05:31.980 +manual itself. It'd be interesting to see + +00:05:33.700 --> 00:05:33.960 +what this is, but I would expect like these + +00:05:35.140 --> 00:05:35.600 +techniques, the retrieval augmented + +00:05:38.840 --> 00:05:39.340 +generation is generally pretty good. + +00:05:41.240 --> 00:05:41.740 +And I suspect it would, + +00:05:43.580 --> 00:05:43.780 +I would bet money on the fact that it's gonna + +00:05:45.820 --> 00:05:46.240 +give you, you know, better results than just, + +00:05:48.160 --> 00:05:48.360 +you know, doing a free form query without any + +00:05:49.440 --> 00:05:49.940 +retrieval augmented generation. + +00:05:54.240 --> 00:05:54.640 +[Speaker 1]: Cool. Let's see. Next question. + +00:05:56.380 --> 00:05:56.880 +When deferring commit messages to an LLM, + +00:05:59.700 --> 00:05:59.920 +what, if anything, do you find you might have + +00:06:02.940 --> 00:06:03.440 +[Speaker 0]: lost? Yeah, it's a good question. + +00:06:06.060 --> 00:06:06.560 +When deferring anything to a computer, + +00:06:08.860 --> 00:06:09.360 +like, you know, I used to have to remember + +00:06:11.200 --> 00:06:11.700 +how to get places, and now, + +00:06:14.540 --> 00:06:15.040 +you know, on the few occasions which I drive, + +00:06:16.560 --> 00:06:16.720 +like, It could just tell me how to get + +00:06:21.960 --> 00:06:22.280 +places. So similar things could occur here + +00:06:24.960 --> 00:06:25.460 +where like, okay, I'm just leaving the LLM. + +00:06:27.380 --> 00:06:27.680 +And so I'm kind of missing out on some + +00:06:30.040 --> 00:06:30.220 +opportunity to think coherently about a + +00:06:32.440 --> 00:06:32.680 +particular commit. Particular commits are + +00:06:36.140 --> 00:06:36.540 +kind of low level. I don't think it's usually + +00:06:39.340 --> 00:06:39.840 +relatively obvious and what they're doing. + +00:06:42.600 --> 00:06:42.800 +And in this case, I think there's not much + +00:06:44.220 --> 00:06:44.540 +loss. But for sure, in other cases, + +00:06:46.400 --> 00:06:46.900 +if you're starting to get into situations + +00:06:48.640 --> 00:06:48.800 +where it's writing your emails and all this + +00:06:52.920 --> 00:06:53.300 +stuff. First of all, it's in 1 sense, + +00:06:55.580 --> 00:06:56.040 +I'm not sure you might be losing something by + +00:06:57.520 --> 00:06:57.780 +delegating things. On the other hand, + +00:06:59.120 --> 00:06:59.280 +you know, when you're interacting with these + +00:07:01.280 --> 00:07:01.560 +LLMs, you have to be extremely specific about + +00:07:03.120 --> 00:07:03.240 +what you want, or else it's just not going to + +00:07:07.540 --> 00:07:07.680 +do a good job. And that might actually be a + +00:07:09.440 --> 00:07:09.860 +good thing. So the question might be that + +00:07:11.820 --> 00:07:12.240 +maybe you might gain things by using an LLM + +00:07:13.860 --> 00:07:14.060 +to do your work. It might not actually even + +00:07:15.060 --> 00:07:15.400 +save you that much time, + +00:07:18.480 --> 00:07:18.640 +at least initially, because you have to kind + +00:07:20.460 --> 00:07:20.660 +of practice again super specific about what + +00:07:22.740 --> 00:07:22.900 +you want to get out of the output it's going + +00:07:26.940 --> 00:07:26.980 +to give you so like oh I'm you know maybe you + +00:07:29.600 --> 00:07:29.820 +know you're on the emacs devel mailing list + +00:07:31.780 --> 00:07:31.980 +and you're like okay write this email about + +00:07:34.000 --> 00:07:34.140 +this about this And here's what I want to + +00:07:35.370 --> 00:07:35.460 +say. And here's the kind of tone I want to + +00:07:36.020 --> 00:07:36.420 +use. And here's the like, + +00:07:37.660 --> 00:07:38.160 +oh, you might want to specify like everything + +00:07:39.620 --> 00:07:40.120 +that you kind of want to get into this. + +00:07:42.180 --> 00:07:42.680 +Usually it's easier just to write the email. + +00:07:45.600 --> 00:07:46.100 +But I think that practice of kind of + +00:07:48.080 --> 00:07:48.420 +understanding what you want is not something + +00:07:52.680 --> 00:07:53.180 +you normally do. And I think it's going to be + +00:07:56.040 --> 00:07:56.480 +an interesting exercise that will help people + +00:07:57.280 --> 00:07:57.540 +understand. That said, + +00:07:58.860 --> 00:07:59.020 +I haven't done that much of that, + +00:07:59.900 --> 00:08:00.040 +so I can't say, oh, yeah, + +00:08:01.080 --> 00:08:01.440 +I've done this and it works for me. + +00:08:03.000 --> 00:08:03.120 +Maybe. I think it's an interesting thing to + +00:08:03.120 --> 00:08:03.620 +explore. + +00:08:07.720 --> 00:08:08.220 +[Speaker 1]: Sure. Thanks. Let's see. + +00:08:10.840 --> 00:08:11.140 +Let's see. Can you share your font settings + +00:08:13.440 --> 00:08:13.740 +in your Emacs config? Those are some nice + +00:08:14.200 --> 00:08:14.700 +fonts for reading. + +00:08:18.900 --> 00:08:19.200 +[Speaker 0]: Yeah, I think I was using Menlo at the time. + +00:08:20.840 --> 00:08:20.980 +Unfortunately, I don't save those kinds of + +00:08:21.940 --> 00:08:22.200 +things, like a history of this. + +00:08:24.000 --> 00:08:24.500 +I've kind of switched now to, + +00:08:27.340 --> 00:08:27.540 +what was that? I think I wrote it down in + +00:08:29.440 --> 00:08:29.940 +the, I switched to MunaSpace, + +00:08:31.920 --> 00:08:32.420 +which just came out like a week or 2 ago, + +00:08:33.340 --> 00:08:33.840 +and is also pretty cool. + +00:08:35.440 --> 00:08:35.940 +So I think it's Menlo. + +00:08:37.380 --> 00:08:37.760 +The internal question, + +00:08:38.400 --> 00:08:38.900 +what font are you using? + +00:08:42.020 --> 00:08:42.340 +[Speaker 1]: Indeed, yeah. It looks like someone guessed + +00:08:43.780 --> 00:08:44.280 +as well that it might be Menlo. + +00:08:47.680 --> 00:08:48.040 +OK, Cool. Yeah, next question. + +00:08:48.900 --> 00:08:49.400 +In terms of standardization, + +00:08:53.260 --> 00:08:53.520 +do you see a need for the medium to large + +00:08:55.840 --> 00:08:56.160 +scale effort needed? And then they also + +00:08:56.960 --> 00:08:57.460 +elaborate about it. + +00:09:03.600 --> 00:09:04.100 +[Speaker 0]: Yeah, I mean, I do think, + +00:09:06.040 --> 00:09:06.300 +I don't know if it's large scale, + +00:09:08.000 --> 00:09:08.500 +but at least it's probably medium scale. + +00:09:10.320 --> 00:09:10.520 +There's a lot of things that are missing that + +00:09:12.260 --> 00:09:12.400 +we don't have right now in emacs when you're + +00:09:13.660 --> 00:09:14.160 +dealing with LLMs. 1 is, + +00:09:18.240 --> 00:09:18.420 +a prompting system. And by that, + +00:09:21.820 --> 00:09:22.020 +I mean, you know, prompts are just like big + +00:09:24.520 --> 00:09:24.720 +blocks of text, but there's also senses that + +00:09:28.260 --> 00:09:28.420 +like prompts need to be composable and you + +00:09:30.480 --> 00:09:30.660 +need to be able to iterate on parts of the + +00:09:36.600 --> 00:09:37.100 +prompt. And so it's also customizable. + +00:09:38.940 --> 00:09:39.060 +Users might want to customize it. + +00:09:41.260 --> 00:09:41.360 +On the other hand, it's not super easy to + +00:09:43.820 --> 00:09:44.320 +write the prompt. So you want really good + +00:09:47.900 --> 00:09:48.040 +defaults. So the whole prompt system is kind + +00:09:51.360 --> 00:09:51.460 +of complicated. That needs to be kind of + +00:09:52.580 --> 00:09:52.760 +standardized, because I don't think there's + +00:09:54.380 --> 00:09:54.720 +any tools for doing something like that right + +00:09:58.380 --> 00:09:58.880 +now. I personally use my system, + +00:10:00.220 --> 00:10:00.600 +my note system for EKG. + +00:10:01.720 --> 00:10:01.920 +I don't think that's appropriate for + +00:10:02.800 --> 00:10:03.120 +everyone, but it does, + +00:10:04.480 --> 00:10:04.640 +I did write it to have some of these + +00:10:06.540 --> 00:10:06.760 +capabilities of composability that I think + +00:10:08.360 --> 00:10:08.860 +are useful for a prompt generation. + +00:10:11.940 --> 00:10:12.280 +It'd be nice to have a system like that, + +00:10:15.660 --> 00:10:16.160 +but for general use. I don't, + +00:10:17.840 --> 00:10:18.060 +this is something I've been meaning to think + +00:10:18.840 --> 00:10:19.000 +about, like how to do it, + +00:10:19.760 --> 00:10:19.920 +but like this, you know, + +00:10:21.260 --> 00:10:21.660 +if someone's interested in getting this area, + +00:10:26.120 --> 00:10:26.420 +like, I would love to chat about that or, + +00:10:27.600 --> 00:10:27.980 +you know, I think there's a lot of + +00:10:31.020 --> 00:10:31.260 +interesting ideas that we could have to have + +00:10:34.080 --> 00:10:34.540 +a system that allows us to make progress + +00:10:38.860 --> 00:10:39.360 +here. And also, I think there's more to + +00:10:40.520 --> 00:10:40.900 +standardization to be done. + +00:10:42.820 --> 00:10:43.140 +1 thing I'd also like to see that we haven't + +00:10:47.020 --> 00:10:47.220 +done yet is a system for standardizing on + +00:10:48.060 --> 00:10:48.560 +getting structured output. + +00:10:49.640 --> 00:10:50.140 +This is gonna be super useful. + +00:10:52.280 --> 00:10:52.780 +I have this for open AIs API, + +00:10:53.560 --> 00:10:54.060 +cause they support it. + +00:10:55.940 --> 00:10:56.040 +And it's really nice, cause then you can + +00:10:57.440 --> 00:10:57.660 +write elist functions that like, + +00:10:59.380 --> 00:10:59.880 +okay, I'm going to call the LLM. + +00:11:00.760 --> 00:11:01.000 +I'm gonna get structured output. + +00:11:02.040 --> 00:11:02.160 +I know what that structure is going to be. + +00:11:03.480 --> 00:11:03.680 +It's not going to be just a big block of + +00:11:05.660 --> 00:11:06.040 +text. I could turn it into a, + +00:11:07.000 --> 00:11:07.480 +you know, a P list or something. + +00:11:09.280 --> 00:11:09.480 +And then I could get the values out of that P + +00:11:11.880 --> 00:11:12.260 +list. And I know that way I could do, + +00:11:14.220 --> 00:11:14.720 +I could write actual apps that are, + +00:11:18.300 --> 00:11:18.720 +you know, very, very sort of, + +00:11:20.200 --> 00:11:20.680 +you know, useful for very specific purposes + +00:11:21.900 --> 00:11:22.400 +and not just for text generation. + +00:11:24.000 --> 00:11:24.320 +And I think that's 1 of the most important + +00:11:27.100 --> 00:11:27.540 +things we want to do. And I have some ideas + +00:11:28.840 --> 00:11:29.160 +about how to do it. I just haven't pursued + +00:11:31.640 --> 00:11:32.040 +those yet. But if other people have ideas, + +00:11:34.340 --> 00:11:34.540 +I think this would be really interesting to + +00:11:35.520 --> 00:11:36.020 +add to the LLM package. + +00:11:37.260 --> 00:11:37.760 +So contact me there. + +00:11:42.100 --> 00:11:42.600 +[Speaker 1]: Awesome. Quick note before we continue. + +00:11:44.440 --> 00:11:44.540 +So I'm not sure how long we're going to be on + +00:11:46.040 --> 00:11:46.280 +stream for, because this is the last talk + +00:11:48.640 --> 00:11:49.120 +before the break. If we are on the stream + +00:11:49.840 --> 00:11:50.200 +long-term, then great. + +00:11:51.820 --> 00:11:52.300 +But if not, folks are welcome to continue + +00:11:53.320 --> 00:11:53.680 +writing questions on the pad. + +00:11:55.140 --> 00:11:55.280 +And hopefully, Andrew will get to them at + +00:11:58.020 --> 00:11:58.280 +some point. Or if Andrew maybe has some extra + +00:11:59.960 --> 00:12:00.140 +time available and wants to stay on + +00:12:01.640 --> 00:12:01.920 +BigBlueButton here, then folks are also + +00:12:03.940 --> 00:12:04.340 +welcome to join here and chat with Andrew + +00:12:08.940 --> 00:12:09.240 +directly as well. Okay, + +00:12:10.740 --> 00:12:10.900 +awesome. So yeah, the next question is, + +00:12:12.040 --> 00:12:12.400 +what are your thoughts on the carbon + +00:12:14.060 --> 00:12:14.560 +footprint of LLM usage? + +00:12:17.200 --> 00:12:17.700 +[Speaker 0]: Yeah, it's a really interesting question. + +00:12:23.180 --> 00:12:23.360 +I don't have any particular knowledge or + +00:12:25.440 --> 00:12:25.580 +opinions about that. It's something I think + +00:12:26.980 --> 00:12:27.180 +we should all be educating ourselves more + +00:12:32.240 --> 00:12:32.380 +about. It is really, I mean, + +00:12:33.040 --> 00:12:33.220 +there's 2 parts of this, + +00:12:35.380 --> 00:12:35.500 +right? They take a, there's a huge amount of + +00:12:37.160 --> 00:12:37.360 +carbon footprint involved in training these + +00:12:38.720 --> 00:12:39.220 +things. Then running them is relatively + +00:12:42.540 --> 00:12:42.880 +lightweight. So the question is not + +00:12:44.440 --> 00:12:44.920 +necessarily like once it's trained, + +00:12:46.480 --> 00:12:46.640 +like I don't feel like it's a big deal to + +00:12:48.280 --> 00:12:48.560 +keep using it, but like training these things + +00:12:50.680 --> 00:12:51.180 +is kind of like the big carbon cost of it. + +00:12:53.680 --> 00:12:54.160 +But like right now, the way everything's + +00:12:56.040 --> 00:12:56.260 +going, like every, you know, + +00:12:59.060 --> 00:12:59.560 +all, you know, the top 5 or 6 tech companies + +00:13:00.900 --> 00:13:01.400 +are all training their LLMs, + +00:13:03.580 --> 00:13:03.740 +and this is all costing a giant amount of + +00:13:06.820 --> 00:13:07.060 +carbon probably. On the other hand these same + +00:13:08.560 --> 00:13:08.680 +companies are pretty good about using the + +00:13:10.260 --> 00:13:10.440 +least amount of carbon necessary you know + +00:13:12.340 --> 00:13:12.740 +they have their own their tricks for doing + +00:13:13.260 --> 00:13:13.760 +things very efficiently. + +00:13:22.100 --> 00:13:22.360 +[Speaker 1]: Cool next question, LLMs are slow and + +00:13:24.000 --> 00:13:24.340 +responding. Do you think Emacs should provide + +00:13:26.680 --> 00:13:27.180 +more async primitives to keep it responsive? + +00:13:29.380 --> 00:13:29.880 +Like the URL retrieve is quite bad at + +00:13:31.720 --> 00:13:31.760 +building API clients with it. + +00:13:31.920 --> 00:13:32.420 +Building API clients with it? + +00:13:36.400 --> 00:13:36.900 +[Speaker 0]: Yeah. Well, OK, so first of all, + +00:13:40.240 --> 00:13:40.740 +people should be using the LLM client. + +00:13:48.740 --> 00:13:48.900 +And So right now, 1 thing I should have + +00:13:50.220 --> 00:13:50.440 +mentioned at the top is that there are new + +00:13:52.500 --> 00:13:52.640 +packages that I recorded this talk that you + +00:13:54.480 --> 00:13:54.920 +just saw several months ago. + +00:13:57.780 --> 00:13:58.180 +And so like Elama, there's this package Elama + +00:13:59.700 --> 00:14:00.080 +that came out that is using the LM package. + +00:14:02.440 --> 00:14:02.680 +And so for example, it doesn't need to worry + +00:14:05.140 --> 00:14:05.580 +about this sort of thing because it just uses + +00:14:07.560 --> 00:14:07.920 +LLM and package and the LLM package worries + +00:14:11.680 --> 00:14:11.820 +about this. And while I'm on the subject of + +00:14:12.540 --> 00:14:12.720 +things I forgot to mention, + +00:14:15.140 --> 00:14:15.340 +I also should just mention very quickly that + +00:14:17.020 --> 00:14:17.520 +there is now an open source model, + +00:14:21.680 --> 00:14:21.960 +Mistral. And so that's kind of this new thing + +00:14:23.860 --> 00:14:24.240 +on the scene that happened after I recorded + +00:14:26.240 --> 00:14:26.420 +my talk. And I think it's super important to + +00:14:28.660 --> 00:14:28.820 +the community and important that we have the + +00:14:30.620 --> 00:14:31.120 +opportunity to use that if we want to. + +00:14:33.160 --> 00:14:33.660 +Okay, but to answer the actual question, + +00:14:37.660 --> 00:14:38.100 +there has been some talk about the problems + +00:14:40.680 --> 00:14:40.840 +with URL retrieve in the URL package in + +00:14:42.200 --> 00:14:42.700 +general in EmacsDevEl. + +00:14:46.760 --> 00:14:47.080 +It's not great. I would like to have better + +00:14:50.900 --> 00:14:51.040 +primitives. And I've asked the author of + +00:14:54.060 --> 00:14:54.560 +Please PLZ to kind of provide some necessary + +00:14:56.120 --> 00:14:56.620 +callbacks. I think that's a great library. + +00:15:00.280 --> 00:15:00.360 +And I'd like to see that kind of like, + +00:15:01.320 --> 00:15:01.680 +It's nice that we have options, + +00:15:03.340 --> 00:15:03.520 +and that is an option that uses curl on the + +00:15:05.140 --> 00:15:05.640 +back end, and that has some benefits. + +00:15:09.060 --> 00:15:09.280 +So there's this big debate about whether we + +00:15:10.600 --> 00:15:11.100 +should have primitives or just use curl. + +00:15:13.340 --> 00:15:13.420 +I'm not exactly sure what the right call is, + +00:15:15.320 --> 00:15:15.820 +but there has been discussions about this. + +00:15:19.540 --> 00:15:20.040 +[Speaker 1]: Excellent. And someone commented that GPTEL + +00:15:21.820 --> 00:15:22.200 +is async and apparently very good at tracking + +00:15:22.300 --> 00:15:22.800 +the point. + +00:15:26.680 --> 00:15:27.180 +[Speaker 0]: Yes, yes, GPTEL has similar functionalities + +00:15:29.800 --> 00:15:30.040 +to LLM, although I believe it's going to move + +00:15:33.040 --> 00:15:33.540 +to LLM itself sometime soon. + +00:15:39.480 --> 00:15:39.860 +[Speaker 1]: Next question, speaking of which, + +00:15:42.440 --> 00:15:42.560 +anyone trained or fine-tuned or prompted a + +00:15:44.680 --> 00:15:44.760 +model with their org data yet and applied it + +00:15:46.560 --> 00:15:47.040 +to interesting use cases like planning, + +00:15:47.920 --> 00:15:48.340 +scheduling, et cetera, + +00:15:49.320 --> 00:15:49.820 +and maybe care to comment? + +00:15:54.620 --> 00:15:55.120 +[Speaker 0]: I don't know anyone who is doing that. + +00:15:55.860 --> 00:15:56.360 +I think it is interesting. + +00:15:57.800 --> 00:15:58.300 +Like this is what I kind of mentioned at the + +00:16:01.060 --> 00:16:01.300 +very end of the talk. There is a lot of stuff + +00:16:02.440 --> 00:16:02.540 +there like you could you know if you + +00:16:04.760 --> 00:16:04.920 +especially mean an LLM can kind of work as + +00:16:07.940 --> 00:16:08.160 +sort of like a secretary kind of person that + +00:16:12.180 --> 00:16:12.440 +could help you prioritize Still it's a + +00:16:14.760 --> 00:16:14.920 +slightly unclear how what the best way to use + +00:16:16.480 --> 00:16:16.720 +it is So I think there's more of a question + +00:16:18.340 --> 00:16:18.480 +for the community about like what people have + +00:16:21.140 --> 00:16:21.320 +been trying. I see someone has mentioned that + +00:16:23.400 --> 00:16:23.900 +they are using it for weekly review. + +00:16:26.940 --> 00:16:27.180 +And it's kind of nice to like, + +00:16:29.060 --> 00:16:29.380 +maybe you could read your agenda or maybe + +00:16:30.480 --> 00:16:30.780 +this for like weekly review. + +00:16:32.040 --> 00:16:32.240 +It could like read all the stuff you've done + +00:16:33.340 --> 00:16:33.480 +and ask you questions about it. + +00:16:35.020 --> 00:16:35.280 +And like, what should happen next? + +00:16:36.520 --> 00:16:36.780 +Or like, is this going to cause a problem? + +00:16:39.060 --> 00:16:39.280 +Like, I can, I can understand if that could + +00:16:40.860 --> 00:16:41.180 +happen? That's like, that's kind of nice. + +00:16:43.660 --> 00:16:44.160 +And this kind of people have had good success + +00:16:48.540 --> 00:16:48.760 +out of using these LLMs to bounce ideas off + +00:16:49.920 --> 00:16:50.420 +of are, you know, for, + +00:16:52.680 --> 00:16:52.800 +you know, I've seen people say that like they + +00:16:55.360 --> 00:16:55.600 +want, they use it for reading and they kind + +00:16:58.520 --> 00:16:58.740 +of dialogue with the LM to kind of like do + +00:16:59.500 --> 00:17:00.000 +sort of active reading. + +00:17:02.500 --> 00:17:02.860 +So you can imagine doing something similar + +00:17:04.400 --> 00:17:04.740 +with your tasks where it's sort of you're + +00:17:06.560 --> 00:17:06.760 +engaged in dialogue about like planning your + +00:17:08.880 --> 00:17:09.000 +tax with some with a alum that could kind of + +00:17:10.800 --> 00:17:11.180 +understand what those are and ask you some + +00:17:13.780 --> 00:17:13.940 +questions I think it. You know, + +00:17:16.839 --> 00:17:17.040 +if it'd be nice. So, the problem is like + +00:17:18.480 --> 00:17:18.980 +there's no great way to share all this stuff. + +00:17:20.720 --> 00:17:21.220 +I guess if you have something like this, + +00:17:23.300 --> 00:17:23.720 +put it on Reddit. If you don't have Reddit, + +00:17:24.599 --> 00:17:24.880 +I don't know what to do. + +00:17:26.000 --> 00:17:26.500 +I would say put it somewhere. + +00:17:28.840 --> 00:17:29.020 +At the very least, I could maybe open up like + +00:17:31.320 --> 00:17:31.820 +an LLM discussion session on the LLM package + +00:17:34.000 --> 00:17:34.500 +GitHub, But not everyone likes to use GitHub. + +00:17:36.100 --> 00:17:36.180 +I don't know. It'd be nice if there's a + +00:17:38.940 --> 00:17:39.060 +mailing list or IRC chat for this sort of + +00:17:40.840 --> 00:17:41.340 +thing. But there isn't at the moment. + +00:17:46.560 --> 00:17:46.720 +[Speaker 1]: All right. Let's see. I think that's the end + +00:17:48.080 --> 00:17:48.580 +of the questions on the pad so far. + +00:17:51.020 --> 00:17:51.180 +There was also some discussion or some + +00:17:52.260 --> 00:17:52.760 +chatter, I believe, on IRC. + +00:17:54.560 --> 00:17:54.820 +I'm not sure. Andrew, are you on IRC right + +00:18:00.060 --> 00:18:00.260 +[Speaker 0]: I am, but I don't think I'm on any place that + +00:18:01.400 --> 00:18:01.640 +has the chatter. So if there's chatter, + +00:18:02.440 --> 00:18:02.940 +then I'm not seeing it. + +00:18:04.600 --> 00:18:05.100 +[Speaker 1]: now? Okay. Yeah, it was in the emacsconf-dev + +00:18:06.760 --> 00:18:07.260 +channel. + +00:18:09.600 --> 00:18:10.100 +[Speaker 0]: Okay, let me see if I can. + +00:18:25.600 --> 00:18:25.840 +Oh, yes. I mean, I could see the channel, + +00:18:27.520 --> 00:18:27.840 +but I missed whatever came before. + +00:18:29.340 --> 00:18:29.480 +So if there's anything you want to kind of + +00:18:30.840 --> 00:18:31.340 +call out, I can try to answer it here. + +00:18:35.320 --> 00:18:35.640 +[Speaker 1]: OK, cool. I believe at least 2 other folks + +00:18:37.500 --> 00:18:38.000 +who are participating in the discussion there + +00:18:40.120 --> 00:18:40.620 +who have also joined here on BigBlueButton, + +00:18:42.440 --> 00:18:42.940 +Codin Quark and AeonTurn92. + +00:18:47.000 --> 00:18:47.480 +So you folks, if Andrew is still available + +00:18:50.460 --> 00:18:50.640 +and has time, you're welcome to chat here and + +00:18:53.000 --> 00:18:53.320 +ask questions or discuss here as well. + +00:18:55.580 --> 00:18:55.840 +[Speaker 0]: 1 Thank you. Thank you for your help, + +00:18:57.740 --> 00:18:58.080 +and thank you for reading all the questions. + +00:18:59.700 --> 00:18:59.820 +[Speaker 1]: AUDIENCE 2 Cheers, and thanks to you for a + +00:19:00.540 --> 00:19:01.040 +great talk and the discussion. + +00:19:01.880 --> 00:19:02.380 +[Speaker 0]: AUDIENCE AUDIENCE 1 Thank you. + +00:19:03.140 --> 00:19:03.640 +[Speaker 1]: AUDIENCE 2 Cheers. + +00:19:07.900 --> 00:19:08.040 +[Speaker 0]: So I'll just, I will wait here and see if + +00:19:08.320 --> 00:19:08.760 +there's any questions. + +00:19:10.760 --> 00:19:11.260 +If not, I will log off after a few minutes. + +00:19:15.900 --> 00:19:16.080 +[Speaker 2]: Well, I guess since we were mentioned that + +00:19:18.480 --> 00:19:18.980 +there was a small chat about local alarms. + +00:19:22.640 --> 00:19:23.000 +Because chat dpt is nice, + +00:19:25.600 --> 00:19:26.100 +no, but privacy concerns, + +00:19:27.380 --> 00:19:27.880 +and it's not free and stuff. + +00:19:31.000 --> 00:19:31.500 +Which, so The question is, + +00:19:36.960 --> 00:19:37.460 +what is the promise for local models? + +00:19:39.660 --> 00:19:40.160 +[Speaker 0]: Yeah, so local is definitely... + +00:19:41.380 --> 00:19:41.880 +[Speaker 2]: Or at least open source. + +00:19:45.680 --> 00:19:46.120 +[Speaker 0]: Yeah, so there is a local open source model, + +00:19:47.960 --> 00:19:48.460 +Misral, which you could run. + +00:19:51.340 --> 00:19:51.840 +The LLM package allows you to use, + +00:19:56.120 --> 00:19:56.260 +I think there's 3 kind of local things you + +00:19:58.100 --> 00:19:58.440 +could use. Like many of these things, + +00:20:00.220 --> 00:20:00.480 +there's like many kind of ways to do the same + +00:20:03.960 --> 00:20:04.460 +sort of thing. So LLM is supporting OLAMMA + +00:20:10.240 --> 00:20:10.520 +and LLAMMA-CPP. And let's see, + +00:20:12.240 --> 00:20:12.740 +1 other. Which 1 is it? + +00:20:18.420 --> 00:20:18.700 +And maybe that's it. Maybe the, + +00:20:21.820 --> 00:20:21.940 +oh, GPT for all. So each 1 of these kind of + +00:20:23.100 --> 00:20:23.600 +has slightly different functionality. + +00:20:26.820 --> 00:20:27.180 +For example, I think GPT for all doesn't + +00:20:31.780 --> 00:20:32.280 +support embeddings. And I hear that Olama's + +00:20:33.740 --> 00:20:34.240 +embeddings are kind of currently broken. + +00:20:35.920 --> 00:20:36.420 +But basically they should support everything. + +00:20:39.100 --> 00:20:39.600 +And the open source models are, + +00:20:43.180 --> 00:20:43.380 +so the local models are reasonably good. + +00:20:44.760 --> 00:20:44.900 +Like I don't think you'd use them and be + +00:20:46.200 --> 00:20:46.639 +like, what is this horrible nonsense? + +00:20:50.200 --> 00:20:50.380 +Like it's, it gives you relatively good + +00:20:51.820 --> 00:20:52.120 +results. Like it's not gonna be at the level + +00:20:56.060 --> 00:20:56.320 +of like GPT 3.5 or 4, but it's not far away + +00:20:57.720 --> 00:20:58.220 +from GPT 3.5, I think. + +00:21:02.380 --> 00:21:02.880 +[Speaker 2]: I'm just saying that Olam has like a presets + +00:21:05.940 --> 00:21:06.300 +for connecting the actual working servers for + +00:21:06.300 --> 00:21:06.800 +Olama? + +00:21:08.560 --> 00:21:08.760 +[Speaker 0]: So, I'll try. Yeah, so you could, + +00:21:09.860 --> 00:21:10.040 +what you could do is you could like for + +00:21:11.940 --> 00:21:12.100 +example you could download Olama which is + +00:21:15.780 --> 00:21:15.940 +just a way of setting up local models and + +00:21:17.320 --> 00:21:17.780 +running local models on your machine. + +00:21:18.580 --> 00:21:18.820 +So typically what it does, + +00:21:19.720 --> 00:21:20.020 +you like download a program, + +00:21:23.720 --> 00:21:23.800 +let's say Olama. Then Olama will have the + +00:21:24.940 --> 00:21:25.440 +ability to download models. + +00:21:27.240 --> 00:21:27.360 +And so you could choose from just a host of + +00:21:29.280 --> 00:21:29.440 +different models. Each 1 of these things has + +00:21:30.200 --> 00:21:30.440 +a bunch of different models. + +00:21:31.920 --> 00:21:32.080 +So it downloads all these things to your + +00:21:36.600 --> 00:21:37.020 +machine. But I would say that the key problem + +00:21:40.200 --> 00:21:40.580 +here is that it requires a fairly beefy + +00:21:40.580 --> 00:21:41.080 +machine. + +00:21:42.600 --> 00:21:43.100 +[Speaker 2]: So. Yeah, yeah, of course. + +00:21:45.060 --> 00:21:45.300 +Why I was asking, because you briefly + +00:21:46.440 --> 00:21:46.880 +mentioned that there are some Israeli + +00:21:52.300 --> 00:21:52.440 +servers. I understand that they run it like a + +00:21:53.680 --> 00:21:54.000 +government or stuff like that? + +00:21:55.440 --> 00:21:55.940 +No, no, sorry. People want everyone? + +00:21:59.340 --> 00:21:59.620 +[Speaker 0]: I don't, I mean, maybe you've said something + +00:22:00.620 --> 00:22:01.020 +that sounded like Israeli servers. + +00:22:01.620 --> 00:22:02.120 +[Speaker 2]: Okay, okay. + +00:22:04.920 --> 00:22:05.080 +[Speaker 0]: I think- There's no government LLMs as far as + +00:22:06.820 --> 00:22:07.280 +I know. Although, I'm sure the governments + +00:22:08.200 --> 00:22:08.700 +are working on their own LLMs, + +00:22:10.980 --> 00:22:11.480 +et cetera. But yeah, basically your choices + +00:22:15.060 --> 00:22:15.220 +are spend a, I mean, if you use open AI or + +00:22:16.080 --> 00:22:16.580 +something or anything else, + +00:22:17.960 --> 00:22:18.460 +you're really not spending any money. + +00:22:20.560 --> 00:22:20.840 +Like I've never been able to spend any money + +00:22:23.680 --> 00:22:24.020 +on OpenAI. Like unless you're doing something + +00:22:25.840 --> 00:22:26.280 +very intensive and really are using it to, + +00:22:28.000 --> 00:22:28.180 +you know, if you're using it for your + +00:22:29.620 --> 00:22:29.780 +personal use, it's just hard to spend any + +00:22:31.720 --> 00:22:31.960 +money. But on the other hand, + +00:22:32.780 --> 00:22:32.860 +it's not free. So you can, + +00:22:33.040 --> 00:22:33.540 +you know, + +00:22:36.300 --> 00:22:36.680 +[Speaker 2]: Actually, it's rather cheap. + +00:22:37.680 --> 00:22:38.180 +There's no question about that. + +00:22:40.580 --> 00:22:40.920 +The problem is that it has a bad track record + +00:22:41.580 --> 00:22:42.080 +on privacy. + +00:22:45.540 --> 00:22:46.040 +[Speaker 0]: Yes, that's, I think that is a key problem. + +00:22:48.120 --> 00:22:48.280 +This is probably the number 1 reason why you + +00:22:51.840 --> 00:22:52.340 +might want to use a local AI, + +00:22:54.720 --> 00:22:55.220 +a local LLM. Another 1 is like, + +00:22:57.400 --> 00:22:57.900 +you may not agree with the decisions. + +00:23:00.360 --> 00:23:00.820 +You know, there's a lot of trust and safety + +00:23:05.140 --> 00:23:05.440 +stuff that these companies have to do. + +00:23:09.020 --> 00:23:09.240 +Like they don't want like the LMs to kind of + +00:23:11.400 --> 00:23:11.640 +like give you, like tell you how you can make + +00:23:13.180 --> 00:23:13.580 +meth or how you can make a bomb, + +00:23:14.960 --> 00:23:15.460 +which they would do. They would totally do + +00:23:19.580 --> 00:23:20.080 +it. So, But each time you kind of restrict + +00:23:22.540 --> 00:23:22.680 +what is happening with what you can get out + +00:23:23.860 --> 00:23:24.360 +of the LM, it gets a little worse. + +00:23:24.960 --> 00:23:25.440 +So some people + +00:23:27.040 --> 00:23:27.540 +[Speaker 2]: want to have local. That's expected. + +00:23:31.080 --> 00:23:31.400 +I guess even open source language modules + +00:23:33.540 --> 00:23:34.040 +will soon have HR spaces because it's simply + +00:23:34.720 --> 00:23:35.220 +a legal issue. + +00:23:40.760 --> 00:23:41.140 +[Speaker 0]: I think that's true. I also think that there + +00:23:42.880 --> 00:23:43.040 +probably will be, although I don't know of + +00:23:45.060 --> 00:23:45.300 +any offhand, that will are completely + +00:23:46.960 --> 00:23:47.200 +uncensored. I know people are interested and + +00:23:48.240 --> 00:23:48.480 +are running uncensored models. + +00:23:49.440 --> 00:23:49.940 +I don't know how to do it. + +00:23:52.280 --> 00:23:52.780 +I think it's a little bit dubious, + +00:23:54.960 --> 00:23:55.040 +but some people do want to do it. + +00:23:56.280 --> 00:23:56.780 +There's another reason for using local + +00:24:02.280 --> 00:24:02.780 +servers. Do you have any recommendation for + +00:24:05.500 --> 00:24:05.720 +models to run locally and also comments on + +00:24:06.780 --> 00:24:07.280 +whether a GPU is required? + +00:24:14.040 --> 00:24:14.160 +Usually a GPU, well, you can run it without a + +00:24:16.960 --> 00:24:17.460 +GPU, but it does run much better. + +00:24:19.480 --> 00:24:19.980 +Like for example, I think when I used, + +00:24:22.560 --> 00:24:23.060 +Lama is sort of like a standard. + +00:24:27.160 --> 00:24:27.320 +This was the model for that Facebook came out + +00:24:31.380 --> 00:24:31.880 +with for local use. And It was, + +00:24:37.260 --> 00:24:37.760 +yeah, it's good. It's, + +00:24:40.400 --> 00:24:40.900 +but it's now it's I think, + +00:24:44.620 --> 00:24:44.920 +Mistral is kind of like has a better + +00:24:46.480 --> 00:24:46.800 +performance, But there's also different model + +00:24:51.000 --> 00:24:51.500 +sizes. There's 7B, like the Lama 7B is OK. + +00:24:52.940 --> 00:24:53.440 +The Mistral 7B, 7 billion, + +00:24:54.800 --> 00:24:55.300 +are like, basically it'll take like, + +00:24:58.380 --> 00:24:58.880 +you can run it with like 16 gigs of RAM, + +00:25:02.720 --> 00:25:03.040 +is pretty good. It's probably about as equal + +00:25:06.900 --> 00:25:07.000 +to the LLAMA13B. Those are the number of + +00:25:08.360 --> 00:25:08.860 +parameters, if I remember correctly. + +00:25:10.680 --> 00:25:11.180 +And then there's a 7B, + +00:25:12.340 --> 00:25:12.840 +which I've never been able to run. + +00:25:16.120 --> 00:25:16.620 +And even if the 7B, if you run it without a + +00:25:19.640 --> 00:25:20.140 +GPU, it takes quite a while to answer. + +00:25:22.080 --> 00:25:22.580 +I think I've had experiences where it took + +00:25:23.940 --> 00:25:24.440 +literally like several, + +00:25:26.480 --> 00:25:26.780 +like 5 minutes before it even started + +00:25:28.880 --> 00:25:29.100 +responding, but you do eventually get + +00:25:32.220 --> 00:25:32.580 +something. And it could be that like things + +00:25:33.840 --> 00:25:33.960 +have gotten better since the last time I + +00:25:35.440 --> 00:25:35.940 +tried this, because things are moving fast. + +00:25:38.360 --> 00:25:38.860 +But it is super recommended to have a GPU. + +00:25:42.440 --> 00:25:42.620 +This is the problem. It's kind of like, + +00:25:43.840 --> 00:25:44.180 +yes, free software is great. + +00:25:46.880 --> 00:25:47.120 +But if free software is requiring that you + +00:25:50.460 --> 00:25:50.760 +have these kind of beefy servers and have all + +00:25:52.000 --> 00:25:52.500 +this hardware, that's not great. + +00:25:53.600 --> 00:25:54.100 +I think there's a case to be made. + +00:25:55.680 --> 00:25:56.180 +[Speaker 1]: a hardware + +00:25:59.040 --> 00:25:59.540 +[Speaker 0]: with slots instead of a laptop. + +00:26:01.560 --> 00:26:02.060 +Yeah, yeah, that's right. + +00:26:03.660 --> 00:26:03.960 +[Speaker 2]: Ideally, you can have Ideally, + +00:26:07.400 --> 00:26:07.660 +it would be nice if FSL for all things could + +00:26:12.040 --> 00:26:12.540 +run something for open source model. + +00:26:16.320 --> 00:26:16.640 +And not free, but the key point is that it's + +00:26:16.640 --> 00:26:17.140 +Libre? + +00:26:22.580 --> 00:26:23.080 +[Speaker 0]: Yes, so actually I think Google does do that. + +00:26:24.720 --> 00:26:24.800 +I'll have to look it up, + +00:26:27.560 --> 00:26:27.820 +but I haven't explored this yet. + +00:26:31.220 --> 00:26:31.720 +But Google's server, which LLM does support, + +00:26:33.800 --> 00:26:34.300 +supports arbitrary models. + +00:26:36.420 --> 00:26:36.600 +So you can run LLMA or things like that. + +00:26:38.940 --> 00:26:39.200 +The problem is that even if you're running + +00:26:40.320 --> 00:26:40.820 +Mistral, which has no restrictions. + +00:26:42.940 --> 00:26:43.140 +So this is the kind of thing that like the + +00:26:44.900 --> 00:26:45.060 +Free Software Foundation cares a lot about. + +00:26:47.240 --> 00:26:47.740 +Like you want it to be like no restrictions, + +00:26:49.740 --> 00:26:49.840 +legal restrictions on you as you run the + +00:26:52.080 --> 00:26:52.580 +model. So even if it's running Mistral, + +00:26:54.800 --> 00:26:55.300 +just by using the server, + +00:26:58.460 --> 00:26:58.660 +the company server, it will impose some + +00:26:59.440 --> 00:26:59.900 +restrictions on you probably, + +00:27:02.320 --> 00:27:02.480 +right? There's gonna be some license that you + +00:27:04.760 --> 00:27:05.260 +have to, or something you have to abide by. + +00:27:08.480 --> 00:27:08.600 +So I think, yes, it depends on how much you + +00:27:09.280 --> 00:27:09.780 +care about it, I guess. + +00:27:19.500 --> 00:27:19.640 +I should find out more about that and make + +00:27:21.580 --> 00:27:22.080 +sure that it's a good point that I should, + +00:27:23.980 --> 00:27:24.180 +you know, people should be able to run free + +00:27:25.920 --> 00:27:26.280 +models over the server. + +00:27:28.320 --> 00:27:28.440 +So I should make sure we support that in the + +00:27:40.360 --> 00:27:40.860 +LLM package. So, is there any other questions + +00:27:48.240 --> 00:27:48.740 +Or is otherwise we can end the session. + +00:28:00.800 --> 00:28:01.040 +Yeah, all right. Thank you. + +00:28:02.440 --> 00:28:02.940 +Thank you. Thank you everyone who listened. + +00:28:04.540 --> 00:28:05.040 +I'm super happy like I, + +00:28:06.560 --> 00:28:07.060 +the interest is great. + +00:28:08.900 --> 00:28:09.220 +I think there's great stuff to be done here + +00:28:10.960 --> 00:28:11.140 +and I'm kind of super excited what we're + +00:28:11.940 --> 00:28:12.160 +going to do in the next year, + +00:28:13.140 --> 00:28:13.440 +so hopefully, like next year, + +00:28:14.600 --> 00:28:14.760 +and the conference we have something even + +00:28:16.440 --> 00:28:16.560 +more exciting to say about LLM and how they + +00:28:17.320 --> 00:28:17.820 +can be used with Emacs. + +00:28:19.620 --> 00:28:20.120 +So thank + +00:28:30.060 --> 00:28:30.560 +you |