diff options
Diffstat (limited to '2023/info/llm-after.md')
| -rw-r--r-- | 2023/info/llm-after.md | 640 |
1 files changed, 640 insertions, 0 deletions
diff --git a/2023/info/llm-after.md b/2023/info/llm-after.md index 1348a2b3..5a020a0b 100644 --- a/2023/info/llm-after.md +++ b/2023/info/llm-after.md @@ -498,6 +498,646 @@ Captioner: bala +<a name="llm-qanda-transcript"></a> +# Q&A transcript (unedited) + +[[!template text="""[Speaker 0]: Okay. Hello, everyone.""" start="00:00:13.099" video="qanda-llm" id="subtitle"]] +[[!template text="""I think this is the start of the Q&A session.""" start="00:00:16.560" video="qanda-llm" id="subtitle"]] +[[!template text="""So people can just ask me questions here.""" start="00:00:25.119" video="qanda-llm" id="subtitle"]] +[[!template text="""Or I think maybe these questions are going to""" start="00:00:28.259" video="qanda-llm" id="subtitle"]] +[[!template text="""be read by someone. Yes,""" start="00:00:30.560" video="qanda-llm" id="subtitle"]] +[[!template text="""thank you. Should I start doing that?""" start="00:00:34.680" video="qanda-llm" id="subtitle"]] +[[!template text="""I also know that there's questions in the""" start="00:00:39.280" video="qanda-llm" id="subtitle"]] +[[!template text="""either pad room, so I could start out""" start="00:00:41.320" video="qanda-llm" id="subtitle"]] +[[!template text="""answering those as well.""" start="00:00:42.280" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Right, sure. Whichever way you prefer.""" start="00:00:45.020" video="qanda-llm" id="subtitle"]] +[[!template text="""If you prefer to read the questions yourself,""" start="00:00:46.860" video="qanda-llm" id="subtitle"]] +[[!template text="""by all means, or if you would prefer me to""" start="00:00:48.940" video="qanda-llm" id="subtitle"]] +[[!template text="""read them to you, that also works.""" start="00:00:50.080" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Oh, I see.""" start="00:00:50.580" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Why don't you read them to me?""" start="00:00:51.600" video="qanda-llm" id="subtitle"]] +[[!template text="""I think it'll just be more interesting then.""" start="00:00:53.260" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Sure. OK, let's see. The first question is,""" start="00:00:56.820" video="qanda-llm" id="subtitle"]] +[[!template text="""what is your use case for embedding,""" start="00:00:58.360" video="qanda-llm" id="subtitle"]] +[[!template text="""mainly for searching?""" start="00:01:00.060" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, I mean, I think the use case really is""" start="00:01:06.180" video="qanda-llm" id="subtitle"]] +[[!template text="""searching. And I think it is very useful when""" start="00:01:12.100" video="qanda-llm" id="subtitle"]] +[[!template text="""you're searching for something in a vague""" start="00:01:15.060" video="qanda-llm" id="subtitle"]] +[[!template text="""way. Just to give you an example,""" start="00:01:18.280" video="qanda-llm" id="subtitle"]] +[[!template text="""I have a note system called EKG.""" start="00:01:23.860" video="qanda-llm" id="subtitle"]] +[[!template text="""I type all my notes on it.""" start="00:01:25.760" video="qanda-llm" id="subtitle"]] +[[!template text="""You can find it on GitHub and Melba.""" start="00:01:28.620" video="qanda-llm" id="subtitle"]] +[[!template text="""But I wrote something at some point a year""" start="00:01:34.140" video="qanda-llm" id="subtitle"]] +[[!template text="""ago or something. I wrote something that I""" start="00:01:35.840" video="qanda-llm" id="subtitle"]] +[[!template text="""just vaguely remembered.""" start="00:01:36.600" video="qanda-llm" id="subtitle"]] +[[!template text="""Oh, this was about a certain kind of""" start="00:01:38.800" video="qanda-llm" id="subtitle"]] +[[!template text="""communication. I wanted communicating to""" start="00:01:41.580" video="qanda-llm" id="subtitle"]] +[[!template text="""large audiences. There's some interesting tip""" start="00:01:43.280" video="qanda-llm" id="subtitle"]] +[[!template text="""that I wrote down that was really cool.""" start="00:01:44.700" video="qanda-llm" id="subtitle"]] +[[!template text="""And I was like, well, I need to find it.""" start="00:01:49.920" video="qanda-llm" id="subtitle"]] +[[!template text="""So I did an embedding search for something""" start="00:01:52.260" video="qanda-llm" id="subtitle"]] +[[!template text="""like, you know, tips for communicating.""" start="00:01:55.479" video="qanda-llm" id="subtitle"]] +[[!template text="""Like those words may not have been in what I""" start="00:01:58.979" video="qanda-llm" id="subtitle"]] +[[!template text="""was trying to find at all,""" start="00:02:00.020" video="qanda-llm" id="subtitle"]] +[[!template text="""But it was able to find it.""" start="00:02:02.680" video="qanda-llm" id="subtitle"]] +[[!template text="""And that is something that's very hard to do""" start="00:02:05.840" video="qanda-llm" id="subtitle"]] +[[!template text="""in other ways. Like, you know,""" start="00:02:07.200" video="qanda-llm" id="subtitle"]] +[[!template text="""if you had to do this with normal search,""" start="00:02:08.520" video="qanda-llm" id="subtitle"]] +[[!template text="""you have to do synonyms.""" start="00:02:09.199" video="qanda-llm" id="subtitle"]] +[[!template text="""And like maybe those synonyms wouldn't cover""" start="00:02:10.940" video="qanda-llm" id="subtitle"]] +[[!template text="""it. Like with embedding,""" start="00:02:11.960" video="qanda-llm" id="subtitle"]] +[[!template text="""you can basically get at like the vague""" start="00:02:13.940" video="qanda-llm" id="subtitle"]] +[[!template text="""sentiment. You're like,""" start="00:02:14.960" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, you're, you know,""" start="00:02:17.320" video="qanda-llm" id="subtitle"]] +[[!template text="""you can really query on like what things are""" start="00:02:19.520" video="qanda-llm" id="subtitle"]] +[[!template text="""about as opposed to what words they have.""" start="00:02:21.760" video="qanda-llm" id="subtitle"]] +[[!template text="""Also, it's super good for similarity search.""" start="00:02:25.600" video="qanda-llm" id="subtitle"]] +[[!template text="""So you could say, look,""" start="00:02:27.720" video="qanda-llm" id="subtitle"]] +[[!template text="""I have a bunch of things that are encoded""" start="00:02:30.040" video="qanda-llm" id="subtitle"]] +[[!template text="""with embeddings that I want to show.""" start="00:02:31.400" video="qanda-llm" id="subtitle"]] +[[!template text="""For example, you can make an embedding for""" start="00:02:34.120" video="qanda-llm" id="subtitle"]] +[[!template text="""every buffer. You'd be like,""" start="00:02:35.220" video="qanda-llm" id="subtitle"]] +[[!template text="""well, show me buffers that are similar to""" start="00:02:37.060" video="qanda-llm" id="subtitle"]] +[[!template text="""this buffer. That doesn't sound super useful,""" start="00:02:38.740" video="qanda-llm" id="subtitle"]] +[[!template text="""but this is the kind of thing you could do.""" start="00:02:40.440" video="qanda-llm" id="subtitle"]] +[[!template text="""And so if you have a bunch of notes or""" start="00:02:45.300" video="qanda-llm" id="subtitle"]] +[[!template text="""something else that you want to search on,""" start="00:02:46.720" video="qanda-llm" id="subtitle"]] +[[!template text="""you'd be like, what's similar to this buffer?""" start="00:02:48.240" video="qanda-llm" id="subtitle"]] +[[!template text="""Or what notes are similar to each other?""" start="00:02:51.500" video="qanda-llm" id="subtitle"]] +[[!template text="""What buffers are similar to each other?""" start="00:02:53.040" video="qanda-llm" id="subtitle"]] +[[!template text="""It's super good for this sort of thing.""" start="00:02:55.380" video="qanda-llm" id="subtitle"]] +[[!template text="""And it's also good for this kind of retrieval""" start="00:03:00.780" video="qanda-llm" id="subtitle"]] +[[!template text="""augmented generation, where you sort of,""" start="00:03:03.080" video="qanda-llm" id="subtitle"]] +[[!template text="""you retrieve things and the purpose is not""" start="00:03:05.080" video="qanda-llm" id="subtitle"]] +[[!template text="""for you to see them, but then you pass that""" start="00:03:06.600" video="qanda-llm" id="subtitle"]] +[[!template text="""to the LLM. And then it's able to be a little""" start="00:03:12.040" video="qanda-llm" id="subtitle"]] +[[!template text="""bit more accurate because it has the actual""" start="00:03:14.340" video="qanda-llm" id="subtitle"]] +[[!template text="""text that you're trying to,""" start="00:03:15.760" video="qanda-llm" id="subtitle"]] +[[!template text="""that is relevant, and it can cite from and""" start="00:03:18.960" video="qanda-llm" id="subtitle"]] +[[!template text="""things like that. And then it could give you""" start="00:03:20.720" video="qanda-llm" id="subtitle"]] +[[!template text="""a much better answer that's kind of,""" start="00:03:22.260" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, not just from its own little neural""" start="00:03:25.520" video="qanda-llm" id="subtitle"]] +[[!template text="""nets and memory.""" start="00:03:26.320" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Cool, thanks. Let's see,""" start="00:03:31.920" video="qanda-llm" id="subtitle"]] +[[!template text="""next question. What do you think about embed""" start="00:03:35.740" video="qanda-llm" id="subtitle"]] +[[!template text="""Emacs manual versus GPT's Emacs manual?""" start="00:03:40.160" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I'm not exactly sure what this question is""" start="00:03:45.480" video="qanda-llm" id="subtitle"]] +[[!template text="""trying to say. So I mean,""" start="00:03:46.980" video="qanda-llm" id="subtitle"]] +[[!template text="""if someone wrote that and wants to expand on""" start="00:03:51.000" video="qanda-llm" id="subtitle"]] +[[!template text="""it a little bit, but I think that maybe""" start="00:03:55.080" video="qanda-llm" id="subtitle"]] +[[!template text="""you're saying like you could embed,""" start="00:03:58.420" video="qanda-llm" id="subtitle"]] +[[!template text="""have embeddings for like various,""" start="00:04:00.280" video="qanda-llm" id="subtitle"]] +[[!template text="""like every paragraph or something of the""" start="00:04:02.520" video="qanda-llm" id="subtitle"]] +[[!template text="""Emacs manual. But it's also the case that""" start="00:04:04.540" video="qanda-llm" id="subtitle"]] +[[!template text="""like GPT is already for sure already read it,""" start="00:04:06.500" video="qanda-llm" id="subtitle"]] +[[!template text="""right? And so you could ask questions that""" start="00:04:09.760" video="qanda-llm" id="subtitle"]] +[[!template text="""are about Emacs and our ELISP or whatever""" start="00:04:13.460" video="qanda-llm" id="subtitle"]] +[[!template text="""part of the manual you want to find.""" start="00:04:15.200" video="qanda-llm" id="subtitle"]] +[[!template text="""And it will do a reasonably good job,""" start="00:04:19.760" video="qanda-llm" id="subtitle"]] +[[!template text="""especially the better models will do a""" start="00:04:22.280" video="qanda-llm" id="subtitle"]] +[[!template text="""reasonably good job of saying you something""" start="00:04:24.620" video="qanda-llm" id="subtitle"]] +[[!template text="""that is vaguely accurate.""" start="00:04:26.040" video="qanda-llm" id="subtitle"]] +[[!template text="""But if you do this retrieval augmented""" start="00:04:29.440" video="qanda-llm" id="subtitle"]] +[[!template text="""generation with embeddings,""" start="00:04:30.580" video="qanda-llm" id="subtitle"]] +[[!template text="""you can get something that is very accurate.""" start="00:04:32.640" video="qanda-llm" id="subtitle"]] +[[!template text="""At least I think. I haven't tried it,""" start="00:04:36.700" video="qanda-llm" id="subtitle"]] +[[!template text="""but this is a technique that works in other""" start="00:04:38.760" video="qanda-llm" id="subtitle"]] +[[!template text="""similar cases. So you can also imagine like,""" start="00:04:43.040" video="qanda-llm" id="subtitle"]] +[[!template text="""oh, this whole thing I said,""" start="00:04:44.320" video="qanda-llm" id="subtitle"]] +[[!template text="""like, oh, you can query for vague things and""" start="00:04:47.860" video="qanda-llm" id="subtitle"]] +[[!template text="""get parts of the manual,""" start="00:04:49.140" video="qanda-llm" id="subtitle"]] +[[!template text="""perhaps. I'm not exactly sure if that would""" start="00:04:52.680" video="qanda-llm" id="subtitle"]] +[[!template text="""be useful, but maybe. Usually when I'm""" start="00:04:55.120" video="qanda-llm" id="subtitle"]] +[[!template text="""looking things up in the Emacs manual or""" start="00:04:57.040" video="qanda-llm" id="subtitle"]] +[[!template text="""Elist manual, I have something extremely""" start="00:04:58.320" video="qanda-llm" id="subtitle"]] +[[!template text="""specific and I kind of know where to look.""" start="00:05:00.020" video="qanda-llm" id="subtitle"]] +[[!template text="""But having other ways to get at this""" start="00:05:02.960" video="qanda-llm" id="subtitle"]] +[[!template text="""information is always good.""" start="00:05:04.000" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Right. Looks like they added a clarification""" start="00:05:10.240" video="qanda-llm" id="subtitle"]] +[[!template text="""if you would like to read that yourself,""" start="00:05:12.280" video="qanda-llm" id="subtitle"]] +[[!template text="""or would you like me to read it for you?""" start="00:05:14.180" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah. Yes, OK. It says,""" start="00:05:17.640" video="qanda-llm" id="subtitle"]] +[[!template text="""I've never tried. Yeah,""" start="00:05:20.460" video="qanda-llm" id="subtitle"]] +[[!template text="""the question is like OK,""" start="00:05:21.500" video="qanda-llm" id="subtitle"]] +[[!template text="""there is a difference between the kind of""" start="00:05:23.100" video="qanda-llm" id="subtitle"]] +[[!template text="""thing as I just described.""" start="00:05:23.860" video="qanda-llm" id="subtitle"]] +[[!template text="""I have not tried the difference with the EMAX""" start="00:05:26.200" video="qanda-llm" id="subtitle"]] +[[!template text="""manual itself. It'd be interesting to see""" start="00:05:31.560" video="qanda-llm" id="subtitle"]] +[[!template text="""what this is, but I would expect like these""" start="00:05:33.700" video="qanda-llm" id="subtitle"]] +[[!template text="""techniques, the retrieval augmented""" start="00:05:35.140" video="qanda-llm" id="subtitle"]] +[[!template text="""generation is generally pretty good.""" start="00:05:38.840" video="qanda-llm" id="subtitle"]] +[[!template text="""And I suspect it would,""" start="00:05:41.240" video="qanda-llm" id="subtitle"]] +[[!template text="""I would bet money on the fact that it's gonna""" start="00:05:43.580" video="qanda-llm" id="subtitle"]] +[[!template text="""give you, you know, better results than just,""" start="00:05:45.820" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, doing a free form query without any""" start="00:05:48.160" video="qanda-llm" id="subtitle"]] +[[!template text="""retrieval augmented generation.""" start="00:05:49.440" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Cool. Let's see. Next question.""" start="00:05:54.240" video="qanda-llm" id="subtitle"]] +[[!template text="""When deferring commit messages to an LLM,""" start="00:05:56.380" video="qanda-llm" id="subtitle"]] +[[!template text="""what, if anything, do you find you might have""" start="00:05:59.700" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: lost? Yeah, it's a good question.""" start="00:06:02.940" video="qanda-llm" id="subtitle"]] +[[!template text="""When deferring anything to a computer,""" start="00:06:06.060" video="qanda-llm" id="subtitle"]] +[[!template text="""like, you know, I used to have to remember""" start="00:06:08.860" video="qanda-llm" id="subtitle"]] +[[!template text="""how to get places, and now,""" start="00:06:11.200" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, on the few occasions which I drive,""" start="00:06:14.540" video="qanda-llm" id="subtitle"]] +[[!template text="""like, It could just tell me how to get""" start="00:06:16.560" video="qanda-llm" id="subtitle"]] +[[!template text="""places. So similar things could occur here""" start="00:06:21.960" video="qanda-llm" id="subtitle"]] +[[!template text="""where like, okay, I'm just leaving the LLM.""" start="00:06:24.960" video="qanda-llm" id="subtitle"]] +[[!template text="""And so I'm kind of missing out on some""" start="00:06:27.380" video="qanda-llm" id="subtitle"]] +[[!template text="""opportunity to think coherently about a""" start="00:06:30.040" video="qanda-llm" id="subtitle"]] +[[!template text="""particular commit. Particular commits are""" start="00:06:32.440" video="qanda-llm" id="subtitle"]] +[[!template text="""kind of low level. I don't think it's usually""" start="00:06:36.140" video="qanda-llm" id="subtitle"]] +[[!template text="""relatively obvious and what they're doing.""" start="00:06:39.340" video="qanda-llm" id="subtitle"]] +[[!template text="""And in this case, I think there's not much""" start="00:06:42.600" video="qanda-llm" id="subtitle"]] +[[!template text="""loss. But for sure, in other cases,""" start="00:06:44.220" video="qanda-llm" id="subtitle"]] +[[!template text="""if you're starting to get into situations""" start="00:06:46.400" video="qanda-llm" id="subtitle"]] +[[!template text="""where it's writing your emails and all this""" start="00:06:48.640" video="qanda-llm" id="subtitle"]] +[[!template text="""stuff. First of all, it's in 1 sense,""" start="00:06:52.920" video="qanda-llm" id="subtitle"]] +[[!template text="""I'm not sure you might be losing something by""" start="00:06:55.580" video="qanda-llm" id="subtitle"]] +[[!template text="""delegating things. On the other hand,""" start="00:06:57.520" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, when you're interacting with these""" start="00:06:59.120" video="qanda-llm" id="subtitle"]] +[[!template text="""LLMs, you have to be extremely specific about""" start="00:07:01.280" video="qanda-llm" id="subtitle"]] +[[!template text="""what you want, or else it's just not going to""" start="00:07:03.120" video="qanda-llm" id="subtitle"]] +[[!template text="""do a good job. And that might actually be a""" start="00:07:07.540" video="qanda-llm" id="subtitle"]] +[[!template text="""good thing. So the question might be that""" start="00:07:09.440" video="qanda-llm" id="subtitle"]] +[[!template text="""maybe you might gain things by using an LLM""" start="00:07:11.820" video="qanda-llm" id="subtitle"]] +[[!template text="""to do your work. It might not actually even""" start="00:07:13.860" video="qanda-llm" id="subtitle"]] +[[!template text="""save you that much time,""" start="00:07:15.060" video="qanda-llm" id="subtitle"]] +[[!template text="""at least initially, because you have to kind""" start="00:07:18.480" video="qanda-llm" id="subtitle"]] +[[!template text="""of practice again super specific about what""" start="00:07:20.460" video="qanda-llm" id="subtitle"]] +[[!template text="""you want to get out of the output it's going""" start="00:07:22.740" video="qanda-llm" id="subtitle"]] +[[!template text="""to give you so like oh I'm you know maybe you""" start="00:07:26.940" video="qanda-llm" id="subtitle"]] +[[!template text="""know you're on the emacs devel mailing list""" start="00:07:29.600" video="qanda-llm" id="subtitle"]] +[[!template text="""and you're like okay write this email about""" start="00:07:31.780" video="qanda-llm" id="subtitle"]] +[[!template text="""this about this And here's what I want to""" start="00:07:34.000" video="qanda-llm" id="subtitle"]] +[[!template text="""say. And here's the kind of tone I want to""" start="00:07:35.370" video="qanda-llm" id="subtitle"]] +[[!template text="""use. And here's the like,""" start="00:07:36.020" video="qanda-llm" id="subtitle"]] +[[!template text="""oh, you might want to specify like everything""" start="00:07:37.660" video="qanda-llm" id="subtitle"]] +[[!template text="""that you kind of want to get into this.""" start="00:07:39.620" video="qanda-llm" id="subtitle"]] +[[!template text="""Usually it's easier just to write the email.""" start="00:07:42.180" video="qanda-llm" id="subtitle"]] +[[!template text="""But I think that practice of kind of""" start="00:07:45.600" video="qanda-llm" id="subtitle"]] +[[!template text="""understanding what you want is not something""" start="00:07:48.080" video="qanda-llm" id="subtitle"]] +[[!template text="""you normally do. And I think it's going to be""" start="00:07:52.680" video="qanda-llm" id="subtitle"]] +[[!template text="""an interesting exercise that will help people""" start="00:07:56.040" video="qanda-llm" id="subtitle"]] +[[!template text="""understand. That said,""" start="00:07:57.280" video="qanda-llm" id="subtitle"]] +[[!template text="""I haven't done that much of that,""" start="00:07:58.860" video="qanda-llm" id="subtitle"]] +[[!template text="""so I can't say, oh, yeah,""" start="00:07:59.900" video="qanda-llm" id="subtitle"]] +[[!template text="""I've done this and it works for me.""" start="00:08:01.080" video="qanda-llm" id="subtitle"]] +[[!template text="""Maybe. I think it's an interesting thing to""" start="00:08:03.000" video="qanda-llm" id="subtitle"]] +[[!template text="""explore.""" start="00:08:03.120" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Sure. Thanks. Let's see.""" start="00:08:07.720" video="qanda-llm" id="subtitle"]] +[[!template text="""Let's see. Can you share your font settings""" start="00:08:10.840" video="qanda-llm" id="subtitle"]] +[[!template text="""in your Emacs config? Those are some nice""" start="00:08:13.440" video="qanda-llm" id="subtitle"]] +[[!template text="""fonts for reading.""" start="00:08:14.200" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, I think I was using Menlo at the time.""" start="00:08:18.900" video="qanda-llm" id="subtitle"]] +[[!template text="""Unfortunately, I don't save those kinds of""" start="00:08:20.840" video="qanda-llm" id="subtitle"]] +[[!template text="""things, like a history of this.""" start="00:08:21.940" video="qanda-llm" id="subtitle"]] +[[!template text="""I've kind of switched now to,""" start="00:08:24.000" video="qanda-llm" id="subtitle"]] +[[!template text="""what was that? I think I wrote it down in""" start="00:08:27.340" video="qanda-llm" id="subtitle"]] +[[!template text="""the, I switched to MunaSpace,""" start="00:08:29.440" video="qanda-llm" id="subtitle"]] +[[!template text="""which just came out like a week or 2 ago,""" start="00:08:31.920" video="qanda-llm" id="subtitle"]] +[[!template text="""and is also pretty cool.""" start="00:08:33.340" video="qanda-llm" id="subtitle"]] +[[!template text="""So I think it's Menlo.""" start="00:08:35.440" video="qanda-llm" id="subtitle"]] +[[!template text="""The internal question,""" start="00:08:37.380" video="qanda-llm" id="subtitle"]] +[[!template text="""what font are you using?""" start="00:08:38.400" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Indeed, yeah. It looks like someone guessed""" start="00:08:42.020" video="qanda-llm" id="subtitle"]] +[[!template text="""as well that it might be Menlo.""" start="00:08:43.780" video="qanda-llm" id="subtitle"]] +[[!template text="""OK, Cool. Yeah, next question.""" start="00:08:47.680" video="qanda-llm" id="subtitle"]] +[[!template text="""In terms of standardization,""" start="00:08:48.900" video="qanda-llm" id="subtitle"]] +[[!template text="""do you see a need for the medium to large""" start="00:08:53.260" video="qanda-llm" id="subtitle"]] +[[!template text="""scale effort needed? And then they also""" start="00:08:55.840" video="qanda-llm" id="subtitle"]] +[[!template text="""elaborate about it.""" start="00:08:56.960" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, I mean, I do think,""" start="00:09:03.600" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't know if it's large scale,""" start="00:09:06.040" video="qanda-llm" id="subtitle"]] +[[!template text="""but at least it's probably medium scale.""" start="00:09:08.000" video="qanda-llm" id="subtitle"]] +[[!template text="""There's a lot of things that are missing that""" start="00:09:10.320" video="qanda-llm" id="subtitle"]] +[[!template text="""we don't have right now in emacs when you're""" start="00:09:12.260" video="qanda-llm" id="subtitle"]] +[[!template text="""dealing with LLMs. 1 is,""" start="00:09:13.660" video="qanda-llm" id="subtitle"]] +[[!template text="""a prompting system. And by that,""" start="00:09:18.240" video="qanda-llm" id="subtitle"]] +[[!template text="""I mean, you know, prompts are just like big""" start="00:09:21.820" video="qanda-llm" id="subtitle"]] +[[!template text="""blocks of text, but there's also senses that""" start="00:09:24.520" video="qanda-llm" id="subtitle"]] +[[!template text="""like prompts need to be composable and you""" start="00:09:28.260" video="qanda-llm" id="subtitle"]] +[[!template text="""need to be able to iterate on parts of the""" start="00:09:30.480" video="qanda-llm" id="subtitle"]] +[[!template text="""prompt. And so it's also customizable.""" start="00:09:36.600" video="qanda-llm" id="subtitle"]] +[[!template text="""Users might want to customize it.""" start="00:09:38.940" video="qanda-llm" id="subtitle"]] +[[!template text="""On the other hand, it's not super easy to""" start="00:09:41.260" video="qanda-llm" id="subtitle"]] +[[!template text="""write the prompt. So you want really good""" start="00:09:43.820" video="qanda-llm" id="subtitle"]] +[[!template text="""defaults. So the whole prompt system is kind""" start="00:09:47.900" video="qanda-llm" id="subtitle"]] +[[!template text="""of complicated. That needs to be kind of""" start="00:09:51.360" video="qanda-llm" id="subtitle"]] +[[!template text="""standardized, because I don't think there's""" start="00:09:52.580" video="qanda-llm" id="subtitle"]] +[[!template text="""any tools for doing something like that right""" start="00:09:54.380" video="qanda-llm" id="subtitle"]] +[[!template text="""now. I personally use my system,""" start="00:09:58.380" video="qanda-llm" id="subtitle"]] +[[!template text="""my note system for EKG.""" start="00:10:00.220" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't think that's appropriate for""" start="00:10:01.720" video="qanda-llm" id="subtitle"]] +[[!template text="""everyone, but it does,""" start="00:10:02.800" video="qanda-llm" id="subtitle"]] +[[!template text="""I did write it to have some of these""" start="00:10:04.480" video="qanda-llm" id="subtitle"]] +[[!template text="""capabilities of composability that I think""" start="00:10:06.540" video="qanda-llm" id="subtitle"]] +[[!template text="""are useful for a prompt generation.""" start="00:10:08.360" video="qanda-llm" id="subtitle"]] +[[!template text="""It'd be nice to have a system like that,""" start="00:10:11.940" video="qanda-llm" id="subtitle"]] +[[!template text="""but for general use. I don't,""" start="00:10:15.660" video="qanda-llm" id="subtitle"]] +[[!template text="""this is something I've been meaning to think""" start="00:10:17.840" video="qanda-llm" id="subtitle"]] +[[!template text="""about, like how to do it,""" start="00:10:18.840" video="qanda-llm" id="subtitle"]] +[[!template text="""but like this, you know,""" start="00:10:19.760" video="qanda-llm" id="subtitle"]] +[[!template text="""if someone's interested in getting this area,""" start="00:10:21.260" video="qanda-llm" id="subtitle"]] +[[!template text="""like, I would love to chat about that or,""" start="00:10:26.120" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, I think there's a lot of""" start="00:10:27.600" video="qanda-llm" id="subtitle"]] +[[!template text="""interesting ideas that we could have to have""" start="00:10:31.020" video="qanda-llm" id="subtitle"]] +[[!template text="""a system that allows us to make progress""" start="00:10:34.080" video="qanda-llm" id="subtitle"]] +[[!template text="""here. And also, I think there's more to""" start="00:10:38.860" video="qanda-llm" id="subtitle"]] +[[!template text="""standardization to be done.""" start="00:10:40.520" video="qanda-llm" id="subtitle"]] +[[!template text="""1 thing I'd also like to see that we haven't""" start="00:10:42.820" video="qanda-llm" id="subtitle"]] +[[!template text="""done yet is a system for standardizing on""" start="00:10:47.020" video="qanda-llm" id="subtitle"]] +[[!template text="""getting structured output.""" start="00:10:48.060" video="qanda-llm" id="subtitle"]] +[[!template text="""This is gonna be super useful.""" start="00:10:49.640" video="qanda-llm" id="subtitle"]] +[[!template text="""I have this for open AIs API,""" start="00:10:52.280" video="qanda-llm" id="subtitle"]] +[[!template text="""cause they support it.""" start="00:10:53.560" video="qanda-llm" id="subtitle"]] +[[!template text="""And it's really nice, cause then you can""" start="00:10:55.940" video="qanda-llm" id="subtitle"]] +[[!template text="""write elist functions that like,""" start="00:10:57.440" video="qanda-llm" id="subtitle"]] +[[!template text="""okay, I'm going to call the LLM.""" start="00:10:59.380" video="qanda-llm" id="subtitle"]] +[[!template text="""I'm gonna get structured output.""" start="00:11:00.760" video="qanda-llm" id="subtitle"]] +[[!template text="""I know what that structure is going to be.""" start="00:11:02.040" video="qanda-llm" id="subtitle"]] +[[!template text="""It's not going to be just a big block of""" start="00:11:03.480" video="qanda-llm" id="subtitle"]] +[[!template text="""text. I could turn it into a,""" start="00:11:05.660" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, a P list or something.""" start="00:11:07.000" video="qanda-llm" id="subtitle"]] +[[!template text="""And then I could get the values out of that P""" start="00:11:09.280" video="qanda-llm" id="subtitle"]] +[[!template text="""list. And I know that way I could do,""" start="00:11:11.880" video="qanda-llm" id="subtitle"]] +[[!template text="""I could write actual apps that are,""" start="00:11:14.220" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, very, very sort of,""" start="00:11:18.300" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, useful for very specific purposes""" start="00:11:20.200" video="qanda-llm" id="subtitle"]] +[[!template text="""and not just for text generation.""" start="00:11:21.900" video="qanda-llm" id="subtitle"]] +[[!template text="""And I think that's 1 of the most important""" start="00:11:24.000" video="qanda-llm" id="subtitle"]] +[[!template text="""things we want to do. And I have some ideas""" start="00:11:27.100" video="qanda-llm" id="subtitle"]] +[[!template text="""about how to do it. I just haven't pursued""" start="00:11:28.840" video="qanda-llm" id="subtitle"]] +[[!template text="""those yet. But if other people have ideas,""" start="00:11:31.640" video="qanda-llm" id="subtitle"]] +[[!template text="""I think this would be really interesting to""" start="00:11:34.340" video="qanda-llm" id="subtitle"]] +[[!template text="""add to the LLM package.""" start="00:11:35.520" video="qanda-llm" id="subtitle"]] +[[!template text="""So contact me there.""" start="00:11:37.260" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Awesome. Quick note before we continue.""" start="00:11:42.100" video="qanda-llm" id="subtitle"]] +[[!template text="""So I'm not sure how long we're going to be on""" start="00:11:44.440" video="qanda-llm" id="subtitle"]] +[[!template text="""stream for, because this is the last talk""" start="00:11:46.040" video="qanda-llm" id="subtitle"]] +[[!template text="""before the break. If we are on the stream""" start="00:11:48.640" video="qanda-llm" id="subtitle"]] +[[!template text="""long-term, then great.""" start="00:11:49.840" video="qanda-llm" id="subtitle"]] +[[!template text="""But if not, folks are welcome to continue""" start="00:11:51.820" video="qanda-llm" id="subtitle"]] +[[!template text="""writing questions on the pad.""" start="00:11:53.320" video="qanda-llm" id="subtitle"]] +[[!template text="""And hopefully, Andrew will get to them at""" start="00:11:55.140" video="qanda-llm" id="subtitle"]] +[[!template text="""some point. Or if Andrew maybe has some extra""" start="00:11:58.020" video="qanda-llm" id="subtitle"]] +[[!template text="""time available and wants to stay on""" start="00:11:59.960" video="qanda-llm" id="subtitle"]] +[[!template text="""BigBlueButton here, then folks are also""" start="00:12:01.640" video="qanda-llm" id="subtitle"]] +[[!template text="""welcome to join here and chat with Andrew""" start="00:12:03.940" video="qanda-llm" id="subtitle"]] +[[!template text="""directly as well. Okay,""" start="00:12:08.940" video="qanda-llm" id="subtitle"]] +[[!template text="""awesome. So yeah, the next question is,""" start="00:12:10.740" video="qanda-llm" id="subtitle"]] +[[!template text="""what are your thoughts on the carbon""" start="00:12:12.040" video="qanda-llm" id="subtitle"]] +[[!template text="""footprint of LLM usage?""" start="00:12:14.060" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, it's a really interesting question.""" start="00:12:17.200" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't have any particular knowledge or""" start="00:12:23.180" video="qanda-llm" id="subtitle"]] +[[!template text="""opinions about that. It's something I think""" start="00:12:25.440" video="qanda-llm" id="subtitle"]] +[[!template text="""we should all be educating ourselves more""" start="00:12:26.980" video="qanda-llm" id="subtitle"]] +[[!template text="""about. It is really, I mean,""" start="00:12:32.240" video="qanda-llm" id="subtitle"]] +[[!template text="""there's 2 parts of this,""" start="00:12:33.040" video="qanda-llm" id="subtitle"]] +[[!template text="""right? They take a, there's a huge amount of""" start="00:12:35.380" video="qanda-llm" id="subtitle"]] +[[!template text="""carbon footprint involved in training these""" start="00:12:37.160" video="qanda-llm" id="subtitle"]] +[[!template text="""things. Then running them is relatively""" start="00:12:38.720" video="qanda-llm" id="subtitle"]] +[[!template text="""lightweight. So the question is not""" start="00:12:42.540" video="qanda-llm" id="subtitle"]] +[[!template text="""necessarily like once it's trained,""" start="00:12:44.440" video="qanda-llm" id="subtitle"]] +[[!template text="""like I don't feel like it's a big deal to""" start="00:12:46.480" video="qanda-llm" id="subtitle"]] +[[!template text="""keep using it, but like training these things""" start="00:12:48.280" video="qanda-llm" id="subtitle"]] +[[!template text="""is kind of like the big carbon cost of it.""" start="00:12:50.680" video="qanda-llm" id="subtitle"]] +[[!template text="""But like right now, the way everything's""" start="00:12:53.680" video="qanda-llm" id="subtitle"]] +[[!template text="""going, like every, you know,""" start="00:12:56.040" video="qanda-llm" id="subtitle"]] +[[!template text="""all, you know, the top 5 or 6 tech companies""" start="00:12:59.060" video="qanda-llm" id="subtitle"]] +[[!template text="""are all training their LLMs,""" start="00:13:00.900" video="qanda-llm" id="subtitle"]] +[[!template text="""and this is all costing a giant amount of""" start="00:13:03.580" video="qanda-llm" id="subtitle"]] +[[!template text="""carbon probably. On the other hand these same""" start="00:13:06.820" video="qanda-llm" id="subtitle"]] +[[!template text="""companies are pretty good about using the""" start="00:13:08.560" video="qanda-llm" id="subtitle"]] +[[!template text="""least amount of carbon necessary you know""" start="00:13:10.260" video="qanda-llm" id="subtitle"]] +[[!template text="""they have their own their tricks for doing""" start="00:13:12.340" video="qanda-llm" id="subtitle"]] +[[!template text="""things very efficiently.""" start="00:13:13.260" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Cool next question, LLMs are slow and""" start="00:13:22.100" video="qanda-llm" id="subtitle"]] +[[!template text="""responding. Do you think Emacs should provide""" start="00:13:24.000" video="qanda-llm" id="subtitle"]] +[[!template text="""more async primitives to keep it responsive?""" start="00:13:26.680" video="qanda-llm" id="subtitle"]] +[[!template text="""Like the URL retrieve is quite bad at""" start="00:13:29.380" video="qanda-llm" id="subtitle"]] +[[!template text="""building API clients with it.""" start="00:13:31.720" video="qanda-llm" id="subtitle"]] +[[!template text="""Building API clients with it?""" start="00:13:31.920" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah. Well, OK, so first of all,""" start="00:13:36.400" video="qanda-llm" id="subtitle"]] +[[!template text="""people should be using the LLM client.""" start="00:13:40.240" video="qanda-llm" id="subtitle"]] +[[!template text="""And So right now, 1 thing I should have""" start="00:13:48.740" video="qanda-llm" id="subtitle"]] +[[!template text="""mentioned at the top is that there are new""" start="00:13:50.220" video="qanda-llm" id="subtitle"]] +[[!template text="""packages that I recorded this talk that you""" start="00:13:52.500" video="qanda-llm" id="subtitle"]] +[[!template text="""just saw several months ago.""" start="00:13:54.480" video="qanda-llm" id="subtitle"]] +[[!template text="""And so like Elama, there's this package Elama""" start="00:13:57.780" video="qanda-llm" id="subtitle"]] +[[!template text="""that came out that is using the LM package.""" start="00:13:59.700" video="qanda-llm" id="subtitle"]] +[[!template text="""And so for example, it doesn't need to worry""" start="00:14:02.440" video="qanda-llm" id="subtitle"]] +[[!template text="""about this sort of thing because it just uses""" start="00:14:05.140" video="qanda-llm" id="subtitle"]] +[[!template text="""LLM and package and the LLM package worries""" start="00:14:07.560" video="qanda-llm" id="subtitle"]] +[[!template text="""about this. And while I'm on the subject of""" start="00:14:11.680" video="qanda-llm" id="subtitle"]] +[[!template text="""things I forgot to mention,""" start="00:14:12.540" video="qanda-llm" id="subtitle"]] +[[!template text="""I also should just mention very quickly that""" start="00:14:15.140" video="qanda-llm" id="subtitle"]] +[[!template text="""there is now an open source model,""" start="00:14:17.020" video="qanda-llm" id="subtitle"]] +[[!template text="""Mistral. And so that's kind of this new thing""" start="00:14:21.680" video="qanda-llm" id="subtitle"]] +[[!template text="""on the scene that happened after I recorded""" start="00:14:23.860" video="qanda-llm" id="subtitle"]] +[[!template text="""my talk. And I think it's super important to""" start="00:14:26.240" video="qanda-llm" id="subtitle"]] +[[!template text="""the community and important that we have the""" start="00:14:28.660" video="qanda-llm" id="subtitle"]] +[[!template text="""opportunity to use that if we want to.""" start="00:14:30.620" video="qanda-llm" id="subtitle"]] +[[!template text="""Okay, but to answer the actual question,""" start="00:14:33.160" video="qanda-llm" id="subtitle"]] +[[!template text="""there has been some talk about the problems""" start="00:14:37.660" video="qanda-llm" id="subtitle"]] +[[!template text="""with URL retrieve in the URL package in""" start="00:14:40.680" video="qanda-llm" id="subtitle"]] +[[!template text="""general in EmacsDevEl.""" start="00:14:42.200" video="qanda-llm" id="subtitle"]] +[[!template text="""It's not great. I would like to have better""" start="00:14:46.760" video="qanda-llm" id="subtitle"]] +[[!template text="""primitives. And I've asked the author of""" start="00:14:50.900" video="qanda-llm" id="subtitle"]] +[[!template text="""Please PLZ to kind of provide some necessary""" start="00:14:54.060" video="qanda-llm" id="subtitle"]] +[[!template text="""callbacks. I think that's a great library.""" start="00:14:56.120" video="qanda-llm" id="subtitle"]] +[[!template text="""And I'd like to see that kind of like,""" start="00:15:00.280" video="qanda-llm" id="subtitle"]] +[[!template text="""It's nice that we have options,""" start="00:15:01.320" video="qanda-llm" id="subtitle"]] +[[!template text="""and that is an option that uses curl on the""" start="00:15:03.340" video="qanda-llm" id="subtitle"]] +[[!template text="""back end, and that has some benefits.""" start="00:15:05.140" video="qanda-llm" id="subtitle"]] +[[!template text="""So there's this big debate about whether we""" start="00:15:09.060" video="qanda-llm" id="subtitle"]] +[[!template text="""should have primitives or just use curl.""" start="00:15:10.600" video="qanda-llm" id="subtitle"]] +[[!template text="""I'm not exactly sure what the right call is,""" start="00:15:13.340" video="qanda-llm" id="subtitle"]] +[[!template text="""but there has been discussions about this.""" start="00:15:15.320" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Excellent. And someone commented that GPTEL""" start="00:15:19.540" video="qanda-llm" id="subtitle"]] +[[!template text="""is async and apparently very good at tracking""" start="00:15:21.820" video="qanda-llm" id="subtitle"]] +[[!template text="""the point.""" start="00:15:22.300" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yes, yes, GPTEL has similar functionalities""" start="00:15:26.680" video="qanda-llm" id="subtitle"]] +[[!template text="""to LLM, although I believe it's going to move""" start="00:15:29.800" video="qanda-llm" id="subtitle"]] +[[!template text="""to LLM itself sometime soon.""" start="00:15:33.040" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: Next question, speaking of which,""" start="00:15:39.480" video="qanda-llm" id="subtitle"]] +[[!template text="""anyone trained or fine-tuned or prompted a""" start="00:15:42.440" video="qanda-llm" id="subtitle"]] +[[!template text="""model with their org data yet and applied it""" start="00:15:44.680" video="qanda-llm" id="subtitle"]] +[[!template text="""to interesting use cases like planning,""" start="00:15:46.560" video="qanda-llm" id="subtitle"]] +[[!template text="""scheduling, et cetera,""" start="00:15:47.920" video="qanda-llm" id="subtitle"]] +[[!template text="""and maybe care to comment?""" start="00:15:49.320" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I don't know anyone who is doing that.""" start="00:15:54.620" video="qanda-llm" id="subtitle"]] +[[!template text="""I think it is interesting.""" start="00:15:55.860" video="qanda-llm" id="subtitle"]] +[[!template text="""Like this is what I kind of mentioned at the""" start="00:15:57.800" video="qanda-llm" id="subtitle"]] +[[!template text="""very end of the talk. There is a lot of stuff""" start="00:16:01.060" video="qanda-llm" id="subtitle"]] +[[!template text="""there like you could you know if you""" start="00:16:02.440" video="qanda-llm" id="subtitle"]] +[[!template text="""especially mean an LLM can kind of work as""" start="00:16:04.760" video="qanda-llm" id="subtitle"]] +[[!template text="""sort of like a secretary kind of person that""" start="00:16:07.940" video="qanda-llm" id="subtitle"]] +[[!template text="""could help you prioritize Still it's a""" start="00:16:12.180" video="qanda-llm" id="subtitle"]] +[[!template text="""slightly unclear how what the best way to use""" start="00:16:14.760" video="qanda-llm" id="subtitle"]] +[[!template text="""it is So I think there's more of a question""" start="00:16:16.480" video="qanda-llm" id="subtitle"]] +[[!template text="""for the community about like what people have""" start="00:16:18.340" video="qanda-llm" id="subtitle"]] +[[!template text="""been trying. I see someone has mentioned that""" start="00:16:21.140" video="qanda-llm" id="subtitle"]] +[[!template text="""they are using it for weekly review.""" start="00:16:23.400" video="qanda-llm" id="subtitle"]] +[[!template text="""And it's kind of nice to like,""" start="00:16:26.940" video="qanda-llm" id="subtitle"]] +[[!template text="""maybe you could read your agenda or maybe""" start="00:16:29.060" video="qanda-llm" id="subtitle"]] +[[!template text="""this for like weekly review.""" start="00:16:30.480" video="qanda-llm" id="subtitle"]] +[[!template text="""It could like read all the stuff you've done""" start="00:16:32.040" video="qanda-llm" id="subtitle"]] +[[!template text="""and ask you questions about it.""" start="00:16:33.340" video="qanda-llm" id="subtitle"]] +[[!template text="""And like, what should happen next?""" start="00:16:35.020" video="qanda-llm" id="subtitle"]] +[[!template text="""Or like, is this going to cause a problem?""" start="00:16:36.520" video="qanda-llm" id="subtitle"]] +[[!template text="""Like, I can, I can understand if that could""" start="00:16:39.060" video="qanda-llm" id="subtitle"]] +[[!template text="""happen? That's like, that's kind of nice.""" start="00:16:40.860" video="qanda-llm" id="subtitle"]] +[[!template text="""And this kind of people have had good success""" start="00:16:43.660" video="qanda-llm" id="subtitle"]] +[[!template text="""out of using these LLMs to bounce ideas off""" start="00:16:48.540" video="qanda-llm" id="subtitle"]] +[[!template text="""of are, you know, for,""" start="00:16:49.920" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, I've seen people say that like they""" start="00:16:52.680" video="qanda-llm" id="subtitle"]] +[[!template text="""want, they use it for reading and they kind""" start="00:16:55.360" video="qanda-llm" id="subtitle"]] +[[!template text="""of dialogue with the LM to kind of like do""" start="00:16:58.520" video="qanda-llm" id="subtitle"]] +[[!template text="""sort of active reading.""" start="00:16:59.500" video="qanda-llm" id="subtitle"]] +[[!template text="""So you can imagine doing something similar""" start="00:17:02.500" video="qanda-llm" id="subtitle"]] +[[!template text="""with your tasks where it's sort of you're""" start="00:17:04.400" video="qanda-llm" id="subtitle"]] +[[!template text="""engaged in dialogue about like planning your""" start="00:17:06.560" video="qanda-llm" id="subtitle"]] +[[!template text="""tax with some with a alum that could kind of""" start="00:17:08.880" video="qanda-llm" id="subtitle"]] +[[!template text="""understand what those are and ask you some""" start="00:17:10.800" video="qanda-llm" id="subtitle"]] +[[!template text="""questions I think it. You know,""" start="00:17:13.780" video="qanda-llm" id="subtitle"]] +[[!template text="""if it'd be nice. So, the problem is like""" start="00:17:16.839" video="qanda-llm" id="subtitle"]] +[[!template text="""there's no great way to share all this stuff.""" start="00:17:18.480" video="qanda-llm" id="subtitle"]] +[[!template text="""I guess if you have something like this,""" start="00:17:20.720" video="qanda-llm" id="subtitle"]] +[[!template text="""put it on Reddit. If you don't have Reddit,""" start="00:17:23.300" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't know what to do.""" start="00:17:24.599" video="qanda-llm" id="subtitle"]] +[[!template text="""I would say put it somewhere.""" start="00:17:26.000" video="qanda-llm" id="subtitle"]] +[[!template text="""At the very least, I could maybe open up like""" start="00:17:28.840" video="qanda-llm" id="subtitle"]] +[[!template text="""an LLM discussion session on the LLM package""" start="00:17:31.320" video="qanda-llm" id="subtitle"]] +[[!template text="""GitHub, But not everyone likes to use GitHub.""" start="00:17:34.000" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't know. It'd be nice if there's a""" start="00:17:36.100" video="qanda-llm" id="subtitle"]] +[[!template text="""mailing list or IRC chat for this sort of""" start="00:17:38.940" video="qanda-llm" id="subtitle"]] +[[!template text="""thing. But there isn't at the moment.""" start="00:17:40.840" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: All right. Let's see. I think that's the end""" start="00:17:46.560" video="qanda-llm" id="subtitle"]] +[[!template text="""of the questions on the pad so far.""" start="00:17:48.080" video="qanda-llm" id="subtitle"]] +[[!template text="""There was also some discussion or some""" start="00:17:51.020" video="qanda-llm" id="subtitle"]] +[[!template text="""chatter, I believe, on IRC.""" start="00:17:52.260" video="qanda-llm" id="subtitle"]] +[[!template text="""I'm not sure. Andrew, are you on IRC right""" start="00:17:54.560" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I am, but I don't think I'm on any place that""" start="00:18:00.060" video="qanda-llm" id="subtitle"]] +[[!template text="""has the chatter. So if there's chatter,""" start="00:18:01.400" video="qanda-llm" id="subtitle"]] +[[!template text="""then I'm not seeing it.""" start="00:18:02.440" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: now? Okay. Yeah, it was in the emacsconf-dev""" start="00:18:04.600" video="qanda-llm" id="subtitle"]] +[[!template text="""channel.""" start="00:18:06.760" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Okay, let me see if I can.""" start="00:18:09.600" video="qanda-llm" id="subtitle"]] +[[!template text="""Oh, yes. I mean, I could see the channel,""" start="00:18:25.600" video="qanda-llm" id="subtitle"]] +[[!template text="""but I missed whatever came before.""" start="00:18:27.520" video="qanda-llm" id="subtitle"]] +[[!template text="""So if there's anything you want to kind of""" start="00:18:29.340" video="qanda-llm" id="subtitle"]] +[[!template text="""call out, I can try to answer it here.""" start="00:18:30.840" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: OK, cool. I believe at least 2 other folks""" start="00:18:35.320" video="qanda-llm" id="subtitle"]] +[[!template text="""who are participating in the discussion there""" start="00:18:37.500" video="qanda-llm" id="subtitle"]] +[[!template text="""who have also joined here on BigBlueButton,""" start="00:18:40.120" video="qanda-llm" id="subtitle"]] +[[!template text="""Codin Quark and AeonTurn92.""" start="00:18:42.440" video="qanda-llm" id="subtitle"]] +[[!template text="""So you folks, if Andrew is still available""" start="00:18:47.000" video="qanda-llm" id="subtitle"]] +[[!template text="""and has time, you're welcome to chat here and""" start="00:18:50.460" video="qanda-llm" id="subtitle"]] +[[!template text="""ask questions or discuss here as well.""" start="00:18:53.000" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: 1 Thank you. Thank you for your help,""" start="00:18:55.580" video="qanda-llm" id="subtitle"]] +[[!template text="""and thank you for reading all the questions.""" start="00:18:57.740" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: AUDIENCE 2 Cheers, and thanks to you for a""" start="00:18:59.700" video="qanda-llm" id="subtitle"]] +[[!template text="""great talk and the discussion.""" start="00:19:00.540" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: AUDIENCE AUDIENCE 1 Thank you.""" start="00:19:01.880" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: AUDIENCE 2 Cheers.""" start="00:19:03.140" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: So I'll just, I will wait here and see if""" start="00:19:07.900" video="qanda-llm" id="subtitle"]] +[[!template text="""there's any questions.""" start="00:19:08.320" video="qanda-llm" id="subtitle"]] +[[!template text="""If not, I will log off after a few minutes.""" start="00:19:10.760" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Well, I guess since we were mentioned that""" start="00:19:15.900" video="qanda-llm" id="subtitle"]] +[[!template text="""there was a small chat about local alarms.""" start="00:19:18.480" video="qanda-llm" id="subtitle"]] +[[!template text="""Because chat dpt is nice,""" start="00:19:22.640" video="qanda-llm" id="subtitle"]] +[[!template text="""no, but privacy concerns,""" start="00:19:25.600" video="qanda-llm" id="subtitle"]] +[[!template text="""and it's not free and stuff.""" start="00:19:27.380" video="qanda-llm" id="subtitle"]] +[[!template text="""Which, so The question is,""" start="00:19:31.000" video="qanda-llm" id="subtitle"]] +[[!template text="""what is the promise for local models?""" start="00:19:36.960" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, so local is definitely...""" start="00:19:39.660" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Or at least open source.""" start="00:19:41.380" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yeah, so there is a local open source model,""" start="00:19:45.680" video="qanda-llm" id="subtitle"]] +[[!template text="""Misral, which you could run.""" start="00:19:47.960" video="qanda-llm" id="subtitle"]] +[[!template text="""The LLM package allows you to use,""" start="00:19:51.340" video="qanda-llm" id="subtitle"]] +[[!template text="""I think there's 3 kind of local things you""" start="00:19:56.120" video="qanda-llm" id="subtitle"]] +[[!template text="""could use. Like many of these things,""" start="00:19:58.100" video="qanda-llm" id="subtitle"]] +[[!template text="""there's like many kind of ways to do the same""" start="00:20:00.220" video="qanda-llm" id="subtitle"]] +[[!template text="""sort of thing. So LLM is supporting OLAMMA""" start="00:20:03.960" video="qanda-llm" id="subtitle"]] +[[!template text="""and LLAMMA-CPP. And let's see,""" start="00:20:10.240" video="qanda-llm" id="subtitle"]] +[[!template text="""1 other. Which 1 is it?""" start="00:20:12.240" video="qanda-llm" id="subtitle"]] +[[!template text="""And maybe that's it. Maybe the,""" start="00:20:18.420" video="qanda-llm" id="subtitle"]] +[[!template text="""oh, GPT for all. So each 1 of these kind of""" start="00:20:21.820" video="qanda-llm" id="subtitle"]] +[[!template text="""has slightly different functionality.""" start="00:20:23.100" video="qanda-llm" id="subtitle"]] +[[!template text="""For example, I think GPT for all doesn't""" start="00:20:26.820" video="qanda-llm" id="subtitle"]] +[[!template text="""support embeddings. And I hear that Olama's""" start="00:20:31.780" video="qanda-llm" id="subtitle"]] +[[!template text="""embeddings are kind of currently broken.""" start="00:20:33.740" video="qanda-llm" id="subtitle"]] +[[!template text="""But basically they should support everything.""" start="00:20:35.920" video="qanda-llm" id="subtitle"]] +[[!template text="""And the open source models are,""" start="00:20:39.100" video="qanda-llm" id="subtitle"]] +[[!template text="""so the local models are reasonably good.""" start="00:20:43.180" video="qanda-llm" id="subtitle"]] +[[!template text="""Like I don't think you'd use them and be""" start="00:20:44.760" video="qanda-llm" id="subtitle"]] +[[!template text="""like, what is this horrible nonsense?""" start="00:20:46.200" video="qanda-llm" id="subtitle"]] +[[!template text="""Like it's, it gives you relatively good""" start="00:20:50.200" video="qanda-llm" id="subtitle"]] +[[!template text="""results. Like it's not gonna be at the level""" start="00:20:51.820" video="qanda-llm" id="subtitle"]] +[[!template text="""of like GPT 3.5 or 4, but it's not far away""" start="00:20:56.060" video="qanda-llm" id="subtitle"]] +[[!template text="""from GPT 3.5, I think.""" start="00:20:57.720" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: I'm just saying that Olam has like a presets""" start="00:21:02.380" video="qanda-llm" id="subtitle"]] +[[!template text="""for connecting the actual working servers for""" start="00:21:05.940" video="qanda-llm" id="subtitle"]] +[[!template text="""Olama?""" start="00:21:06.300" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: So, I'll try. Yeah, so you could,""" start="00:21:08.560" video="qanda-llm" id="subtitle"]] +[[!template text="""what you could do is you could like for""" start="00:21:09.860" video="qanda-llm" id="subtitle"]] +[[!template text="""example you could download Olama which is""" start="00:21:11.940" video="qanda-llm" id="subtitle"]] +[[!template text="""just a way of setting up local models and""" start="00:21:15.780" video="qanda-llm" id="subtitle"]] +[[!template text="""running local models on your machine.""" start="00:21:17.320" video="qanda-llm" id="subtitle"]] +[[!template text="""So typically what it does,""" start="00:21:18.580" video="qanda-llm" id="subtitle"]] +[[!template text="""you like download a program,""" start="00:21:19.720" video="qanda-llm" id="subtitle"]] +[[!template text="""let's say Olama. Then Olama will have the""" start="00:21:23.720" video="qanda-llm" id="subtitle"]] +[[!template text="""ability to download models.""" start="00:21:24.940" video="qanda-llm" id="subtitle"]] +[[!template text="""And so you could choose from just a host of""" start="00:21:27.240" video="qanda-llm" id="subtitle"]] +[[!template text="""different models. Each 1 of these things has""" start="00:21:29.280" video="qanda-llm" id="subtitle"]] +[[!template text="""a bunch of different models.""" start="00:21:30.200" video="qanda-llm" id="subtitle"]] +[[!template text="""So it downloads all these things to your""" start="00:21:31.920" video="qanda-llm" id="subtitle"]] +[[!template text="""machine. But I would say that the key problem""" start="00:21:36.600" video="qanda-llm" id="subtitle"]] +[[!template text="""here is that it requires a fairly beefy""" start="00:21:40.200" video="qanda-llm" id="subtitle"]] +[[!template text="""machine.""" start="00:21:40.580" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: So. Yeah, yeah, of course.""" start="00:21:42.600" video="qanda-llm" id="subtitle"]] +[[!template text="""Why I was asking, because you briefly""" start="00:21:45.060" video="qanda-llm" id="subtitle"]] +[[!template text="""mentioned that there are some Israeli""" start="00:21:46.440" video="qanda-llm" id="subtitle"]] +[[!template text="""servers. I understand that they run it like a""" start="00:21:52.300" video="qanda-llm" id="subtitle"]] +[[!template text="""government or stuff like that?""" start="00:21:53.680" video="qanda-llm" id="subtitle"]] +[[!template text="""No, no, sorry. People want everyone?""" start="00:21:55.440" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I don't, I mean, maybe you've said something""" start="00:21:59.340" video="qanda-llm" id="subtitle"]] +[[!template text="""that sounded like Israeli servers.""" start="00:22:00.620" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Okay, okay.""" start="00:22:01.620" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I think- There's no government LLMs as far as""" start="00:22:04.920" video="qanda-llm" id="subtitle"]] +[[!template text="""I know. Although, I'm sure the governments""" start="00:22:06.820" video="qanda-llm" id="subtitle"]] +[[!template text="""are working on their own LLMs,""" start="00:22:08.200" video="qanda-llm" id="subtitle"]] +[[!template text="""et cetera. But yeah, basically your choices""" start="00:22:10.980" video="qanda-llm" id="subtitle"]] +[[!template text="""are spend a, I mean, if you use open AI or""" start="00:22:15.060" video="qanda-llm" id="subtitle"]] +[[!template text="""something or anything else,""" start="00:22:16.080" video="qanda-llm" id="subtitle"]] +[[!template text="""you're really not spending any money.""" start="00:22:17.960" video="qanda-llm" id="subtitle"]] +[[!template text="""Like I've never been able to spend any money""" start="00:22:20.560" video="qanda-llm" id="subtitle"]] +[[!template text="""on OpenAI. Like unless you're doing something""" start="00:22:23.680" video="qanda-llm" id="subtitle"]] +[[!template text="""very intensive and really are using it to,""" start="00:22:25.840" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, if you're using it for your""" start="00:22:28.000" video="qanda-llm" id="subtitle"]] +[[!template text="""personal use, it's just hard to spend any""" start="00:22:29.620" video="qanda-llm" id="subtitle"]] +[[!template text="""money. But on the other hand,""" start="00:22:31.720" video="qanda-llm" id="subtitle"]] +[[!template text="""it's not free. So you can,""" start="00:22:32.780" video="qanda-llm" id="subtitle"]] +[[!template text="""you know,""" start="00:22:33.040" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Actually, it's rather cheap.""" start="00:22:36.300" video="qanda-llm" id="subtitle"]] +[[!template text="""There's no question about that.""" start="00:22:37.680" video="qanda-llm" id="subtitle"]] +[[!template text="""The problem is that it has a bad track record""" start="00:22:40.580" video="qanda-llm" id="subtitle"]] +[[!template text="""on privacy.""" start="00:22:41.580" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yes, that's, I think that is a key problem.""" start="00:22:45.540" video="qanda-llm" id="subtitle"]] +[[!template text="""This is probably the number 1 reason why you""" start="00:22:48.120" video="qanda-llm" id="subtitle"]] +[[!template text="""might want to use a local AI,""" start="00:22:51.840" video="qanda-llm" id="subtitle"]] +[[!template text="""a local LLM. Another 1 is like,""" start="00:22:54.720" video="qanda-llm" id="subtitle"]] +[[!template text="""you may not agree with the decisions.""" start="00:22:57.400" video="qanda-llm" id="subtitle"]] +[[!template text="""You know, there's a lot of trust and safety""" start="00:23:00.360" video="qanda-llm" id="subtitle"]] +[[!template text="""stuff that these companies have to do.""" start="00:23:05.140" video="qanda-llm" id="subtitle"]] +[[!template text="""Like they don't want like the LMs to kind of""" start="00:23:09.020" video="qanda-llm" id="subtitle"]] +[[!template text="""like give you, like tell you how you can make""" start="00:23:11.400" video="qanda-llm" id="subtitle"]] +[[!template text="""meth or how you can make a bomb,""" start="00:23:13.180" video="qanda-llm" id="subtitle"]] +[[!template text="""which they would do. They would totally do""" start="00:23:14.960" video="qanda-llm" id="subtitle"]] +[[!template text="""it. So, But each time you kind of restrict""" start="00:23:19.580" video="qanda-llm" id="subtitle"]] +[[!template text="""what is happening with what you can get out""" start="00:23:22.540" video="qanda-llm" id="subtitle"]] +[[!template text="""of the LM, it gets a little worse.""" start="00:23:23.860" video="qanda-llm" id="subtitle"]] +[[!template text="""So some people""" start="00:23:24.960" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: want to have local. That's expected.""" start="00:23:27.040" video="qanda-llm" id="subtitle"]] +[[!template text="""I guess even open source language modules""" start="00:23:31.080" video="qanda-llm" id="subtitle"]] +[[!template text="""will soon have HR spaces because it's simply""" start="00:23:33.540" video="qanda-llm" id="subtitle"]] +[[!template text="""a legal issue.""" start="00:23:34.720" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: I think that's true. I also think that there""" start="00:23:40.760" video="qanda-llm" id="subtitle"]] +[[!template text="""probably will be, although I don't know of""" start="00:23:42.880" video="qanda-llm" id="subtitle"]] +[[!template text="""any offhand, that will are completely""" start="00:23:45.060" video="qanda-llm" id="subtitle"]] +[[!template text="""uncensored. I know people are interested and""" start="00:23:46.960" video="qanda-llm" id="subtitle"]] +[[!template text="""are running uncensored models.""" start="00:23:48.240" video="qanda-llm" id="subtitle"]] +[[!template text="""I don't know how to do it.""" start="00:23:49.440" video="qanda-llm" id="subtitle"]] +[[!template text="""I think it's a little bit dubious,""" start="00:23:52.280" video="qanda-llm" id="subtitle"]] +[[!template text="""but some people do want to do it.""" start="00:23:54.960" video="qanda-llm" id="subtitle"]] +[[!template text="""There's another reason for using local""" start="00:23:56.280" video="qanda-llm" id="subtitle"]] +[[!template text="""servers. Do you have any recommendation for""" start="00:24:02.280" video="qanda-llm" id="subtitle"]] +[[!template text="""models to run locally and also comments on""" start="00:24:05.500" video="qanda-llm" id="subtitle"]] +[[!template text="""whether a GPU is required?""" start="00:24:06.780" video="qanda-llm" id="subtitle"]] +[[!template text="""Usually a GPU, well, you can run it without a""" start="00:24:14.040" video="qanda-llm" id="subtitle"]] +[[!template text="""GPU, but it does run much better.""" start="00:24:16.960" video="qanda-llm" id="subtitle"]] +[[!template text="""Like for example, I think when I used,""" start="00:24:19.480" video="qanda-llm" id="subtitle"]] +[[!template text="""Lama is sort of like a standard.""" start="00:24:22.560" video="qanda-llm" id="subtitle"]] +[[!template text="""This was the model for that Facebook came out""" start="00:24:27.160" video="qanda-llm" id="subtitle"]] +[[!template text="""with for local use. And It was,""" start="00:24:31.380" video="qanda-llm" id="subtitle"]] +[[!template text="""yeah, it's good. It's,""" start="00:24:37.260" video="qanda-llm" id="subtitle"]] +[[!template text="""but it's now it's I think,""" start="00:24:40.400" video="qanda-llm" id="subtitle"]] +[[!template text="""Mistral is kind of like has a better""" start="00:24:44.620" video="qanda-llm" id="subtitle"]] +[[!template text="""performance, But there's also different model""" start="00:24:46.480" video="qanda-llm" id="subtitle"]] +[[!template text="""sizes. There's 7B, like the Lama 7B is OK.""" start="00:24:51.000" video="qanda-llm" id="subtitle"]] +[[!template text="""The Mistral 7B, 7 billion,""" start="00:24:52.940" video="qanda-llm" id="subtitle"]] +[[!template text="""are like, basically it'll take like,""" start="00:24:54.800" video="qanda-llm" id="subtitle"]] +[[!template text="""you can run it with like 16 gigs of RAM,""" start="00:24:58.380" video="qanda-llm" id="subtitle"]] +[[!template text="""is pretty good. It's probably about as equal""" start="00:25:02.720" video="qanda-llm" id="subtitle"]] +[[!template text="""to the LLAMA13B. Those are the number of""" start="00:25:06.900" video="qanda-llm" id="subtitle"]] +[[!template text="""parameters, if I remember correctly.""" start="00:25:08.360" video="qanda-llm" id="subtitle"]] +[[!template text="""And then there's a 7B,""" start="00:25:10.680" video="qanda-llm" id="subtitle"]] +[[!template text="""which I've never been able to run.""" start="00:25:12.340" video="qanda-llm" id="subtitle"]] +[[!template text="""And even if the 7B, if you run it without a""" start="00:25:16.120" video="qanda-llm" id="subtitle"]] +[[!template text="""GPU, it takes quite a while to answer.""" start="00:25:19.640" video="qanda-llm" id="subtitle"]] +[[!template text="""I think I've had experiences where it took""" start="00:25:22.080" video="qanda-llm" id="subtitle"]] +[[!template text="""literally like several,""" start="00:25:23.940" video="qanda-llm" id="subtitle"]] +[[!template text="""like 5 minutes before it even started""" start="00:25:26.480" video="qanda-llm" id="subtitle"]] +[[!template text="""responding, but you do eventually get""" start="00:25:28.880" video="qanda-llm" id="subtitle"]] +[[!template text="""something. And it could be that like things""" start="00:25:32.220" video="qanda-llm" id="subtitle"]] +[[!template text="""have gotten better since the last time I""" start="00:25:33.840" video="qanda-llm" id="subtitle"]] +[[!template text="""tried this, because things are moving fast.""" start="00:25:35.440" video="qanda-llm" id="subtitle"]] +[[!template text="""But it is super recommended to have a GPU.""" start="00:25:38.360" video="qanda-llm" id="subtitle"]] +[[!template text="""This is the problem. It's kind of like,""" start="00:25:42.440" video="qanda-llm" id="subtitle"]] +[[!template text="""yes, free software is great.""" start="00:25:43.840" video="qanda-llm" id="subtitle"]] +[[!template text="""But if free software is requiring that you""" start="00:25:46.880" video="qanda-llm" id="subtitle"]] +[[!template text="""have these kind of beefy servers and have all""" start="00:25:50.460" video="qanda-llm" id="subtitle"]] +[[!template text="""this hardware, that's not great.""" start="00:25:52.000" video="qanda-llm" id="subtitle"]] +[[!template text="""I think there's a case to be made.""" start="00:25:53.600" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 1]: a hardware""" start="00:25:55.680" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: with slots instead of a laptop.""" start="00:25:59.040" video="qanda-llm" id="subtitle"]] +[[!template text="""Yeah, yeah, that's right.""" start="00:26:01.560" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 2]: Ideally, you can have Ideally,""" start="00:26:03.660" video="qanda-llm" id="subtitle"]] +[[!template text="""it would be nice if FSL for all things could""" start="00:26:07.400" video="qanda-llm" id="subtitle"]] +[[!template text="""run something for open source model.""" start="00:26:12.040" video="qanda-llm" id="subtitle"]] +[[!template text="""And not free, but the key point is that it's""" start="00:26:16.320" video="qanda-llm" id="subtitle"]] +[[!template text="""Libre?""" start="00:26:16.640" video="qanda-llm" id="subtitle"]] +[[!template text="""[Speaker 0]: Yes, so actually I think Google does do that.""" start="00:26:22.580" video="qanda-llm" id="subtitle"]] +[[!template text="""I'll have to look it up,""" start="00:26:24.720" video="qanda-llm" id="subtitle"]] +[[!template text="""but I haven't explored this yet.""" start="00:26:27.560" video="qanda-llm" id="subtitle"]] +[[!template text="""But Google's server, which LLM does support,""" start="00:26:31.220" video="qanda-llm" id="subtitle"]] +[[!template text="""supports arbitrary models.""" start="00:26:33.800" video="qanda-llm" id="subtitle"]] +[[!template text="""So you can run LLMA or things like that.""" start="00:26:36.420" video="qanda-llm" id="subtitle"]] +[[!template text="""The problem is that even if you're running""" start="00:26:38.940" video="qanda-llm" id="subtitle"]] +[[!template text="""Mistral, which has no restrictions.""" start="00:26:40.320" video="qanda-llm" id="subtitle"]] +[[!template text="""So this is the kind of thing that like the""" start="00:26:42.940" video="qanda-llm" id="subtitle"]] +[[!template text="""Free Software Foundation cares a lot about.""" start="00:26:44.900" video="qanda-llm" id="subtitle"]] +[[!template text="""Like you want it to be like no restrictions,""" start="00:26:47.240" video="qanda-llm" id="subtitle"]] +[[!template text="""legal restrictions on you as you run the""" start="00:26:49.740" video="qanda-llm" id="subtitle"]] +[[!template text="""model. So even if it's running Mistral,""" start="00:26:52.080" video="qanda-llm" id="subtitle"]] +[[!template text="""just by using the server,""" start="00:26:54.800" video="qanda-llm" id="subtitle"]] +[[!template text="""the company server, it will impose some""" start="00:26:58.460" video="qanda-llm" id="subtitle"]] +[[!template text="""restrictions on you probably,""" start="00:26:59.440" video="qanda-llm" id="subtitle"]] +[[!template text="""right? There's gonna be some license that you""" start="00:27:02.320" video="qanda-llm" id="subtitle"]] +[[!template text="""have to, or something you have to abide by.""" start="00:27:04.760" video="qanda-llm" id="subtitle"]] +[[!template text="""So I think, yes, it depends on how much you""" start="00:27:08.480" video="qanda-llm" id="subtitle"]] +[[!template text="""care about it, I guess.""" start="00:27:09.280" video="qanda-llm" id="subtitle"]] +[[!template text="""I should find out more about that and make""" start="00:27:19.500" video="qanda-llm" id="subtitle"]] +[[!template text="""sure that it's a good point that I should,""" start="00:27:21.580" video="qanda-llm" id="subtitle"]] +[[!template text="""you know, people should be able to run free""" start="00:27:23.980" video="qanda-llm" id="subtitle"]] +[[!template text="""models over the server.""" start="00:27:25.920" video="qanda-llm" id="subtitle"]] +[[!template text="""So I should make sure we support that in the""" start="00:27:28.320" video="qanda-llm" id="subtitle"]] +[[!template text="""LLM package. So, is there any other questions""" start="00:27:40.360" video="qanda-llm" id="subtitle"]] +[[!template text="""Or is otherwise we can end the session.""" start="00:27:48.240" video="qanda-llm" id="subtitle"]] +[[!template text="""Yeah, all right. Thank you.""" start="00:28:00.800" video="qanda-llm" id="subtitle"]] +[[!template text="""Thank you. Thank you everyone who listened.""" start="00:28:02.440" video="qanda-llm" id="subtitle"]] +[[!template text="""I'm super happy like I,""" start="00:28:04.540" video="qanda-llm" id="subtitle"]] +[[!template text="""the interest is great.""" start="00:28:06.560" video="qanda-llm" id="subtitle"]] +[[!template text="""I think there's great stuff to be done here""" start="00:28:08.900" video="qanda-llm" id="subtitle"]] +[[!template text="""and I'm kind of super excited what we're""" start="00:28:10.960" video="qanda-llm" id="subtitle"]] +[[!template text="""going to do in the next year,""" start="00:28:11.940" video="qanda-llm" id="subtitle"]] +[[!template text="""so hopefully, like next year,""" start="00:28:13.140" video="qanda-llm" id="subtitle"]] +[[!template text="""and the conference we have something even""" start="00:28:14.600" video="qanda-llm" id="subtitle"]] +[[!template text="""more exciting to say about LLM and how they""" start="00:28:16.440" video="qanda-llm" id="subtitle"]] +[[!template text="""can be used with Emacs.""" start="00:28:17.320" video="qanda-llm" id="subtitle"]] +[[!template text="""So thank""" start="00:28:19.620" video="qanda-llm" id="subtitle"]] +[[!template text="""you""" start="00:28:30.060" video="qanda-llm" id="subtitle"]] + Questions or comments? Please e-mail [ahyatt@gmail.com](mailto:ahyatt@gmail.com?subject=Comment%20for%20EmacsConf%202023%20llm%3A%20LLM%20clients%20in%20Emacs%2C%20functionality%20and%20standardization) |