diff options
Diffstat (limited to '')
| -rw-r--r-- | 2025/captions/emacsconf-2025-private-ai--emacs-and-private-ai-a-great-match--aaron-grothe--main.vtt | 2602 |
1 files changed, 2602 insertions, 0 deletions
diff --git a/2025/captions/emacsconf-2025-private-ai--emacs-and-private-ai-a-great-match--aaron-grothe--main.vtt b/2025/captions/emacsconf-2025-private-ai--emacs-and-private-ai-a-great-match--aaron-grothe--main.vtt new file mode 100644 index 00000000..a8e76986 --- /dev/null +++ b/2025/captions/emacsconf-2025-private-ai--emacs-and-private-ai-a-great-match--aaron-grothe--main.vtt @@ -0,0 +1,2602 @@ +WEBVTT + +00:00:00.000 --> 00:00:04.859 +Hey, everybody. Welcome from frigid Omaha, Nebraska. + +00:00:04.860 --> 00:00:06.619 +I'm just going to kick off my talk here, + +00:00:06.620 --> 00:00:23.899 +and we'll see how it all goes. Thanks for attending. + +00:00:23.900 --> 00:00:26.939 +So the slides will be available on my site, growthy.us, + +00:00:26.940 --> 00:00:29.899 +in the presentation section tonight or tomorrow. + +00:00:29.900 --> 00:00:33.099 +This is a quick intro to one way to do private AI in Emacs. + +00:00:33.100 --> 00:00:35.299 +There are a lot of other ways to do it. + +00:00:35.300 --> 00:00:38.899 +This one is really just more or less the easiest way to do it. + +00:00:38.900 --> 00:00:40.379 +It's a minimal viable product + +00:00:40.380 --> 00:00:42.379 +to get you an idea of how to get started with it + +00:00:42.380 --> 00:00:43.859 +and how to give it a spin. + +00:00:43.860 --> 00:00:45.819 +Really hope some of you give it a shot + +00:00:45.820 --> 00:00:48.179 +and learn something along the way. + +00:00:48.180 --> 00:00:50.379 +So the overview of the talk. + +00:00:50.380 --> 00:00:54.939 +broke down these basic bullet points of why private AI, + +00:00:54.940 --> 00:00:58.939 +what do I need to do private AI, Emacs and private AI, + +00:00:58.940 --> 00:01:02.739 +pieces for an AI Emacs solution, + +00:01:02.740 --> 00:01:08.059 +a demo of a minimal viable product, and the summary. + +00:01:08.060 --> 00:01:10.779 +Why private AI? This is pretty simple. + +00:01:10.780 --> 00:01:12.099 +Just read the terms and conditions + +00:01:12.100 --> 00:01:14.819 +for any AI system you're currently using. + +00:01:14.820 --> 00:01:17.019 +If you're using the free tiers, your queries, + +00:01:17.020 --> 00:01:18.619 +code uploaded information + +00:01:18.620 --> 00:01:20.699 +is being used to train the models. + +00:01:20.700 --> 00:01:22.939 +In some cases, you are giving the company + +00:01:22.940 --> 00:01:25.419 +a perpetual license to your data. + +00:01:25.420 --> 00:01:27.059 +You have no control over this, + +00:01:27.060 --> 00:01:29.219 +except for not using the engine. + +00:01:29.220 --> 00:01:30.699 +And keep in mind, the terms + +00:01:30.700 --> 00:01:32.179 +are changing all the time on that, + +00:01:32.180 --> 00:01:34.139 +and they're not normally changing for our benefit. + +00:01:34.140 --> 00:01:38.259 +So that's not necessarily a good thing. + +00:01:38.260 --> 00:01:40.339 +If you're using the paid tiers, + +00:01:40.340 --> 00:01:43.459 +you may be able to opt out of the data collection. + +00:01:43.460 --> 00:01:45.539 +But keep in mind, this can change, + +00:01:45.540 --> 00:01:48.619 +or they may start charging for that option. + +00:01:48.620 --> 00:01:51.419 +Every AI company wants more and more data. + +00:01:51.420 --> 00:01:53.779 +They need more and more data to train their models. + +00:01:53.780 --> 00:01:56.019 +It is just the way it is. + +00:01:56.020 --> 00:01:57.899 +They need more and more information + +00:01:57.900 --> 00:02:00.459 +to get it more and more accurate to keep it up to date. + +00:02:00.460 --> 00:02:03.219 +There's been a story about Stack Overflow. + +00:02:03.220 --> 00:02:05.819 +It has like half the number of queries they had a year ago + +00:02:05.820 --> 00:02:07.379 +because people are using AI. + +00:02:07.380 --> 00:02:08.579 +The problem with that is now + +00:02:08.580 --> 00:02:10.379 +there's less data going to Stack Overflow + +00:02:10.380 --> 00:02:12.979 +for the AI to get. vicious cycle, + +00:02:12.980 --> 00:02:14.619 +especially when you start looking at + +00:02:14.620 --> 00:02:16.579 +newer language like Ruby and stuff like that. + +00:02:16.580 --> 00:02:21.419 +So it comes down to being an interesting time. + +00:02:21.420 --> 00:02:24.739 +Another reason why to go private AI is your costs are going to vary. + +00:02:24.740 --> 00:02:27.019 +Right now, these services are being heavily subsidized. + +00:02:27.020 --> 00:02:29.419 +If you're paying Claude $20 a month, + +00:02:29.420 --> 00:02:32.579 +it is not costing Claude, those guys $20 a month + +00:02:32.580 --> 00:02:34.099 +to host all the infrastructure + +00:02:34.100 --> 00:02:35.619 +to build all these data centers. + +00:02:35.620 --> 00:02:38.779 +They are severely subsidizing that + +00:02:38.780 --> 00:02:41.259 +at a very much a loss right now. + +00:02:41.260 --> 00:02:43.659 +When they start charging the real costs plus a profit, + +00:02:43.660 --> 00:02:45.499 +it's going to change. + +00:02:45.500 --> 00:02:48.019 +Right now, I use a bunch of different services. + +00:02:48.020 --> 00:02:50.019 +I've played with Grok and a bunch of other ones. + +00:02:50.020 --> 00:02:52.459 +But Grok right now is like $30 a month + +00:02:52.460 --> 00:02:54.139 +for a regular Super Grok. + +00:02:54.140 --> 00:02:56.419 +When they start charging the real cost of that, + +00:02:56.420 --> 00:02:59.819 +it's going to go from $30 to something a great deal more, + +00:02:59.820 --> 00:03:02.379 +perhaps, I think, $100 or $200 + +00:03:02.380 --> 00:03:04.459 +or whatever really turns out to be the cost + +00:03:04.460 --> 00:03:06.059 +when you figure everything into it. + +00:03:06.060 --> 00:03:07.539 +When you start adding that cost into that, + +00:03:07.540 --> 00:03:10.179 +a lot of people are using public AI right now + +00:03:10.180 --> 00:03:11.899 +are going to have no option but to move to private AI + +00:03:11.900 --> 00:03:16.019 +or give up on AI overall. + +00:03:16.020 --> 00:03:18.659 +What do you need to be able to do private AI? + +00:03:18.660 --> 00:03:21.179 +If you're going to run your own AI, + +00:03:21.180 --> 00:03:23.579 +you're going to need a system with either some cores, + +00:03:23.580 --> 00:03:25.699 +a graphics processor unit, + +00:03:25.700 --> 00:03:28.339 +or a neural processing unit, a GPU or an NPU. + +00:03:28.340 --> 00:03:29.819 +I currently have four systems + +00:03:29.820 --> 00:03:32.979 +I'm experimenting with and playing around with on a daily basis. + +00:03:32.980 --> 00:03:37.979 +I have a System76 Pangolin AMD Ryzen 7 78040U + +00:03:37.980 --> 00:03:41.099 +with a Radeon 7080M integrated graphics card. + +00:03:41.100 --> 00:03:42.539 +It's got 32 gigs of RAM. + +00:03:42.540 --> 00:03:45.259 +It's a beautiful piece of hardware. I really do like it. + +00:03:45.260 --> 00:03:46.499 +I have my main workstation, + +00:03:46.500 --> 00:03:50.579 +it's an HP Z620 with dual Intel Xeons + +00:03:50.580 --> 00:03:53.179 +with four NVIDIA K2200 graphics cards in it. + +00:03:53.180 --> 00:03:56.699 +Why the four NVIDIA K2200 graphics card on it? + +00:03:56.700 --> 00:03:59.739 +Because I could buy four of them on eBay for $100 + +00:03:59.740 --> 00:04:02.379 +and it was still supported by the NVIDIA drivers for Debian. + +00:04:02.380 --> 00:04:08.179 +So that's why that is. A MacBook Air with an M1 processor, + +00:04:08.180 --> 00:04:10.939 +a very nice piece of kit I picked up a couple years ago, + +00:04:10.940 --> 00:04:14.139 +very cheap, but it runs AI surprisingly well, + +00:04:14.140 --> 00:04:18.099 +and an Acer Aspire 1 with an AMD Ryzen 5700H in it. + +00:04:18.100 --> 00:04:22.099 +This was my old laptop. It was a sturdy beast. + +00:04:22.100 --> 00:04:24.379 +It was able to do enough AI to do demos and stuff, + +00:04:24.380 --> 00:04:25.859 +and I liked it quite a bit for that. + +00:04:25.860 --> 00:04:28.339 +I'm using the Pangolin for this demonstration + +00:04:28.340 --> 00:04:30.979 +because it's just better. + +00:04:30.980 --> 00:04:37.219 +Apple's M4 chip has 38 teraflops of MPU performance. + +00:04:37.220 --> 00:04:40.099 +The Microsoft co-pilots are now requiring + +00:04:40.100 --> 00:04:41.459 +45 teraflops of MPU + +00:04:41.460 --> 00:04:43.939 +to be able to have the co-pilot badge on it. + +00:04:43.940 --> 00:04:48.299 +And Raspberry Pi's new AI top is about 18 teraflops + +00:04:48.300 --> 00:04:51.219 +and is $70 on top of the cost of Raspberry Pi 5. + +00:04:51.220 --> 00:04:56.059 +Keep in mind Raspberry recently + +00:04:56.060 --> 00:04:59.499 +raised the cost of their Pi 5s because of RAM pricing, + +00:04:59.500 --> 00:05:00.379 +which is going to be affecting + +00:05:00.380 --> 00:05:02.459 +a lot of these types of solutions in the near future. + +00:05:02.460 --> 00:05:05.299 +But there's going to be a lot of + +00:05:05.300 --> 00:05:06.699 +local power available in the future. + +00:05:06.700 --> 00:05:08.219 +That's what it really comes down to. + +00:05:08.220 --> 00:05:11.179 +A lot of people are going to have PCs on their desks. + +00:05:11.180 --> 00:05:13.459 +They're going to run a decent private AI + +00:05:13.460 --> 00:05:18.059 +without much issue. So for Emacs and private AI, + +00:05:18.060 --> 00:05:20.139 +there's a couple popular solutions. + +00:05:20.140 --> 00:05:22.099 +Gptel, which is the one we're going to talk about. + +00:05:22.100 --> 00:05:24.739 +It's a simple interface. It's a minimal interface. + +00:05:24.740 --> 00:05:26.579 +It integrates easily into your workflow. + +00:05:26.580 --> 00:05:29.019 +It's just, quite honestly, chef's kiss, + +00:05:29.020 --> 00:05:31.059 +just a beautifully well-done piece of software. + +00:05:31.060 --> 00:05:33.859 +OllamaBuddy has more features, + +00:05:33.860 --> 00:05:36.259 +a menu interface, has quick access + +00:05:36.260 --> 00:05:37.499 +for things like code refactoring, + +00:05:37.500 --> 00:05:38.979 +text-free formatting, et cetera. + +00:05:38.980 --> 00:05:41.979 +This is the one that you spend a little more time with, + +00:05:41.980 --> 00:05:43.939 +but you also get a little bit more back from it. + +00:05:43.940 --> 00:05:49.419 +Elama is another one, has some really good features to it, + +00:05:49.420 --> 00:05:51.059 +more different capabilities, + +00:05:51.060 --> 00:05:54.979 +but it's a different set of rules and capabilities to it. + +00:05:54.980 --> 00:05:59.179 +Itermac, which is programming with your AI and Emacs. + +00:05:59.180 --> 00:06:01.219 +The closest thing I can come up + +00:06:01.220 --> 00:06:04.139 +to comparing this to is Cursor, except it's an Emacs. + +00:06:04.140 --> 00:06:05.659 +It's really quite well done. + +00:06:05.660 --> 00:06:07.299 +These are all really quite well done. + +00:06:07.300 --> 00:06:08.499 +There's a bunch of other projects out there. + +00:06:08.500 --> 00:06:10.819 +If you go out to GitHub, type Emacs AI, + +00:06:10.820 --> 00:06:13.219 +you'll find a lot of different options. + +00:06:13.220 --> 00:06:18.459 +So what is a minimal viable product that can be done? + +00:06:18.460 --> 00:06:23.379 +A minimal viable product to show what an AI Emacs solution is + +00:06:23.380 --> 00:06:27.179 +can be done with only needing two pieces of software. + +00:06:27.180 --> 00:06:31.179 +Llamafile, this is an amazing piece of software. + +00:06:31.180 --> 00:06:32.899 +This is a whole LLM contained in one file. + +00:06:32.900 --> 00:06:36.059 +And the same file runs on Mac OS X, + +00:06:36.060 --> 00:06:39.379 +Linux, Windows, and the BSDs. + +00:06:39.380 --> 00:06:42.179 +It's a wonderful piece of kit + +00:06:42.180 --> 00:06:44.179 +based on these people who created + +00:06:44.180 --> 00:06:45.899 +this thing called Cosmopolitan + +00:06:45.900 --> 00:06:46.779 +that lets you create and execute + +00:06:46.780 --> 00:06:48.699 +while it runs on a bunch of different systems. + +00:06:48.700 --> 00:06:51.299 +And Gptel, which is an easy plug-in for Emacs, + +00:06:51.300 --> 00:06:54.979 +which we talked about in the last slide a bit. + +00:06:54.980 --> 00:07:00.179 +So setting up the LLM, you have to just go out + +00:07:00.180 --> 00:07:01.699 +and just hit the a page for it + +00:07:01.700 --> 00:07:05.099 +and go out and do a wget of it. + +00:07:05.100 --> 00:07:07.099 +That's all it takes there. + +00:07:07.100 --> 00:07:10.259 +Chmodding it so you can actually execute the executable. + +00:07:10.260 --> 00:07:12.939 +And then just go ahead and actually running it. + +00:07:12.940 --> 00:07:16.939 +And let's go ahead and do that. + +00:07:16.940 --> 00:07:18.899 +I've already downloaded it because I don't want to wait. + +00:07:18.900 --> 00:07:21.259 +And let's just take a look at it. + +00:07:21.260 --> 00:07:22.899 +I've actually downloaded several of them, + +00:07:22.900 --> 00:07:25.699 +but let's go ahead and just run lava 3.2b + +00:07:25.700 --> 00:07:31.179 +with the 3 billion instructions. And that's it firing up. + +00:07:31.180 --> 00:07:33.899 +And it is nice enough to actually be listening in port 8080, + +00:07:33.900 --> 00:07:35.339 +which we'll need in a minute. + +00:07:35.340 --> 00:07:43.139 +So once you do that, you have to install gptel and emacs. + +00:07:43.140 --> 00:07:45.659 +That's as simple as firing up emacs, + +00:07:45.660 --> 00:07:48.339 +doing the meta x install package, + +00:07:48.340 --> 00:07:49.779 +and then just typing gptel + +00:07:49.780 --> 00:07:51.499 +if you have your repository set up right, + +00:07:51.500 --> 00:07:52.299 +which hopefully you do. + +00:07:52.300 --> 00:07:54.499 +And then you just go ahead and have it. + +00:07:54.500 --> 00:07:58.139 +You also have to set up a config file. + +00:07:58.140 --> 00:08:01.739 +Here's my example config file as it currently set up, + +00:08:01.740 --> 00:08:04.019 +requiring ensuring Gptel is loaded, + +00:08:04.020 --> 00:08:05.899 +defining the Llamafile backend. + +00:08:05.900 --> 00:08:07.779 +You can put multiple backends into it, + +00:08:07.780 --> 00:08:09.859 +but I just have the one defined on this example. + +00:08:09.860 --> 00:08:12.059 +But it's pretty straightforward. + +00:08:12.060 --> 00:08:16.739 +Llama local file, name for it, stream, protocol HTTP. + +00:08:16.740 --> 00:08:20.859 +If you have HTTPS set up, that's obviously preferable, + +00:08:20.860 --> 00:08:22.779 +but a lot of people don't for their home labs. + +00:08:22.780 --> 00:08:26.379 +Host is just 127.0.0.1 port 8080. + +00:08:26.380 --> 00:08:30.099 +Keep in mind, some of the AIs run on a different port, + +00:08:30.100 --> 00:08:31.499 +so you may be 8081 + +00:08:31.500 --> 00:08:34.619 +if you're running OpenWebView at the same time. The key, + +00:08:34.620 --> 00:08:37.019 +we don't need an API key because it's a local server. + +00:08:37.020 --> 00:08:40.259 +And the models just, uh, we can put multiple models + +00:08:40.260 --> 00:08:41.339 +on there if we want to. + +00:08:41.340 --> 00:08:43.699 +So if we create one with additional stuff + +00:08:43.700 --> 00:08:45.379 +or like rag and stuff like that, + +00:08:45.380 --> 00:08:47.459 +we can actually name those models by their domain, + +00:08:47.460 --> 00:08:48.699 +which is really kind of cool. + +00:08:48.700 --> 00:08:52.099 +But, uh, that's all that takes. + +00:08:52.100 --> 00:09:03.779 +So let's go ahead and go to a quick test of it. + +00:09:03.780 --> 00:09:11.019 +Oops. Alt-X, gptel. And we're going to just choose + +00:09:11.020 --> 00:09:12.499 +the default buffer to make things easier. + +00:09:12.500 --> 00:09:15.339 +Going to resize it up a bit. + +00:09:15.340 --> 00:09:19.859 +And usually the go-to question I go to is, who was David Bowie? + +00:09:19.860 --> 00:09:24.499 +This one is actually a question + +00:09:24.500 --> 00:09:26.219 +that's turned out to be really good + +00:09:26.220 --> 00:09:28.019 +for figuring out whether or not AI is complete. + +00:09:28.020 --> 00:09:31.139 +This is one that some engines do well on, other ones don't. + +00:09:31.140 --> 00:09:33.739 +And we can just do, we can either do + +00:09:33.740 --> 00:09:36.059 +the alt X and send the gptel-send, + +00:09:36.060 --> 00:09:37.979 +or we can just do control C and hit enter. + +00:09:37.980 --> 00:09:39.139 +We'll just do control C and enter. + +00:09:39.140 --> 00:09:43.659 +And now it's going ahead and hitting our local AI system + +00:09:43.660 --> 00:09:46.659 +running on port 8080. And that looks pretty good, + +00:09:46.660 --> 00:09:50.739 +but let's go ahead and say, hey, it's set to terse mode right now. + +00:09:50.740 --> 00:10:03.859 +Please expand upon this. And there we go. + +00:10:03.860 --> 00:10:05.379 +We're getting a full description + +00:10:05.380 --> 00:10:08.739 +of the majority of, uh, about David Bowie's life + +00:10:08.740 --> 00:10:10.139 +and other information about him. + +00:10:10.140 --> 00:10:21.699 +So very, very happy with that. + +00:10:21.700 --> 00:10:23.539 +One thing to keep in mind is you look at things + +00:10:23.540 --> 00:10:24.699 +when you're looking for hallucinations, + +00:10:24.700 --> 00:10:26.899 +how accurate AI is, how it's compressed + +00:10:26.900 --> 00:10:29.259 +is it will tend to screw up on things like + +00:10:29.260 --> 00:10:30.859 +how many children he had and stuff like that. + +00:10:30.860 --> 00:10:32.459 +Let me see if it gets to that real quick. + +00:10:32.460 --> 00:10:39.739 +Is it not actually on this one? + +00:10:39.740 --> 00:10:42.179 +Alright, so that's the first question I always ask one. + +00:10:42.180 --> 00:10:44.659 +The next one is what are sea monkeys? + +00:10:44.660 --> 00:10:48.979 +It gives you an idea of the breadth of the system. + +00:10:48.980 --> 00:11:10.619 +It's querying right now. Pulls it back correctly. Yes. + +00:11:10.620 --> 00:11:12.339 +And it's smart enough to actually detect David Bowie + +00:11:12.340 --> 00:11:15.019 +even referenced see monkeys in the song sea of love, + +00:11:15.020 --> 00:11:16.179 +which came at hit single. + +00:11:16.180 --> 00:11:18.859 +So it's actually keeping the context alive + +00:11:18.860 --> 00:11:20.419 +and that which is very cool feature. + +00:11:20.420 --> 00:11:21.459 +I did not see that coming. + +00:11:21.460 --> 00:11:24.139 +Here's one that some people say is a really good one + +00:11:24.140 --> 00:11:25.739 +to ask ours in strawberry. + +00:11:25.740 --> 00:11:46.179 +All right, now she's going off the reservation. + +00:11:46.180 --> 00:11:48.139 +She's going in a different direction. + +00:11:48.140 --> 00:11:49.979 +Let me go ahead and reopen that again, + +00:11:49.980 --> 00:11:52.979 +because it's went down a bad hole there for a second. + +00:11:52.980 --> 00:11:58.419 +Let me ask it to do write hello world in Emacs list. + +00:11:58.420 --> 00:12:10.419 +Yep, that works. So the point being here, + +00:12:10.420 --> 00:12:14.939 +that was like two minutes of setup. + +00:12:14.940 --> 00:12:18.019 +And now we have a small AI embedded inside the system. + +00:12:18.020 --> 00:12:20.539 +So that gives you an idea just how easy it can be. + +00:12:20.540 --> 00:12:22.299 +And it's just running locally on the system. + +00:12:22.300 --> 00:12:25.259 +We also have the default system here as well. + +00:12:25.260 --> 00:12:32.579 +So not that bad. + +00:12:32.580 --> 00:12:35.379 +That's a basic solution, that's a basic setup + +00:12:35.380 --> 00:12:37.059 +that will get you to the point where you can go like, + +00:12:37.060 --> 00:12:39.859 +it's a party trick, but it's a very cool party trick. + +00:12:39.860 --> 00:12:42.859 +The way that Gptel works is it puts it into buffers, + +00:12:42.860 --> 00:12:45.099 +it doesn't interfere with your flow that much, + +00:12:45.100 --> 00:12:47.179 +it's just an additional window you can pop open + +00:12:47.180 --> 00:12:49.019 +to ask questions and get information for, + +00:12:49.020 --> 00:12:51.459 +dump code into it and have it refactored. + +00:12:51.460 --> 00:12:53.339 +Gptel has a lot of additional options + +00:12:53.340 --> 00:12:55.699 +for things that are really cool for that. + +00:12:55.700 --> 00:12:57.099 +But if you want a better solution, + +00:12:57.100 --> 00:12:59.939 +I recommend Ollama or LM Studio. + +00:12:59.940 --> 00:13:01.899 +They're both more capable than llama file. + +00:13:01.900 --> 00:13:03.859 +They can accept a lot of different models. + +00:13:03.860 --> 00:13:05.739 +You can do things like RAG. + +00:13:05.740 --> 00:13:09.219 +You can do loading of things onto the GPU more explicitly. + +00:13:09.220 --> 00:13:10.379 +It can speed stuff up. + +00:13:10.380 --> 00:13:13.059 +One of the things about the retrieval augmentation is + +00:13:13.060 --> 00:13:15.539 +it will let you put your data into the system + +00:13:15.540 --> 00:13:17.779 +so you can start uploading your code, your information, + +00:13:17.780 --> 00:13:20.139 +and actually being able to do analysis of it. + +00:13:20.140 --> 00:13:23.539 +OpenWebUI provides more capabilities. + +00:13:23.540 --> 00:13:24.859 +It provides an interface that's similar + +00:13:24.860 --> 00:13:25.899 +to what you're used to seeing + +00:13:25.900 --> 00:13:28.179 +for chat, GPT, and the other systems. + +00:13:28.180 --> 00:13:29.419 +It's really quite well done. + +00:13:29.420 --> 00:13:32.539 +And once again, gptel, I have to mention that + +00:13:32.540 --> 00:13:34.779 +because that's the one I really kind of like. + +00:13:34.780 --> 00:13:36.899 +And OlamaBuddy is also another really nice one. + +00:13:36.900 --> 00:13:41.019 +So what about the licensing of these models? + +00:13:41.020 --> 00:13:42.299 +Since I'm going out pulling down + +00:13:42.300 --> 00:13:43.579 +a model and doing this stuff. + +00:13:43.580 --> 00:13:46.579 +Let's take a look at a couple of highlights + +00:13:46.580 --> 00:13:49.379 +from the MetaLlama 3 community license scale. + +00:13:49.380 --> 00:13:52.579 +If your service exceeds 700 million monthly users, + +00:13:52.580 --> 00:13:54.099 +you need additional licensing. + +00:13:54.100 --> 00:13:56.099 +Probably not going to be a problem for most of us. + +00:13:56.100 --> 00:13:58.379 +There's a competition restriction. + +00:13:58.380 --> 00:14:00.899 +You can't use this model to enhance competing models. + +00:14:00.900 --> 00:14:04.219 +And there's some limitations on using the Meta trademarks. + +00:14:04.220 --> 00:14:05.939 +Not that big a deal. + +00:14:05.940 --> 00:14:09.139 +And the other ones are it's a permissive one + +00:14:09.140 --> 00:14:10.939 +designed to encourage innovation, + +00:14:10.940 --> 00:14:13.779 +open development, commercial use is allowed, + +00:14:13.780 --> 00:14:15.219 +but there are some restrictions on it. + +00:14:15.220 --> 00:14:17.259 +Yeah, you can modify the model, + +00:14:17.260 --> 00:14:20.419 +but you have to rely on the license terms. + +00:14:20.420 --> 00:14:22.339 +And you can distribute the model with derivatives. + +00:14:22.340 --> 00:14:24.059 +And there are some very cool ones out there. + +00:14:24.060 --> 00:14:25.259 +There's people who've done things + +00:14:25.260 --> 00:14:29.579 +to try and make the llama bee less, what's the phrase, + +00:14:29.580 --> 00:14:31.939 +ethical if you're doing penetration testing research + +00:14:31.940 --> 00:14:32.619 +and stuff like that. + +00:14:32.620 --> 00:14:34.459 +It has some very nice value there. + +00:14:34.460 --> 00:14:37.739 +Keep in mind licenses also vary + +00:14:37.740 --> 00:14:39.619 +depending on the model you're using. + +00:14:39.620 --> 00:14:42.419 +Mistral AI has the non-production license. + +00:14:42.420 --> 00:14:45.219 +It's designed to keep it to research and development. + +00:14:45.220 --> 00:14:46.739 +You can't use it commercially. + +00:14:46.740 --> 00:14:50.419 +So it's designed to clearly delineate + +00:14:50.420 --> 00:14:52.939 +between research and development + +00:14:52.940 --> 00:14:54.259 +and somebody trying to actually build + +00:14:54.260 --> 00:14:55.379 +something on top of it. + +00:14:55.380 --> 00:14:57.979 +And another question I get asked is, + +00:14:57.980 --> 00:14:59.899 +are there open source data model options? + +00:14:59.900 --> 00:15:02.819 +Yeah, but most of them are small or specialized currently. + +00:15:02.820 --> 00:15:05.499 +MoMo is a whole family of them, + +00:15:05.500 --> 00:15:07.339 +but there tend to be more specialized, + +00:15:07.340 --> 00:15:09.019 +but it's very cool to see where it's going. + +00:15:09.020 --> 00:15:11.339 +And it's another thing that's just going forward. + +00:15:11.340 --> 00:15:13.379 +It's under the MIT license. + +00:15:13.380 --> 00:15:15.819 +Some things to know to help you + +00:15:15.820 --> 00:15:17.499 +have a better experience with this. + +00:15:17.500 --> 00:15:21.059 +Get a Llama and OpenWebUI working by themselves, + +00:15:21.060 --> 00:15:22.659 +then set up your config file. + +00:15:22.660 --> 00:15:24.819 +I was fighting both at the same time, + +00:15:24.820 --> 00:15:26.699 +and it turned out I had a problem with my LLAMA. + +00:15:26.700 --> 00:15:28.899 +I had a conflict, so that was what my problem is. + +00:15:28.900 --> 00:15:32.819 +Llamafile, gptel is a great way to start experimenting + +00:15:32.820 --> 00:15:34.299 +just to get you an idea of how it works + +00:15:34.300 --> 00:15:36.939 +and figure out how the interfaces work. Tremendous. + +00:15:36.940 --> 00:15:40.739 +RAG loading documents into it is really easy with open web UI. + +00:15:40.740 --> 00:15:43.019 +You can create models, you can put things like + +00:15:43.020 --> 00:15:46.419 +help desk developers and stuff like that, breaking it out. + +00:15:46.420 --> 00:15:51.019 +The Hacker News has a how to build a $300 AI computer. + +00:15:51.020 --> 00:15:52.859 +This is for March 2024, + +00:15:52.860 --> 00:15:55.099 +but it still has a lot of great information + +00:15:55.100 --> 00:15:56.819 +on how to benchmark the environments, + +00:15:56.820 --> 00:16:01.339 +what some values are like the Ryzen 5700U + +00:16:01.340 --> 00:16:02.579 +inside my Acer Aspire, + +00:16:02.580 --> 00:16:04.419 +that's where I got the idea doing that. + +00:16:04.420 --> 00:16:06.739 +Make sure you do the RockM stuff correctly + +00:16:06.740 --> 00:16:09.899 +to get the GUI extensions. But it's just really good stuff. + +00:16:09.900 --> 00:16:13.059 +You don't need a great GPU or CPU to get started. + +00:16:13.060 --> 00:16:14.819 +Smaller models like Tiny Llama + +00:16:14.820 --> 00:16:16.179 +can run on very small systems. + +00:16:16.180 --> 00:16:18.499 +It gets you the ability to start playing with it + +00:16:18.500 --> 00:16:21.619 +and start experimenting and figure out if that's for you + +00:16:21.620 --> 00:16:23.379 +and to move forward with it. + +00:16:23.380 --> 00:16:29.219 +The AMD Ryzen AI Max 395 plus is a mini PC + +00:16:29.220 --> 00:16:31.179 +makes it really nice dedicated host. + +00:16:31.180 --> 00:16:34.619 +You used to be able to buy these for about $1200 now + +00:16:34.620 --> 00:16:35.579 +with the RAM price increase, + +00:16:35.580 --> 00:16:38.779 +you want to get 120 gig when you're pushing two brands so. + +00:16:38.780 --> 00:16:40.739 +It gets a little tighter. + +00:16:40.740 --> 00:16:44.099 +Macs work remarkably well with AI. + +00:16:44.100 --> 00:16:47.659 +My MacBook Air was one of my go-tos for a while, + +00:16:47.660 --> 00:16:49.779 +but once I started doing anything AI, + +00:16:49.780 --> 00:16:50.779 +I had a five-minute window + +00:16:50.780 --> 00:16:52.619 +before the thermal throttling became an issue. + +00:16:52.620 --> 00:16:54.619 +Keep in mind that's a MacBook Air, + +00:16:54.620 --> 00:16:56.659 +so it doesn't have the greatest ventilation. + +00:16:56.660 --> 00:16:58.339 +If you get the MacBook Pros and stuff, + +00:16:58.340 --> 00:17:00.139 +they tend to have more ventilation, + +00:17:00.140 --> 00:17:02.499 +but still you're going to be pushing against that. + +00:17:02.500 --> 00:17:04.939 +So Mac Minis and the Mac Ultras and stuff like that + +00:17:04.940 --> 00:17:06.099 +tend to work really well for that. + +00:17:06.100 --> 00:17:09.779 +Alex Ziskin on YouTube has a channel. + +00:17:09.780 --> 00:17:11.899 +He does a lot of AI performance benchmarking, + +00:17:11.900 --> 00:17:14.819 +like I load a 70 billion parameter model + +00:17:14.820 --> 00:17:16.699 +on this mini PC and stuff like that. + +00:17:16.700 --> 00:17:19.019 +It's a lot of fun and interesting stuff there. + +00:17:19.020 --> 00:17:21.219 +And it's influencing my decision + +00:17:21.220 --> 00:17:22.979 +to buy my next AI style PC. + +00:17:22.980 --> 00:17:27.619 +Small domain specific LLMs are happening. + +00:17:27.620 --> 00:17:29.939 +An LLM that has all your code and information, + +00:17:29.940 --> 00:17:31.659 +it sounds like a really cool idea. + +00:17:31.660 --> 00:17:34.299 +It gives you capabilities to start training stuff + +00:17:34.300 --> 00:17:35.899 +that you couldn't do with like the big ones. + +00:17:35.900 --> 00:17:38.059 +Even with in terms of fine tuning and stuff, + +00:17:38.060 --> 00:17:40.539 +it's remarkable to see where that space is coming along + +00:17:40.540 --> 00:17:41.739 +in the next year or so. + +00:17:41.740 --> 00:17:46.219 +Hugging Face Co has pointers to tons of AI models. + +00:17:46.220 --> 00:17:49.259 +You'll find the one that works for you, hopefully there. + +00:17:49.260 --> 00:17:50.539 +If you're doing cybersecurity, + +00:17:50.540 --> 00:17:52.059 +there's a whole bunch out there for that, + +00:17:52.060 --> 00:17:54.619 +that have certain training on it, information. + +00:17:54.620 --> 00:17:56.139 +It's really good. + +00:17:56.140 --> 00:18:00.099 +One last thing to keep in mind is hallucinations are real. + +00:18:00.100 --> 00:18:02.779 +You will get BS back from the AI occasionally, + +00:18:02.780 --> 00:18:05.179 +so do validate everything you get from it. + +00:18:05.180 --> 00:18:08.459 +Don't be using it for court cases like some people have + +00:18:08.460 --> 00:18:14.539 +and run into those problems. So, That is my talk. + +00:18:14.540 --> 00:18:17.219 +What I would like you to get out of that is, + +00:18:17.220 --> 00:18:21.859 +if you haven't tried it, give GPTEL and LlamaFile a shot. + +00:18:21.860 --> 00:18:23.979 +Fire up a little small AI instance, + +00:18:23.980 --> 00:18:27.339 +play around with a little bit inside your Emacs, + +00:18:27.340 --> 00:18:30.139 +and see if it makes your life better. Hopefully it will. + +00:18:30.140 --> 00:18:32.139 +And I really hope you guys + +00:18:32.140 --> 00:18:34.659 +learned something from this talk. And thanks for listening. + +00:18:34.660 --> 00:18:38.979 +And the links are at the end of the talk, if you have any questions. + +00:18:38.980 --> 00:18:42.739 +Let me see if we got anything you want, Pat. You do. + +00:18:42.740 --> 00:18:43.899 +You've got a few questions. + +00:18:43.900 --> 00:18:48.059 +Hey, this is Corwin. Thank you so much. Thank you, Aaron. + +00:18:48.060 --> 00:18:50.339 +What an awesome talk this was, actually. + +00:18:50.340 --> 00:18:52.179 +If you don't have a camera, + +00:18:52.180 --> 00:18:54.339 +I can get away with not having one too. + +00:18:54.340 --> 00:18:56.299 +I've got, I'll turn the camera on. + +00:18:56.300 --> 00:19:01.499 +Okay. All right. I'll turn mine back on. Here I come. + +00:19:01.500 --> 00:19:03.139 +Yeah, so there are a few questions, + +00:19:03.140 --> 00:19:04.579 +but first let me say thank you + +00:19:04.580 --> 00:19:06.339 +for a really captivating talk. + +00:19:06.340 --> 00:19:10.939 +I think a lot of people will be empowered from this + +00:19:10.940 --> 00:19:15.259 +to try to do more with less, especially locally. + +00:19:15.260 --> 00:19:20.179 +concerned about the data center footprint, + +00:19:20.180 --> 00:19:23.659 +environmentally concerned + +00:19:23.660 --> 00:19:26.979 +about the footprint of LLM inside data centers. + +00:19:26.980 --> 00:19:28.219 +So just thinking about how we can + +00:19:28.220 --> 00:19:32.419 +put infrastructure we have at home to use + +00:19:32.420 --> 00:19:34.019 +and get more done with less. + +00:19:34.020 --> 00:19:37.499 +Yeah, the data center impact's interesting + +00:19:37.500 --> 00:19:39.979 +because there was a study a while ago. + +00:19:39.980 --> 00:19:42.099 +Someone said every time you do a Gemini query, + +00:19:42.100 --> 00:19:45.019 +it's like boiling a cup of water. + +00:19:45.020 --> 00:19:48.619 +Yeah, I've heard that one too. So do you want to, you know, + +00:19:48.620 --> 00:19:51.699 +I don't know how much direction you want. + +00:19:51.700 --> 00:19:53.859 +I'd be very happy to read out the questions for you. + +00:19:53.860 --> 00:19:55.219 +Yeah, that would be great. + +00:19:55.220 --> 00:19:57.619 +I'm having trouble getting to that tab. + +00:19:57.620 --> 00:20:02.779 +Okay, I'm there, so I'll put it into our chat too, + +00:20:02.780 --> 00:20:07.419 +so you can follow along if you'd like. + +00:20:07.420 --> 00:20:11.219 +The first question was, why is the David Bowie question + +00:20:11.220 --> 00:20:12.219 +a good one to start with? + +00:20:12.220 --> 00:20:14.419 +Does it have interesting failure conditions + +00:20:14.420 --> 00:20:17.299 +or what made you choose that? + +00:20:17.300 --> 00:20:21.979 +First off, huge fan of David Bowie. + +00:20:21.980 --> 00:20:24.499 +But I came down to it really taught me a few things + +00:20:24.500 --> 00:20:26.299 +about how old the models work + +00:20:26.300 --> 00:20:28.819 +in terms of things like how many kids he had, + +00:20:28.820 --> 00:20:31.779 +because deep seek, which is a very popular Chinese model + +00:20:31.780 --> 00:20:33.179 +that a lot of people are using now, + +00:20:33.180 --> 00:20:35.619 +misidentifies him having three daughters, + +00:20:35.620 --> 00:20:38.459 +and he has like one son and one, one, I think, + +00:20:38.460 --> 00:20:40.899 +two sons and a daughter or something like that. + +00:20:40.900 --> 00:20:43.659 +so there's differences on that and it just goes over + +00:20:43.660 --> 00:20:45.299 +there's a whole lot of stuff + +00:20:45.300 --> 00:20:47.779 +because his story spans like 60 years + +00:20:47.780 --> 00:20:49.659 +so it gives a good good feedback + +00:20:49.660 --> 00:20:51.539 +that's the real main reason I asked that question + +00:20:51.540 --> 00:20:53.699 +because I just needed one that sea monkeys I just picked + +00:20:53.700 --> 00:20:56.579 +because it was obscure and just always have right + +00:20:56.580 --> 00:20:58.939 +I used to have it right hello world and forth + +00:20:58.940 --> 00:21:01.019 +because I thought was an interesting one as well so + +00:21:01.020 --> 00:21:03.899 +It's just picking random ones like that. + +00:21:03.900 --> 00:21:06.499 +One question asked, sorry, a lot of models is, + +00:21:06.500 --> 00:21:09.419 +what is the closest star to the Earth? + +00:21:09.420 --> 00:21:12.019 +Because most of them will say Alpha Centauri + +00:21:12.020 --> 00:21:13.739 +or Proxima Centauri and not the sun. + +00:21:13.740 --> 00:21:15.899 +And I have a whole nother talk + +00:21:15.900 --> 00:21:17.899 +where I just argue with the LLM + +00:21:17.900 --> 00:21:20.019 +trying to say, hey, the sun is a star. + +00:21:20.020 --> 00:21:26.579 +And he just wouldn't accept it, so. What? + +00:21:26.580 --> 00:21:28.419 +Oh, I can hear that. + +00:21:28.420 --> 00:21:34.379 +So what specific tasks do you like to use your local AI? + +00:21:34.380 --> 00:21:37.459 +I like to load a lot of my code into + +00:21:37.460 --> 00:21:39.739 +and actually have it do analysis of it. + +00:21:39.740 --> 00:21:42.339 +I was actually going through some code + +00:21:42.340 --> 00:21:45.619 +I have for some pen testing, and I was having it modified + +00:21:45.620 --> 00:21:47.259 +to update it for the newer version, + +00:21:47.260 --> 00:21:48.459 +because I hate to say this, + +00:21:48.460 --> 00:21:49.859 +but it was written for Python 2, + +00:21:49.860 --> 00:21:51.459 +and I needed to update it for Python 3. + +00:21:51.460 --> 00:21:53.859 +And the 2 to 3 tool did not do all of it, + +00:21:53.860 --> 00:21:56.659 +but the actual tool was able to do the refactoring. + +00:21:56.660 --> 00:21:58.499 +It's part of my laziness. + +00:21:58.500 --> 00:22:01.459 +But I use that for anything I don't want to hit the web. + +00:22:01.460 --> 00:22:03.259 +And that's a lot of stuff when you start thinking about + +00:22:03.260 --> 00:22:04.979 +if you're doing cyber security researching. + +00:22:04.980 --> 00:22:06.819 +and you have your white papers + +00:22:06.820 --> 00:22:10.779 +and stuff like that and stuff in there. + +00:22:10.780 --> 00:22:13.979 +I've got a lot of that loaded into RAG + +00:22:13.980 --> 00:22:15.659 +in one model on my OpenWebUI system. + +00:22:15.660 --> 00:22:21.059 +Neat. Have you used have you used + +00:22:21.060 --> 00:22:25.739 +any small domain specific LLMs? What kind of tasks? + +00:22:25.740 --> 00:22:30.419 +If so, what kind of tasks that they specialize in? + +00:22:30.420 --> 00:22:32.139 +And you know, how? + +00:22:32.140 --> 00:22:34.979 +Not to be honest, but there are some out there like once again, + +00:22:34.980 --> 00:22:36.779 +for cybersecurity and stuff like that, + +00:22:36.780 --> 00:22:39.739 +that I really need to dig into that's on my to do list. + +00:22:39.740 --> 00:22:41.699 +I've got a couple weeks off at the end of the year. + +00:22:41.700 --> 00:22:43.779 +And that's a big part of my plan for that. + +00:22:43.780 --> 00:22:49.379 +Are the various models updated pretty regularly? + +00:22:49.380 --> 00:22:52.059 +Can you add your own data to the pre-built models? + +00:22:52.060 --> 00:22:56.699 +Yes. The models are updated pretty reasonably. + +00:22:56.700 --> 00:22:59.699 +You can add data to a model in a couple of different ways. + +00:22:59.700 --> 00:23:01.099 +You can do something called fine-tuning, + +00:23:01.100 --> 00:23:03.819 +which requires a really nice GPU and a lot of CPU time. + +00:23:03.820 --> 00:23:05.499 +Probably not going to do that. + +00:23:05.500 --> 00:23:07.419 +You can do retrieval augmentation generation, + +00:23:07.420 --> 00:23:09.499 +which is you load your data on top of the system + +00:23:09.500 --> 00:23:11.299 +and puts inside a database + +00:23:11.300 --> 00:23:12.859 +and you can actually scan that and stuff. + +00:23:12.860 --> 00:23:14.619 +I have another talk where I go through + +00:23:14.620 --> 00:23:16.219 +and I start asking questions about, + +00:23:16.220 --> 00:23:18.579 +I load the talk into the engine + +00:23:18.580 --> 00:23:20.099 +and I ask questions against that. + +00:23:20.100 --> 00:23:22.179 +I would have one more time would have done that + +00:23:22.180 --> 00:23:26.499 +but it comes down to how many That's that's rag rag + +00:23:26.500 --> 00:23:29.419 +is pretty easy to do through open web UI or LM studio + +00:23:29.420 --> 00:23:31.419 +It's a great way you just like point a folder + +00:23:31.420 --> 00:23:34.099 +point it to a folder and it just sucks all that state into + +00:23:34.100 --> 00:23:35.499 +and it'll hit that data first + +00:23:35.500 --> 00:23:36.859 +you have like helpdesk and stuff and + +00:23:36.860 --> 00:23:39.619 +The other options there's vector databases, + +00:23:39.620 --> 00:23:41.819 +which is like if you use PostgreSQL. + +00:23:41.820 --> 00:23:43.699 +It has a PG vector I can do a lot of that stuff. + +00:23:43.700 --> 00:23:44.739 +I've not dug into that yet, + +00:23:44.740 --> 00:23:46.099 +but that is also on that to-do list + +00:23:46.100 --> 00:23:48.459 +I've got a lot of stuff planned for Cool. + +00:23:48.460 --> 00:23:51.819 +So what are your experience with rags? + +00:23:51.820 --> 00:23:54.339 +I don't even know what that means. + +00:23:54.340 --> 00:23:57.419 +Do you know what that means? + +00:23:57.420 --> 00:23:59.619 +Do you remember this question again? + +00:23:59.620 --> 00:24:03.979 +What is your experience with RAGs? RAGs is great. + +00:24:03.980 --> 00:24:07.459 +That's Retrieval Augmentation Generation. + +00:24:07.460 --> 00:24:09.739 +That loads your data first, and it hits yours, + +00:24:09.740 --> 00:24:11.499 +and it'll actually cite it and stuff. + +00:24:11.500 --> 00:24:14.659 +There's a guy who wrote a RAG in 100 lines of Python, + +00:24:14.660 --> 00:24:16.899 +and it's an impressive piece of software. + +00:24:16.900 --> 00:24:18.779 +I think if you hit one of my site, + +00:24:18.780 --> 00:24:22.099 +I've got a private AI talk where I actually refer to that. + +00:24:22.100 --> 00:24:25.219 +But retrieval augmentation, it's easy, it's fast, + +00:24:25.220 --> 00:24:26.699 +it puts your data into the system, + +00:24:26.700 --> 00:24:31.339 +Yeah, start with that and go then iterate on top of that. + +00:24:31.340 --> 00:24:32.659 +That's one of the great things about AI, + +00:24:32.660 --> 00:24:33.619 +especially private AI, + +00:24:33.620 --> 00:24:37.739 +is you can do whatever you want to with it + +00:24:37.740 --> 00:24:43.179 +and build up with it as you get more experience. + +00:24:43.180 --> 00:24:44.219 +Any thoughts on running things + +00:24:44.220 --> 00:24:49.179 +on AWS, DigitalOcean, and so on? + +00:24:49.180 --> 00:24:50.619 +AWS is not bad. + +00:24:50.620 --> 00:24:52.659 +The DigitalOcean, they have some of their GPUs. + +00:24:52.660 --> 00:24:54.379 +I still don't like having the data + +00:24:54.380 --> 00:24:57.419 +leave my house, to be honest, or at work, + +00:24:57.420 --> 00:24:59.019 +because I tend to do some stuff + +00:24:59.020 --> 00:25:01.259 +that I don't want it even hitting that situation. + +00:25:01.260 --> 00:25:03.699 +But they have pretty good stuff. + +00:25:03.700 --> 00:25:05.579 +Another one to consider is Oracle Cloud. + +00:25:05.580 --> 00:25:09.059 +Oracle has their AI infrastructure that's really well done. + +00:25:09.060 --> 00:25:12.379 +But I mean, once again, then you start looking at potential + +00:25:12.380 --> 00:25:13.779 +is saying your data is private, + +00:25:13.780 --> 00:25:14.819 +I don't necessarily trust it. + +00:25:14.820 --> 00:25:17.859 +But they do have good stuff, both DigitalOcean, AWS, + +00:25:17.860 --> 00:25:20.339 +Oracle Cloud has the free service, which isn't too bad, + +00:25:20.340 --> 00:25:21.339 +usually a certain number of stuff. + +00:25:21.340 --> 00:25:23.179 +And Google's also has it, + +00:25:23.180 --> 00:25:26.739 +but I still tend to keep more stuff on local PCs, + +00:25:26.740 --> 00:25:33.299 +because I just paranoid that way. Gotcha. + +00:25:33.300 --> 00:25:35.579 +What has your experience been using AI? + +00:25:35.580 --> 00:25:40.139 +Do you want to get into that, using AI for cybersecurity? + +00:25:40.140 --> 00:25:42.019 +You might have already touched on this. + +00:25:42.020 --> 00:25:44.379 +Yeah, really, for cybersecurity, + +00:25:44.380 --> 00:25:46.259 +what I've had to do is I've dumped logs + +00:25:46.260 --> 00:25:47.299 +to have a due correlation. + +00:25:47.300 --> 00:25:49.859 +Keep in mind, the size of that LLAMA file we were using + +00:25:49.860 --> 00:25:52.059 +for figuring out David Bowie, writing the hello world, + +00:25:52.060 --> 00:25:54.179 +all that stuff, is like six gig. + +00:25:54.180 --> 00:25:56.859 +How does it get the entire world in six gig? + +00:25:56.860 --> 00:25:59.739 +I still haven't figured that out in terms of quantization. + +00:25:59.740 --> 00:26:02.499 +So I'm really interested in seeing the ability + +00:26:02.500 --> 00:26:05.139 +to take all this stuff out of all my logs, + +00:26:05.140 --> 00:26:06.339 +dump it all in there, + +00:26:06.340 --> 00:26:08.459 +and actually be able to do intelligent queries against that. + +00:26:08.460 --> 00:26:10.899 +Microsoft has a project called Security Copilot, + +00:26:10.900 --> 00:26:12.819 +which is trying to do that in the Cloud. + +00:26:12.820 --> 00:26:15.299 +But I want to work on something to do that more locally + +00:26:15.300 --> 00:26:19.019 +and be able to actually drive this stuff over that. + +00:26:19.020 --> 00:26:21.979 +That's one also on the long-term goals. + +00:26:21.980 --> 00:26:26.059 +So we got any other questions or? + +00:26:26.060 --> 00:26:29.099 +Those are the questions that I see. + +00:26:29.100 --> 00:26:31.179 +I want to just read out a couple of comments + +00:26:31.180 --> 00:26:33.419 +that I saw in IRC though. + +00:26:33.420 --> 00:26:36.699 +Jay Rutabaga says, it went very well + +00:26:36.700 --> 00:26:39.259 +from an audience perspective. + +00:26:39.260 --> 00:26:43.619 +And G Gundam says, respect your commitment to privacy. + +00:26:43.620 --> 00:26:45.619 +And then somebody is telling us + +00:26:45.620 --> 00:26:46.779 +we might have skipped a question. + +00:26:46.780 --> 00:26:50.019 +So I'm just going to run back to my list. + +00:26:50.020 --> 00:26:52.819 +Updated regularly experience. + +00:26:52.820 --> 00:26:57.659 +I just didn't type in the answer here's + +00:26:57.660 --> 00:26:59.659 +and there's a couple more questions coming in so + +00:26:59.660 --> 00:27:04.699 +Is there a disparity where you go to paid models + +00:27:04.700 --> 00:27:08.619 +because they are better and what problems? + +00:27:08.620 --> 00:27:14.019 +You know what would drive you to? That's a good question. + +00:27:14.020 --> 00:27:17.819 +Paid models, I don't mind them. I think they're good, + +00:27:17.820 --> 00:27:21.299 +but I don't think they're actually economically sustainable + +00:27:21.300 --> 00:27:22.659 +under their current system. + +00:27:22.660 --> 00:27:24.299 +Because right now, if you're paying + +00:27:24.300 --> 00:27:26.899 +20 bucks a month for Copilot and that goes up to 200 bucks, + +00:27:26.900 --> 00:27:28.499 +I'm not going to be as likely to use it. + +00:27:28.500 --> 00:27:29.579 +You know what I mean? + +00:27:29.580 --> 00:27:33.059 +But it does do some things in a way that I did not expect. + +00:27:33.060 --> 00:27:35.459 +For example, Grok was refactoring + +00:27:35.460 --> 00:27:38.019 +some of my code in the comments and dropped an F-bomb. + +00:27:38.020 --> 00:27:39.979 +which I did not see coming, + +00:27:39.980 --> 00:27:41.619 +but the other code before + +00:27:41.620 --> 00:27:43.219 +that I had gotten off GitHub + +00:27:43.220 --> 00:27:44.059 +had F bombs in it. + +00:27:44.060 --> 00:27:45.899 +So it was just emulating the style, + +00:27:45.900 --> 00:27:47.779 +but would that be something + +00:27:47.780 --> 00:27:49.979 +I'd want to turn in a pull request? I don't know. + +00:27:49.980 --> 00:27:52.139 +But, uh, there's, there's a lot of money + +00:27:52.140 --> 00:27:53.899 +going into these AIs and stuff, + +00:27:53.900 --> 00:27:56.219 +but in terms of the ability to get a decent one, + +00:27:56.220 --> 00:27:57.979 +like the llama, llama three, two, + +00:27:57.980 --> 00:28:01.699 +and load your data into it, you can be pretty competitive. + +00:28:01.700 --> 00:28:04.779 +You're not going to get all the benefits, + +00:28:04.780 --> 00:28:07.299 +but you have more control over it. + +00:28:07.300 --> 00:28:11.819 +So it's, it's a, this and that it's a, + +00:28:11.820 --> 00:28:13.139 +it's a balancing act. + +00:28:13.140 --> 00:28:15.539 +Okay, and I think I see a couple more questions coming in. + +00:28:15.540 --> 00:28:19.619 +What is the largest parameter size for local models + +00:28:19.620 --> 00:28:22.459 +that you've been able to successfully run locally + +00:28:22.460 --> 00:28:26.059 +and do run into issues with limited context window size? + +00:28:26.060 --> 00:28:29.659 +The top eight models will tend to have a larger ceiling. + +00:28:29.660 --> 00:28:32.859 +Yes, yes, yes, yes, yes. + +00:28:32.860 --> 00:28:37.019 +By default, the context size is I think 1024. + +00:28:37.020 --> 00:28:44.619 +But I've upped it to 8192 on the on this box, the Pangolin + +00:28:44.620 --> 00:28:46.939 +because it seems to be some reason + +00:28:46.940 --> 00:28:49.459 +it's just a very working quite well. + +00:28:49.460 --> 00:28:52.219 +But the largest ones I've loaded have been in + +00:28:52.220 --> 00:28:54.059 +the have not been that huge. + +00:28:54.060 --> 00:28:55.699 +I've loaded this the last biggest one I've done. + +00:28:55.700 --> 00:28:57.459 +That's the reason why I'm planning + +00:28:57.460 --> 00:29:01.339 +on breaking down and buying a Ryzen. + +00:29:01.340 --> 00:29:03.619 +Actually, I'm going to buy + +00:29:03.620 --> 00:29:06.979 +an Intel i285H with 96 gig of RAM. + +00:29:06.980 --> 00:29:08.379 +Then I should be able to load + +00:29:08.380 --> 00:29:12.059 +a 70 billion parameter model in that. How fast will it run? + +00:29:12.060 --> 00:29:13.819 +It's going to run slow as dog, + +00:29:13.820 --> 00:29:15.819 +but it's going to be cool to be able to do it. + +00:29:15.820 --> 00:29:17.379 +It's an AI bragging rights thing, + +00:29:17.380 --> 00:29:20.019 +but I mostly stick with the smaller size models + +00:29:20.020 --> 00:29:22.819 +and the ones that are more quantitized + +00:29:22.820 --> 00:29:26.619 +because it just tends to work better for me. + +00:29:26.620 --> 00:29:29.179 +We've still got over 10 minutes before we're cutting away, + +00:29:29.180 --> 00:29:30.179 +but I'm just anticipating + +00:29:30.180 --> 00:29:32.859 +that we're going to be going strong at the 10 minute mark. + +00:29:32.860 --> 00:29:34.899 +So I'm just, just letting, you know, + +00:29:34.900 --> 00:29:37.379 +we can go as long as we like here at a certain point. + +00:29:37.380 --> 00:29:41.059 +I may have to jump away and check in with the next speaker, + +00:29:41.060 --> 00:29:44.419 +but we'll post the entirety of this, + +00:29:44.420 --> 00:29:47.979 +even if we aren't able to stay with it all. + +00:29:47.980 --> 00:29:49.739 +Okay. And we've got 10 minutes + +00:29:49.740 --> 00:29:52.379 +where we're still going to stay live. + +00:29:52.380 --> 00:30:00.139 +So next question coming in, I see, are there free as in freedom, + +00:30:00.140 --> 00:30:05.739 +free as in FSF issues with the data? + +00:30:05.740 --> 00:30:11.699 +Yes, where's the data coming from is a huge question with AI. + +00:30:11.700 --> 00:30:13.739 +It's astonishing you can ask questions + +00:30:13.740 --> 00:30:16.899 +to models that you don't know where it's coming from. + +00:30:16.900 --> 00:30:19.979 +That is gonna be one of the big issues long-term. + +00:30:19.980 --> 00:30:21.499 +There are people who are working + +00:30:21.500 --> 00:30:22.979 +on trying to figure out that stuff, + +00:30:22.980 --> 00:30:25.259 +but it's, I mean, if you look at, God, + +00:30:25.260 --> 00:30:27.059 +I can't remember who it was. + +00:30:27.060 --> 00:30:28.659 +Somebody was actually out torrenting books + +00:30:28.660 --> 00:30:30.939 +just to be able to build into their AI system. + +00:30:30.940 --> 00:30:32.339 +I think it might've been Meta. + +00:30:32.340 --> 00:30:34.819 +So there's a lot of that going on. + +00:30:34.820 --> 00:30:38.139 +The open source of the stuff is going to be tough. + +00:30:38.140 --> 00:30:39.459 +There's going to be there's some models + +00:30:39.460 --> 00:30:41.419 +like the mobile guys have got their own license, + +00:30:41.420 --> 00:30:42.739 +but where they're getting their data from, + +00:30:42.740 --> 00:30:45.499 +I'm not sure on so that that's a huge question. + +00:30:45.500 --> 00:30:47.979 +That's a that's a talk in itself. + +00:30:47.980 --> 00:30:51.979 +But yeah, but you if you train on your RAG and your data, + +00:30:51.980 --> 00:30:53.499 +you know what it's come, you know, + +00:30:53.500 --> 00:30:54.379 +you have a license that + +00:30:54.380 --> 00:30:55.139 +but the other stuff is just + +00:30:55.140 --> 00:30:56.739 +more lines of supplement + +00:30:56.740 --> 00:31:01.379 +if you're using a smaller model, + +00:31:01.380 --> 00:31:05.419 +but the comment online, I see a couple of them. + +00:31:05.420 --> 00:31:08.339 +I'll read them out in order here. Really interesting stuff. + +00:31:08.340 --> 00:31:11.659 +Thank you for your talk. Given that large AI companies + +00:31:11.660 --> 00:31:14.899 +are openly stealing intellectual property and copyright + +00:31:14.900 --> 00:31:18.939 +and therefore eroding the authority of such laws + +00:31:18.940 --> 00:31:21.579 +and maybe obscuring the truth itself, + +00:31:21.580 --> 00:31:26.579 +can you see a future where IP and copyright flaw become untenable? + +00:31:26.580 --> 00:31:29.619 +I think that's a great question. + +00:31:29.620 --> 00:31:34.979 +I'm not a lawyer, but it is really getting complicated. + +00:31:34.980 --> 00:31:37.859 +It is getting to the point, I asked a question from, + +00:31:37.860 --> 00:31:41.179 +I played with Sora a little bit, and it generated someone, + +00:31:41.180 --> 00:31:42.819 +you can go like, oh, that's Jon Hamm, + +00:31:42.820 --> 00:31:44.099 +that's Christopher Walken, + +00:31:44.100 --> 00:31:45.379 +you start figuring out who the people + +00:31:45.380 --> 00:31:47.019 +they're modeling stuff after. + +00:31:47.020 --> 00:31:48.979 +There is an apocalypse, something + +00:31:48.980 --> 00:31:52.459 +going to happen right now. + +00:31:52.460 --> 00:31:53.579 +There is, but this is once again, + +00:31:53.580 --> 00:31:56.059 +my personal opinion, and I'm not a lawyer, + +00:31:56.060 --> 00:31:57.459 +and I do not have money. + +00:31:57.460 --> 00:31:58.859 +So don't sue me, is there's going to be + +00:31:58.860 --> 00:32:02.899 +the current administration tends is very AI pro AI. + +00:32:02.900 --> 00:32:05.499 +And there's very a great deal of lobbying by those groups. + +00:32:05.500 --> 00:32:07.139 +And it's on both sides. + +00:32:07.140 --> 00:32:09.699 +And it's going to be, it's gonna be interesting to see + +00:32:09.700 --> 00:32:11.699 +what happens to copyright the next 510 years. + +00:32:11.700 --> 00:32:13.339 +I just don't know how it keeps up + +00:32:13.340 --> 00:32:16.059 +without there being some adjustments and stuff. + +00:32:16.060 --> 00:32:20.419 +Okay, and then another comment I saw, + +00:32:20.420 --> 00:32:23.219 +file size is not going to be a bottleneck. + +00:32:23.220 --> 00:32:25.819 +RAM is. You'll need 16 gigabytes of RAM + +00:32:25.820 --> 00:32:28.259 +to run the smallest local models + +00:32:28.260 --> 00:32:31.979 +and 512 gigabytes of RAM to run the larger ones. + +00:32:31.980 --> 00:32:35.059 +You'll need a GPU with that much memory + +00:32:35.060 --> 00:32:39.099 +if you want it to run quickly. Yeah. Oh no. + +00:32:39.100 --> 00:32:41.259 +It also depends upon how your memory is laid out. + +00:32:41.260 --> 00:32:45.699 +Like example being the Ultra i285H + +00:32:45.700 --> 00:32:47.899 +I plan to buy, that has 96 gig of memory. + +00:32:47.900 --> 00:32:50.499 +It's unified between the GPU and the CPU share it, + +00:32:50.500 --> 00:32:52.739 +but they go over the same bus. + +00:32:52.740 --> 00:32:55.779 +So the overall bandwidth of it tends to be a bit less, + +00:32:55.780 --> 00:32:57.579 +but you're able to load more of it into memory. + +00:32:57.580 --> 00:32:59.419 +So it's able to do some additional stuff with it + +00:32:59.420 --> 00:33:00.819 +as opposed to come off disk. + +00:33:00.820 --> 00:33:03.699 +It's all balancing act. If you hit Zyskin's website, + +00:33:03.700 --> 00:33:05.819 +that guy's done some great work on it. + +00:33:05.820 --> 00:33:07.499 +I'm trying to figure out how big a model you can do, + +00:33:07.500 --> 00:33:08.619 +what you can do with it. + +00:33:08.620 --> 00:33:12.699 +And some of the stuff seems to be not obvious, + +00:33:12.700 --> 00:33:15.299 +because like example, being that MacBook Air, + +00:33:15.300 --> 00:33:17.619 +for the five minutes I can run the model, + +00:33:17.620 --> 00:33:19.379 +it runs it faster than a lot of other things + +00:33:19.380 --> 00:33:21.339 +that should be able to run it faster, + +00:33:21.340 --> 00:33:24.619 +just because of the way the ARM cores and the unified memory work on it. + +00:33:24.620 --> 00:33:26.019 +So it's a learning process. + +00:33:26.020 --> 00:33:29.579 +But if you want to, Network Chuck had a great video + +00:33:29.580 --> 00:33:30.939 +talking about building his own system + +00:33:30.940 --> 00:33:34.379 +with a couple really powerful NVIDIA cards + +00:33:34.380 --> 00:33:35.379 +and stuff like that in it. + +00:33:35.380 --> 00:33:38.859 +And just actually setting up on his system as a node + +00:33:38.860 --> 00:33:41.459 +and using a web UI on it. So there's a lot of stuff there, + +00:33:41.460 --> 00:33:43.899 +but it is a process of learning how big your data is, + +00:33:43.900 --> 00:33:44.899 +which models you want to use, + +00:33:44.900 --> 00:33:46.219 +how much information you need, + +00:33:46.220 --> 00:33:48.019 +but it's part of the learning. + +00:33:48.020 --> 00:33:52.899 +And you can run models, even as a Raspberry PI fives, + +00:33:52.900 --> 00:33:54.499 +if you want to, they'll run slow. + +00:33:54.500 --> 00:33:56.459 +Don't get me wrong, but they're possible. + +00:33:56.460 --> 00:34:02.179 +Okay, and I think there's other questions coming in too, + +00:34:02.180 --> 00:34:04.019 +so I'll just bam for another second. + +00:34:04.020 --> 00:34:06.299 +We've got about five minutes before we'll, + +00:34:06.300 --> 00:34:09.739 +before we'll be cutting over, + +00:34:09.740 --> 00:34:13.179 +but I just want to say in case we get close for time here, + +00:34:13.180 --> 00:34:14.859 +how much I appreciate your talk. + +00:34:14.860 --> 00:34:15.979 +This is another one that I'm going to + +00:34:15.980 --> 00:34:18.339 +have to study after the conference. + +00:34:18.340 --> 00:34:21.099 +We greatly appreciate, all of us appreciate + +00:34:21.100 --> 00:34:22.459 +you guys putting on the conference. + +00:34:22.460 --> 00:34:26.299 +It's a great conference. It's well done. + +00:34:26.300 --> 00:34:28.019 +It's an honor to be on the stage + +00:34:28.020 --> 00:34:30.899 +with the brains of the project, which is you. + +00:34:30.900 --> 00:34:34.699 +So what else we got? Question wise. + +00:34:34.700 --> 00:34:39.499 +Okay, so just scanning here. + +00:34:39.500 --> 00:34:50.699 +Have you used local models capable of tool calling? + +00:34:50.700 --> 00:34:54.779 +I'm, I'm scared of agentic. + +00:34:54.780 --> 00:34:58.739 +I, I am, I'm going to be a slow adopter of that. + +00:34:58.740 --> 00:35:02.459 +I want to do it, but I just don't have the, uh, + +00:35:02.460 --> 00:35:04.339 +four decimal fortitude right now to do it. + +00:35:04.340 --> 00:35:07.179 +I, I, I've had to give me the commands, + +00:35:07.180 --> 00:35:08.739 +but I still run the commands by hand. + +00:35:08.740 --> 00:35:10.539 +I'm looking into it and it's on once again, + +00:35:10.540 --> 00:35:14.139 +it's on that list, but I just, that's a big step for me. + +00:35:14.140 --> 00:35:23.139 +So. Awesome. All right. + +00:35:23.140 --> 00:35:27.179 +Well, maybe it's, let me just scroll through + +00:35:27.180 --> 00:35:31.539 +because we might have missed one question. Oh, I see. + +00:35:31.540 --> 00:35:36.899 +Here was the piggyback question. + +00:35:36.900 --> 00:35:38.419 +Now I see the question that I missed. + +00:35:38.420 --> 00:35:41.139 +So this was piggybacking on the question + +00:35:41.140 --> 00:35:44.859 +about model updates and adding data. + +00:35:44.860 --> 00:35:46.579 +And will models reach out to the web + +00:35:46.580 --> 00:35:47.819 +if they need more info? + +00:35:47.820 --> 00:35:51.779 +Or have you worked with any models that work that way? + +00:35:51.780 --> 00:35:55.259 +No, I've not seen any models to do that + +00:35:55.260 --> 00:35:57.739 +There's there was like a group + +00:35:57.740 --> 00:35:59.899 +working on something like a package updater + +00:35:59.900 --> 00:36:02.499 +that would do different diffs on it, + +00:36:02.500 --> 00:36:03.939 +but it's so Models change so much + +00:36:03.940 --> 00:36:05.739 +even who make minor changes and fine-tuning. + +00:36:05.740 --> 00:36:07.659 +It's hard just to update them in place + +00:36:07.660 --> 00:36:10.099 +So I haven't seen one, but that doesn't mean + +00:36:10.100 --> 00:36:16.259 +they're not out there. I'm curious topic though Awesome + +00:36:16.260 --> 00:36:19.539 +Well, it's probably pretty good timing. + +00:36:19.540 --> 00:36:21.299 +Let me just scroll and make sure. + +00:36:21.300 --> 00:36:23.499 +And of course, before I can say that, + +00:36:23.500 --> 00:36:25.899 +there's one more question. So let's go ahead and have that. + +00:36:25.900 --> 00:36:28.299 +I want to make sure while we're still live, though, + +00:36:28.300 --> 00:36:31.299 +I give you a chance to offer any closing thoughts. + +00:36:31.300 --> 00:36:35.779 +So what scares you most about the agentic tools? + +00:36:35.780 --> 00:36:38.419 +How would you think about putting a sandbox around that + +00:36:38.420 --> 00:36:42.139 +if you did adopt an agentic workflow? + +00:36:42.140 --> 00:36:42.899 +That is a great question. + +00:36:42.900 --> 00:36:45.939 +In terms of that, I would just control + +00:36:45.940 --> 00:36:48.099 +what it's able to talk to, what machines, + +00:36:48.100 --> 00:36:50.059 +I would actually have it be air gap. + +00:36:50.060 --> 00:36:52.099 +I work for a defense contractor, + +00:36:52.100 --> 00:36:53.819 +and we spend a lot of time dealing with air gap systems, + +00:36:53.820 --> 00:36:55.979 +because that's just kind of the way it works out for us. + +00:36:55.980 --> 00:36:58.499 +So agentic, it's just going to take a while to get trust. + +00:36:58.500 --> 00:37:01.059 +I want to want to see more stuff happening. + +00:37:01.060 --> 00:37:02.819 +Humans screw up stuff enough. + +00:37:02.820 --> 00:37:04.819 +The last thing we need is to multiply that by 1000. + +00:37:04.820 --> 00:37:09.419 +So in terms of that, I would be restricting what it can do. + +00:37:09.420 --> 00:37:10.859 +If you look at the capabilities, + +00:37:10.860 --> 00:37:13.579 +if I created a user and gave it permissions, + +00:37:13.580 --> 00:37:15.299 +I would have a lockdown through sudo, + +00:37:15.300 --> 00:37:17.379 +what it's able to do, what the account's able to do. + +00:37:17.380 --> 00:37:18.899 +I would do those kind of things, + +00:37:18.900 --> 00:37:20.859 +but it's going to be, it's happening. + +00:37:20.860 --> 00:37:25.819 +It's just, I'm going to be one of the laggards on that one. + +00:37:25.820 --> 00:37:29.259 +So airgab, jail, extremely locked down environments, + +00:37:29.260 --> 00:37:34.899 +like we're talking about separate physicals, not Docker. + +00:37:34.900 --> 00:37:37.499 +Yeah, hopefully. Right, fair. + +00:37:37.500 --> 00:37:39.899 +So tool calling can be read-only, + +00:37:39.900 --> 00:37:42.539 +such as giving models the ability to search the web + +00:37:42.540 --> 00:37:43.979 +before answering your question, + +00:37:43.980 --> 00:37:46.219 +you know, write access, execute access. + +00:37:46.220 --> 00:37:49.219 +I'm interested to know if local models + +00:37:49.220 --> 00:37:51.419 +are any good at that. + +00:37:51.420 --> 00:37:55.579 +Yes, local models can do a lot of that stuff. + +00:37:55.580 --> 00:37:56.819 +It's their capabilities. + +00:37:56.820 --> 00:37:59.019 +If you load LM studio, you can do a lot of wonderful stuff + +00:37:59.020 --> 00:38:02.419 +with that or with open web UI with a llama. + +00:38:02.420 --> 00:38:05.739 +It's a lot of capabilities. It's amazing. + +00:38:05.740 --> 00:38:08.139 +Open web UI is actually what a lot of companies are using now + +00:38:08.140 --> 00:38:10.259 +to put their data behind that. + +00:38:10.260 --> 00:38:12.139 +They're curated data and stuff like that. So works well. + +00:38:12.140 --> 00:38:15.819 +I can confirm that from my own professional experience. + +00:38:15.820 --> 00:38:19.659 +Excellent. Okay, well, our timing should be just perfect + +00:38:19.660 --> 00:38:22.659 +if you want to give us like a 30-second, 45-second wrap-up. + +00:38:22.660 --> 00:38:24.419 +Aaron, let me squeeze in mine. + +00:38:24.420 --> 00:38:26.779 +Thank you again so much for preparing this talk + +00:38:26.780 --> 00:38:30.499 +and for entertaining all of our questions. + +00:38:30.500 --> 00:38:33.299 +Yeah, let me just thank you guys for the conference again. + +00:38:33.300 --> 00:38:35.179 +This is a great one. I've enjoyed a lot of it. + +00:38:35.180 --> 00:38:37.339 +I've only had a couple of talks so far, + +00:38:37.340 --> 00:38:41.659 +but I'm looking forward to hitting the ones after this and tomorrow. + +00:38:41.660 --> 00:38:44.739 +But the AI stuff is coming. Get on board. + +00:38:44.740 --> 00:38:46.939 +Definitely recommend it. If you want to just try it out + +00:38:46.940 --> 00:38:48.419 +and get a little taste of it, + +00:38:48.420 --> 00:38:49.779 +what my minimal viable product + +00:38:49.780 --> 00:38:51.619 +with just LlamaFile and GPTEL + +00:38:51.620 --> 00:38:53.139 +will get you to the point where you start figuring out. + +00:38:53.140 --> 00:38:55.579 +Gptel is an amazing thing. It just gets out of your way, + +00:38:55.580 --> 00:39:00.459 +but it works solo with Emacs. Design because it takes + +00:39:00.460 --> 00:39:01.699 +doesn't take your hands off the keyboard. + +00:39:01.700 --> 00:39:02.499 +It's just another buffer + +00:39:02.500 --> 00:39:04.059 +and you just put information in there. + +00:39:04.060 --> 00:39:06.979 +It's quite quite a wonderful It's a wonderful time. + +00:39:06.980 --> 00:39:10.819 +Let's put that way That's all I got Thank you + +00:39:10.820 --> 00:39:14.339 +so much for once again, and we're we're just cut away. + +00:39:14.340 --> 00:39:15.779 +So I'll stop the recording + +00:39:15.780 --> 00:39:18.259 +and you're on your own recognizance + +00:39:18.260 --> 00:39:19.699 +Well, I'm gonna punch out + +00:39:19.700 --> 00:39:21.059 +if anybody has any questions or anything + +00:39:21.060 --> 00:39:24.699 +my email address is ajgrothe@yahoo.com or at gmail and + +00:39:24.700 --> 00:39:26.779 +Thank you all for attending + +00:39:26.780 --> 00:39:29.939 +and thanks again for the conference + +00:39:29.940 --> 00:39:32.579 +Okay, I'm gonna go ahead and end the room there, thank you. + +00:39:32.580 --> 00:39:34.100 +Excellent, thanks, bye. |