WEBVTT


00:00:00.000 --> 00:00:48.179
Introduction

00:00:48.180 --> 00:01:08.059
Overview of talk

00:01:08.060 --> 00:03:16.019
Why private AI?

00:03:16.020 --> 00:05:16.347
What do I need for private AI?

00:05:16.348 --> 00:06:13.219
Emacs and private AI

00:06:13.220 --> 00:07:56.339
Pieces for an AI Emacs solution

00:07:56.340 --> 00:08:52.099
Config file

00:08:52.100 --> 00:10:21.699
Demo: Who was David Bowie?

00:10:21.700 --> 00:10:42.179
Hallucinations

00:10:42.180 --> 00:11:57.179
Next question: What are sea monkeys?

00:11:57.180 --> 00:12:32.579
Writing Hello World in Emacs Lisp

00:12:32.580 --> 00:13:36.899
Pieces for a better solution

00:13:36.900 --> 00:14:56.579
What about the license?

00:14:56.580 --> 00:15:14.519
Are there open source data model options?

00:15:14.520 --> 00:20:07.419
Things to know

00:20:07.420 --> 00:21:30.739
Q: Why is the David Bowie question a good one for testing a model? e.g. does it fail in interesting ways?

00:21:30.740 --> 00:22:16.879
Q: What specific tasks do you use local AI for?

00:22:16.880 --> 00:22:46.539
Q: Have you used any small domain-specific LLMs? What are the kinds of tasks they specialize in, and how do I find and use them?

00:22:46.540 --> 00:23:48.055
Q: Are the various models updated regularly? Can you add your own data to pre-built models?

00:23:48.056 --> 00:24:38.833
Q: What is your experience with RAG? Are you using them and how have they helped?

00:24:38.834 --> 00:25:31.077
Q: Thoughts on running things on AWS/digital ocean instances, etc?

00:25:31.078 --> 00:26:59.659
Q: What has your experience been using AI for cyber security applications? What do you usually use it for?

00:26:59.660 --> 00:28:14.125
Q: Is there a disparity where you go to paid models becouse they are better and what problems would those be?

00:28:14.126 --> 00:29:52.379
Q: What's the largest (in parameter size) local model you've been able to successfully run locally, and do you run into issues with limited context window size?

00:29:52.380 --> 00:31:09.556
Q: Are there "Free" as in FSF/open source issues with the data?

00:31:09.557 --> 00:32:18.059
Q: Given that large AI companies are openly stealing IP and copyright, thereby eroding the authority of such law (and eroding truth itself as well), can you see a future where IP & copyright flaw become untenable and what sort of onwards effect might that have?

00:32:18.060 --> 00:34:46.899
Comment: File size is not going to be the bottleneck, your RAM is.

00:34:46.900 --> 00:35:44.859
Q: Have you used local models capable of tool-calling?

00:35:44.860 --> 00:36:31.299
Q: Will the models reach out to the web if they need to for more info?

00:36:31.300 --> 00:37:36.577
Q: What scares you most about agentic tools? How would you think about putting a sandbox around it if you adopt an agentic workflow?

00:37:36.578 --> 00:38:41.659
Q: Tool calling can be read-only, such as giving models the ability to search the web before answersing your question. (No write access or execute access) I'm interested to know if local models are any good at calling tools, though.

00:38:41.660 --> 00:39:34.100
Wrapping up