All right. I think we're live. Thank you, Andrew, for the
great talk.
Folks, if you'd like to ask your questions, please post
them on the pad. And at some point
in like a minute or two, we'll also open this big blue
button room if you'd like to
join here directly and ask questions here. Take it away,
Andrew.
Yeah, thanks. So should I be reading the questions from the
pad? Yeah.
Yeah, that would be great.
OK, so I see there's a question about is this built into Em
acs? I guess that refers to SQLite.
And is it possible to have multiple schemas, multiple
databases? Yes, you could do all that
with Emacs 29. SQLite is built in. You can open up multiple
databases simultaneously and do just
the normal things that you can do with databases. You can
create whatever tables you want.
You can create-- you can have transactions. So it's pretty
full featured. With SQLite,
there's also a notion of additions. You could kind of
compile new things into it. And I'm not
exactly sure to the extent that that is possible as well.
So it's like how full featured of a
SQLite is it? I'm not sure. But the basic stuff, you can
certainly do. And that basic stuff is
fantastic. Next question is, what about collaborative
editing with this? Can you
have multiple computers with multiple Emacs with org mode?
I think it sort of depends. Yes,
I mean, databases are fantastic for having multiple users
doing multiple things at the same time.
And they're really designed for this use case. So I would
say, yes, you could have multiple
editing of the database, multiple editing, whereas you see
some [AUDIO OUT] probably not the best
solution. I suspect it might be possible, but I wouldn't
want to try it. I think it's better for,
like, oh, you can have multiple processes, multiple users,
perhaps contributing to the
same database. And you're growing the database, but not
such tight synchronous interactivity that
I think you might be imagining here. What about using this
on multiple computers? How do you
synchronize your data? I don't know. I need to figure that
out. I think there's a-- like,
yes, you can synchronize your database in the same [AUDIO
OUT] Emacs doesn't have any great
solutions for this. That is, like, when-- let's say I want
to synchronize the things I'm editing
on Emacs with my phone. There's all sorts of various
solutions. You could use Dropbox, Google
Drive, these kinds of things. The database makes that a
little bit harder in the sense that it's
one giant database. I suspect there are interesting ways to
do this, but I haven't explored this area
yet. I think it's a great question, though, and I would
like to investigate this further.
Are you planning to further-- this is a new other question.
Are you planning to further develop EKG?
It's highly [AUDIO OUT] me, and I do prefer SQLite over
Text. Thank you for buying into
the vision. I really like to hear that. Yeah, I am
personally using EKG. I am developing it.
I don't think it's ready for general use yet. I'd like to
get it to a state where it is ready
for general use probably by the end of the month. I think I
'll have a lot of time. There's a pretty
slow month. So that would be really nice. I've just added a
bunch of stuff to it.
I'm exploring. I think the difficult thing is not
necessarily to make it production,
like to make it stable and everything like that and to do
all the developer things that you want
to see in the state of the art era, but really, it's a
different model of how to have notes.
I'm still thinking about some of these fundamental concepts
. It took me a long
time to iterate on various designs before I came up with EK
G, which I think is [AUDIO OUT]
and so we're going to really see how this goes. But yes,
look for it soon. I'll probably announce it
and maybe on email. And then I will make it available
probably maybe on
new alpha is what I would like to do. Is it possible to
combine the
triples database with some custom tables in the same SQLite
file? Yes. I'm sorry. Let me continue
reading. You need to build a log table next to the triples
table or a quick query of event data.
Yeah, you certainly could do that. And I've been thinking
of adding,
like, just having a record of changes might be an
interesting thing. But yeah, it's just one table
and you really could use this one table called triples, the
triples table, in any other database
or with any other database. It's kind of designed to be its
own database though. But yeah, you could
certainly add things to the side and I can imagine people
writing extensions that do
add some tables to the side to do whatever. It's just that
I think you want to be careful because
that once you start doing that, you might want to, you don
't want to kind of duplicate information or
start having things go out of sync. It's kind of nice just
to have one kind of data. But I think
in certain circumstances, like the kind you're thinking
about, it'd be appropriate. Yeah, so
seems like a good idea. And then I see this final question
here, final for now. What are your
thoughts on adding a timestamp attribute to triple so that
the database becomes append-only and by
default you return the latest fact for a subject-object
pair? Oh, really interesting question.
Let's just say I haven't thought of that and I'll have to
think about it. I don't think,
I haven't seen other triples databases do this. I'd be
interested in what you thought
would be some of the, what you thought might be some of the
interesting
things that come out of this. Yes, you can maybe go back in
time, which is nice,
but then it seems hard to do things like deletes. I guess
you could have tombstones or something
like that, but it becomes a little bit more complicated.
And also just one thing is like,
these kind of databases, I didn't really talk about this in
the talk, but they take up a little
bit of room and I'm a little bit reluctant to have them
even more kind of heavyweight.
On the other hand, perhaps this would go well with the
question about synchronization. If you
were to do as you suggest and have timestamped triples,
synchronization actually becomes pretty
easy. So definitely an interesting thought that I'll have
to think a little bit more about. I
would love to solve the synchronization problem. All right.
I don't see any other, oh yeah,
I don't see any other questions, but you can ask on this,
on the big blue button or type in any
other questions. Otherwise I think we might be done. Cool.
And I will add that. I think we still
have about like, I guess, 15 minutes or so of live Q&A talk
for this talk on stream. If there are no
questions, you're welcome to cut it short, but otherwise we
have plenty of time. So yeah, if
people want to get more questions in, please do. I'm just
going to sit back and wait for the
questions to roll in then. Sounds good. Thank you.
All right. I see another question pop up.
With EKG, what about views like org-roam-node-mindmap-views
? Yeah. I personally have
not found those views to be super helpful, but I know
people like them a lot. I don't see any
reason why you couldn't do this. It just would be a
slightly different implementation. Like this
stuff forms a graph. You can do whatever you, you know, you
could easily transform it like that same
org-roam UI. You just have to have like another sort of
translation layer basically to kind of
like figure out how to deal with the stuff in EKG or really
anything with triples. Like fundamentally
it's a graph database, so it lends itself naturally to
things that you're going to be doing,
that anything you're going to be doing with the triples
library.
[silence]
I see another question. Can ordinary Lisp data types, Lisp
symbols, etc. be stored in the
database? Yes, they can. In fact, it's primarily like if
you don't specify anything, it kind of
defaults to a list. That's sort of, I'm not exactly sure
that's the right design choice, but that's
kind of, it seems like a lot of the times it, this kind of
makes sense. Like what if I just added
like a title, I think, if I remember. I did add a title,
but I'm trying to remember. If I just added
the notion of like having a title to EKG and I believe that
's a list. Because you think about it,
you could maybe have alternate titles or multiple titles.
So these are ordered lists and so it kind
of stores it. So basically what happens is you store in the
list of like, but each list item is
a separate row in the table and it has, we call them tri
ples, but there's actually one more column
that I don't talk about a lot because it just confuses the
story. But in this case it's relevant
because there you just say what the index is of the item
that you're doing it. So like you'd be
subject and then the predicate would be title in a sense.
And then title one and then like,
okay, that's index one on the fourth column. And then same
thing with the other items.
You can store symbols, SQLite and Emac SQL allow you to
sort of kind of, well, Emac SQL kind of
naturally lets you store lots of different things. And SQL
ite, I kind of had to do something that
was very similar to Emac SQL. Emac SQL, in SQL, you have to
just, in your database, it basically
has to be string. Like almost everything besides the predic
ates, like the object has to be a string
because it could really be anything. And the subject has to
be a string because it could be
anything. And because of that, we kind of have to make sure
that we kind of have to just store
things in a certain way. Like symbols are just strings and
lists. You could even have a list,
you could have really any list expression as a subject,
right? But it gets just written to a
string and then it gets, when it gets read, it basically,
if it's just a string, it gets read
basically as a list symbol or expression. And if it's a
string, it's a string with a string inside,
that is like the string actually has quotes on either end.
Then we say it's actually a string.
That's how kind of the internal implementation works. And
that's what Emac SQL does as well.
For those who are not familiar, Emac SQL is like this other
way to kind of interact with SQL in
your databases and it works well with SQLite and allows you
to do things like store numbers and
lists as well. And it does this exact same thing. So we try
to keep compatibility.
So you could start a database in Emacs 28 using Emacs SQL
and then you could read it with Emacs 29
using the built in SQLite. That's the hope. I mean, I have
tests, but I don't think anyone
has actually tried to do that exact thing yet. Let's see.
Are there any other new questions?
Oh, there are some. Beyond note taking, what kind of
packages do you
think would benefit from a triple library? I think,
I mean, I think it's generally just an easy way to store
things. So
anything in which you, think of packages that have to store
like
list expressions on the side. Like for example, like BBDB,
if you've used that,
it's just a, it is a database. It's the big brother
database. It's not in a database because
it was written before. Like there was an easy way to use
databases in Emacs. Like that should,
that could definitely, like you don't need the triples
library, but you could easily do this
in the triples library. And I would kind of think triples,
you know, just takes care of a lot of
things and gives a little option over everything. So I mean
, there's all sorts of things like,
as I, as I kind of demoed like bookmarks, like why are book
marks not in the database? They should be.
So I think if you just look around and look at all these,
all these things as,
as just, just dumping, dumping list expressions to a file,
those probably
SQLite and, and, you know, maybe triples would be a useful
way to do that.
Are you, another question, are you trying to PIM? I think
that means personal information management
with EKG. And what information do you want to manage? I'm
kind of wondering
how I, what I want to do, but like, as I just said in the
previous question, like, I think
you should be storing everything. Yeah. Like libraries,
like, yeah, actually, yeah. BBBB,
you can store contacts and email addresses. And I would
like to, you know, why, if you're storing
that, then it's kind of natural, like, oh, well, you know,
sometimes I want to have a annotation.
I want to write something about this person and, you know,
maybe I want to have something and have,
you know, maybe I want to tag this person. Maybe I want to,
maybe I want to have notes that refer
to this person. So like, it all starts to look like one
giant database when I, when you kind
of think about it a lot, which is kind of why in a, if you
look at the develop, develop branch of
triples, I, I'm work, I think what's going to be in the
next version is, is going to be a
standardized place to, to have a database. That is, you can
just connect to a, to a, the standard
database and you don't have to worry about where the
database is. We'll keep track of that for you.
And, and then everything should be working very easily and
like allow this kind of collaborative
use of, of these things. So like, yeah, it could be a
personal information manager that has all
sorts of information because I think that's what people
tend to use Emacs for is that, you know,
I have my own notes, my own contacts, my own tasks, you
know, why aren't tasks in, tasks should
also perhaps be in, in, in the database. Like, yeah, it's
kind of, org mode is completely awesome,
like really incredible. And it's, it's nice to mix to do's
with things with text, but sometimes
maybe you just want to do's that are not with text and
maybe those could be also in the database.
I don't know. It's just things I, things I'm thinking about
. I'm not sure how far I want to
take that. Another question. What about using other
database programs like Postgres, MonoDB?
You, so I think you could, yes, you could do that. I don't
see it's a, I don't know why you,
I don't know if there's any advantage to doing this. Like
SQLite is kind of nice because it is
light. It is kind of, it is built into Emacs and we don't
really need anything incredibly
sophisticated here. Just having a database is like such a
higher level of sophistication than
what we've been doing in Emacs are so far. Then you know, I
think the other stuff might be overkill,
but I see no reason why it wouldn't work. I just don't
really see if there's really any big
advantages. What is your preferred reference to underscan
triples graph DBs? To sort of think
about better schema design? Oh, that's a really awesome
question. And I would, I don't, I don't
think I can answer that immediately. I kind of know all
this stuff from talking to people
basically, instead of like reading things. And like in my
work, I use these triples. That is,
that's kind of like, you know, I kind of work on a
knowledge graph. So I just have a, from,
you know, working knowledge of how these things work. So,
but it is an awesome question that let
me, let me look into it after I kind of finished this and I
'll, I'll try to fill it in, in the,
in the either pad that is connected to this talk. So thank
you for that great question.
So another question, will it slow down with the growth of a
database? Yeah, I need to do
tests about this. Cause I'd like to see like how much, how
much the database blows up,
how much it slows down for sure. Triples is not the most
efficient way. It's not even close
to being a greatly efficient way to store data for doing
any arbitrary things. But what it is,
it you're trading off standardization, which is, so you get
standardization, but you lose,
you lose the power of getting things in one SQL expression.
So you often have to kind of like do
a bunch of round trips and that's slow. However, I I'm
using this now. I have a bunch of data
and you know, quite a bit. I've, I kind of poured it over
my old org Rome to, to EKG.
And you know, I've been accumulating stuff in our room for
like, you know, two years now,
and it's pretty big, but it's very fast, really just no, no
problems at all. So yes, I'm sure
that there is a, a size in which it's too big to be fast.
But I haven't hit it. And I suspect for
most ordinary uses of, of, of Emacs, I doubt other people
will hit it as well. I mean, it's kind of
similar to, you know, when people think about org Rome and
things like, well, you know, you're
storing everything in one database, like how many notes
could have possibly hold, like that's also
going to get slow. Right. So yeah, it's, it's I think these
limits exist, but people tend not to
hit them and it's like, yes, this is going to get slow, but
it'll certainly do better. I think that
files in the file system. What are the, here's another
question. What are your thoughts on
allowing for a true graph DB backend? Whatever the current
best free software alternative to Neo4j
is, I guess. Okay. Great question. I've actually, yeah, I
actually used graph databases and several
companies I used to work for and you know, but we always
ended up switching to SQL. I like it's,
it's I can't, it's just at least for serious applications,
they tended to be slow and
lend themselves to a use that was not, that was very iter
ative and not very well thought out.
So however, I don't think it's as I said, with the last
question for the way people want to,
you know, the kind of data I think Emacs users will want, I
think a graph database would work
quite well. I don't know if there would be like, what are
the advantages over triples
aside from sort of just a more, you don't have to kind of
even deal with the SQL layer, but like,
I'm kind of, I'm kind of building, you know, with triples,
we're sort of like, don't even worry about
what the layer is, right? It's just, you don't even have to
worry about SQL if you're just using
kind of the normal functionality. So you can yeah. So, you
know, maybe there are interesting
implications for using a graph database, but I can't really
think of any right now.
It's a good question though. Thank you. - And I'll quickly
add that we have about,
I think two more minutes ish on this live stream. And then
folks are welcome to either join here for
continuing the Q and A or keep posting questions on the pad
and you can continue here in this room
after the stream moves on. - Okay. In that case, I'll,
I will stick around until I don't get any more questions
for a few minutes, then I'll probably
log off. - Sounds great. Thank you.
- Yeah.
- I'm hungry. Did I get writing and recording this? So I
actually did a month ago. So like,
I don't even, I hardly even remember what it was, what that
time was like, but yes, it's,
it took a long time. These things, you know, these
recordings are a little bit challenging. And in
fact, I only kind of screwed up in several instances of the
recording. And like, I just,
I just don't have energy to kind of keep redoing. It's like
, it's a long recording. It's also,
I think I chopped it up into three bits. I'm not sure if it
was that, is that obvious, but like,
I guess, notes for people doing future sessions, it's like,
make sure you have natural pauses that
you can sort of just like record a segment because
something would inevitably go wrong. And then like,
if you're trying to repeat a 20 minute talk, it just takes
forever. Yes. Oh yeah. The recipes.
That's right. Yeah. I, well, yes, I'm glad you enjoyed the
recipe. This is great. You put the
recipes in there. Like one thing that I think is, is useful
for, you know, regardless of what
thing you're using in org-roam or EKG or whatever, you know
, when you're making a recipe,
you know, record in your daily diary, like, you know, when
you, what you did it, if it worked,
like what, what, you know, what adjustments you had to make
, how it turned out, how it could be
better. Then like, if it's linked to the recipe, then you
have a kind of a record of like, oh,
these are, you know, not just the recipe, but like your
experiences making the recipe. I think
it's really powerful because then you can see how, you know
, you can kind of refine, refine and
refine. It's really hard to do this in other ways. Like if
just, you're just using your memory,
you're, you're kind of like, well, it's, I can't remember
what I did the last time I made this
recipe, but like, you know, it seems a little bit extreme,
but I think it's quite useful to use
these note-taking applications for this use case. Okay. I
think the stream has moved on, but again,
you're all welcome to stay here and continue to Q&A as long
as there are any questions.
Thank you for your moderation.
Cheers. Happy to. Thanks for the great talk.
[end of transcript]
While I see if there's any more questions, I'm going to
fill in, I think not, not all
the replies are filled in. So I'm going to start filling in
this, you know,
the pad that's associated with this.
[end of transcript]
Okay.
[end of transcript]
Okay.
[end of transcript]
Okay.
All right. I, there's no one here and I don't like any
other questions. So I am going to log out. Thank you
everyone for your questions.
[end of transcript]