([Amin Bandali]: I mean, we are ahead of the schedule
a little bit, but I mean we also were
a little bit earlier in the day.
Yeah, I'd say that's fine. Go for it.
Take it away.)
[Leo Vivier]: Sure. Thank you. So am I live already?
[Amin]: Yep, you're live.
[Leo]: Hello
Splendid. So I'll just start the timer to
make sure that I don't go overboard.
All right. ([BBB:] you are now muted) Okay, good
to go now.
Hello again, everyone! I hope you had,
well, quite a lot of talks
ever since the last one I did, and all
more interesting
one after the other.
You know, I'm a bit in a bit of a weird
spot right now, because I'm supposed to
be presenting to you (as you can see on my screen)
"One big-ass Org file or multiple tiny ones:
finally, the end of the debate,"
and it sounds about as clickbaity
as you can possibly get with those
topics. By the way,
credit where credit is due, the title
is not mine. It's actually from Bastien
Guerry, the current Org
maintainer. Yeah, I wanted to
talk to you a little bit today about
this question because
if you are used to going on reddit.com/r/emacs ,
you know the subreddit that we have,
if you go on Hacker News often, you know
it's a question that
you see pop up every once in a while.
"Should I be using
one big file, or should I be using
a lot of tiny files?" I believe you know
we've got defenders on both sides. If I
just show you one example...
We have Karl Voit. He's one
of the organizers for the conference.
He is the guy who probably has the
biggest Org Mode files
right now in all the people I know, and
god knows I know plenty of people use
Org Mode.
But if you just look at this line--I hope
it's not too small; you just
make it a little larger--but
Karl basically has a file with
126,000 lines. I'm just going to pause
and try to have you imagine how large a
file it actually is.
Just think about all of these lines
being tasks in your days.
Think about all those lines being about little thoughts
you know that you've had throughout the
day or project that you were working on.
It's massive. You know one of the
problems that Karl Voit actually approaches on
this topic is that it takes him roughly
20 seconds to get his Org agenda going,
which is a massive amount of time.
I mean, we have very fast computers now.
You know, ever since Emacs was created in
1976, computers... I have no idea how much faster
they've gotten.
And yet, you know, for 100,000 lines,
Emacs seems to be choking. It's
certainly not reasonable, in a way, to
have to wait 20 seconds
just for your entire file to be parsed.
So basically what I want to do--
By the way, I forgot to introduce the
presentation, but I'm Leo Vivier. I did this
before, for those who were around.
I help maintain a software which
is called org-roam,
and that's the expertise that I have on
the topic.
Actually, if you go online, I do have a
Github page. I will make sure that you
have all the links available afterwards.
But I do publish my init files,
and you can see, if you scroll at the
bottom, I have a little demonstration
which shows you
the fancy things that I can do with my
Org Mode setup. That might be even interesting
in light of the talk you've just had
about GTD stuff, because
the first one is about how I handle my
projects, the second one is about
the flow from a task as I work on it...
So I won't spend too much time on this, but
basically that's my expertise.
I have spent eight years
working with Org Mode,
three of them actually thinking about
writing packages.
The thing is, if I go into a little
bit of detail (and obviously it's only a
lighting talk, so I won't have time to
actually go really in depth about it), but
there is something
in the Org Mode library which is
called org-element.
You have the name right there,
org-element.el,
.el being for Elisp file. As you can see,
the page is on the Worg wiki, so it's
accessible by everyone.
It's basically the API that
Org Mode uses to parse Org Mode files.
For those who don't know, parsing
means basically
checking a file, checking all the contents
of the file, and extracting all the
information that we need from that file.
As you can imagine, you all have
Org Mode files in your mind,
well you know they can be fairly complex.
You can have properties,
you can have contextual information, like
if you write a line
which starts at column zero (which means
at the left),
it doesn't have the same
meaning, whether or not it is
before the beginning of a headline or if
it is after the beginning of a headline.
It's going to be
relatively different, hierarchically
speaking.
So the problem, when it comes to the question of
many files versus one big file or
few big files, is that we always have to
keep in mind
what org-element wants you to do.
The thing is, there are plenty of
problems when it comes to parsing files,
the first one being obviously
that Emacs is a single-thread
process (or has some threading
capabilities; we're not going to go into
the details right now, that's not my goal).
It makes it incredibly hard to
parallelize parsing processes
with the current technology.
So you'd have to imagine that if you
have a very large file--if you go back
to the example of Karl Voit from before:
100,000 lines--that means that you have
to scan
through every single line, basically.
Because sometimes... Let's just say that
you have a property drawer, for instance,
which tells you, oh okay, this tree
has the tag :foo:. So the problem is, there
are multiple ways for you
to define a tag. You can use the
usual way, which is about wrapping in
columns the :tag: at the end of a heading.
For instance, if I... (I'm not going to
switch to Emacs, that's going to waste too
much time)
That's one way to say your tag. But say,
you have tag inheritance, which means
that when you have a parent with a tag,
you also want the child to inherit the
tag. If you have
first heading with the tag :foo:, you have
the first subheading,
and the tag :foo: is implied. Now imagine
having to do that with a file that is
completely nested,
a file that has maybe 9, 10,
11 levels of depth to it.
It's mind-bogglingly complicated for the
software to do that, knowing that... I've
told you about tags,
but any property can be inheritable.
Anything like priorities, even. Though why
would you do this?
You can have groups. You can have all this.
And as someone who went through the trouble
of optimizing his Org agenda... So
basically, if we go back to the
GIFs--oh god we've already had this discussion
between the "git" and "magit" and now I've
started "gif" and "gif" and I only have one
more minute left to do so, so let's just
say I'm going to say "gif"
just to spite people... So if you go on
the way I organize my agenda, what I did
in order to keep my agenda build time
under two seconds,
is that I've rewritten a whole lot of
codes to be able to parse my Org agenda files.
So the thing is, I'm going to be talking
more about this later.
I only have, let's say, one minute to
conclude.
So as you've gathered, I'm not going to
be giving you the answer right now.
I'm going to be talking about
org-roam a little later, which is about
following the principle of having many
small files.
But as someone who has been using
one large file to manage my life,
you know, I'm sitting on the fence.
I do not know which one is the best,
but I hope that my presentation has
given you a little idea
of what goes on behind the principles.
You also need to think about the
philosophy behind the organization of
your notes.
I hope to be approaching this topic
with you in about
two hours or so (maybe one hour actually).
I'm actually finished. I've decided to
leave you two minutes of questions.
If someone could feed me the
questions, that might be best,
because I don't want... oh actually I can
just open the pad.
I can just open it. Give me a second, okay.
Just loading up. I might stop showing
my screen.
That might make it easier. So I mean if
you can make
myself big now on the screen, that would
be splendid. ([Amin]: yeah sure)
Thank you. Where are we...
Question 12. Okay, so what's better, one
big file or...?
Is it a jab to tell me that I
haven't answered the question because
someone just
asked me the question? Well, personally, if
I were to give you a quick answer in
20 seconds, personally, I think it's a
question that is contextually based.
Do you want something that is efficient
as far as optimization is concerned?
Then you need to think about
this. Personally, for all the
organization that I do,
all this stuff, all the TODOs that I handle,
I like to do this in one simple big file
because you benefit from all the
refiling capabilities of Org Mode,
so I would do that. But for knowledge management,
for note-taking and all this, well I'd
much rather follow the
org-roam way of doing things, which is
about having many
small files. I'm not getting any
more questions. I'm not sure if there is
one on IRC that could be fed to me.
Otherwise, I'm happy to pass over to the next speaker.
By the way, just before I finish, your
world is a lie. It's not a three-piece
suit. I'm wearing jeans below, so I hope
that satisfies your curiosity.
Okay, there's one more question appearing.
"but otherwise one big file to have
everything..." So I'm putting you on the
spot, I believe. It was such a short talk.
You know the problem is,
I just wanted to give you a little answer.
A little, you know, path of thinking on
this topic.
Obviously it's a topic I could be
spending 40 minutes on,
but I'm going to be drained, you're going to
be drained, nobody's going to be happy
if I do this.
Someone asked me if I switch between
British and French accents.
A little secret for you:
when I'm stressed, I tend to revert to a
French accent,
so you can measure the amount of stress
that I'm feeling during this talk
with the amount of h's that I drop and
the amount of sheer fright that you can see sometimes
in my eyes, when I'm thinking about what
to say next.
All right sir. So, Amin, do you believe
we can leave it at that? I'll be...
People will see plenty more of me
later on, anyway.
([Amin:] So, looking at the schedule, I think
your talk has until like 2:02, meaning
like five or six minutes from now.)
Oh, right.
([Amin:] So if you do like to take one
or two questions, to add two more
questions, by all means.)
So someone has asked me what is
the Emacs
icon (sorry, see, another French accent)
here in my status bar...
Oh sorry, I'm not sharing any more.
I might just share again just so that
everyone can catch a glimpse of that.
There we go. Allow...
So it should be... So if you could make me
small again, Amin, I'm not sure if it's
going to do it
by itself, but I do have a little icon
here in my status bar
which is basically a way to interact
with org-protocol.
I'm not going to look for it right now,
but it's a browser extension that is
developed by one of my friends
over at Ranger whose name is Li Fong (??) and
it's very useful. I'm someone who uses a
lot of Org protocols. And by the way,
I used to teach English to high schoolers, and they
were supremely worried
when I showed them my status line and they
saw "kill" and "explore" in my status line.
As fellow Emacs users, you know that
obviously kill means to kill a selection of text and
keep it inside your clipboard,
but for my students, they
were very worried
about what their professor was up to
during his nights.
So let's see if we've got more questions.
I'm showing you the questions on the
rainbow. Let's see if
we've got more. People are posting a
lot of questions now.
So how do you feel about archiving files
in Org Mode and how can that work?
So one of the things when we think about
optimization is: yes, archiving done trees
is a good idea because it means that if
we go back to the org-element, the way it
works (and we'll get into technical details
afterwards; I'm giving a presentation
about org-roam technical
aspects, sorry, so I'll have a chance to
expand a little more on this)
but basically, org-element needs to...
Every time it sees a TODO, it has to consider it,
even though it is a done TODO. Why?
Because let's say, for instance, that in
your agenda you want to activate
log mode, which is going to show the
tasks which are done... Now you could be
clever and say, oh okay, the Org agenda does not
need to show done items, so it's not going
to look for them,
but the problem is that org-element is
always called. It always needs to parse the buffer.
You know, Nicolas Goaziou, who is the
French developer who's worked a whole
lot on org-element
has gone through a lot of trouble to
optimize org-element, but the problem is
there's just so much that we can do with
a concurrent process.
Right now it leaves somewhat
things to be desired, but
we're working on it.
One more time... I feel like I spent
half of this talk
teasing my next talks, but I'll be
talking more about this
in my future talks in about one to two
hours.
So, continuing with questions, how big are
my Org files?
So in the background, I'm just going to
check how many lines I have in my main file.
In my own file, so the one I told you
about where I keep all
my TODO GTD stuff, I have
38,000 lines, which is...
It's sizable, definitely.
But I do archive a lot of stuff,
so that might be a slight difference
between myself and Karl Voit,
even though I don't remember if they
actually archive stuff.
So does it not consume more resources
and time to load multiple files
files than a large file or the same
content now?
Theoretically, yes, having many files
open concurrently is slightly slower
than having one main file opened.
Now the problem is for those of you who
have large files,
you may have noticed that when you are
scrolling in a very large file,
it starts taking quite a bit of time. Why?
It's because in Org Mode,
you have a lot of content
that is hidden, so when you have
the view mode which hides as much stuff
as possible, meaning that you only see
the top heading--and I'm checking the
time, Amin, don't worry, I'm finished
on this one--
when you're hiding a whole lot of stuff,
Org Mode needs to keep track, or I should say,
Emacs needs to keep track of which areas
of text to show
and which areas of text to hide.
The problem is that when you're hiding stuff--
let's say you're moving from the
first heading to the second heading,
but you've got like 10,000 lines between
those two headings--
well, Emacs needs to compute the
difference between the two passages,
and that takes quite a lot of time.
That's why you might realize that it's a little
choppy when you start scrolling in large files.
Anyway I could be answering questions
about Org Mode for literally two hours straight,
so I'm gonna hand it over to the next
speakers. I'll be seeing
you guys a little later. ([Amin]: Thank you
very much, Leo.)
Oh, thank you. ([Amin:] Yes. Bye.)
Bye.