Hello, everyone. My name is Ihor Radchenko,
and I'm the new official Org Mode maintainer.
Today, I'll briefly introduce myself
and then share my ideas about the future of
Org Mode development. I will start by passing
the word from Bastien, the previous maintainer.
Then I will tell you a little bit about my story,
starting from ordinary Org Mode user
all the way to the maintainer.
Then I will detail the new directions of development
and specific features which I want to see
in Org Mode in the coming years.
And I will conclude by asking you, Org Mode users,
to contribute to Org Mode
because all the features which are too many
can only be implemented with the help of the community,
with the help of more contributors.
And for that, of course, I will also talk
about Org community and how I see it evolve
so that we have a good communication
between the development of Org Mode
and the community ideas.
Let me first pass the word to Bastien.
Hello, everyone. Ihor Radchenko is the new Org Mode
maintainer.
He's been acting as such for a couple of years now,
and I'm really glad he's finally agreed to take on the
role officially. As a maintainer, I've probably done
a few things right and certainly made many mistakes.
Here are 4 lessons I've learned in 14 years.
The first is that maintaining Org Mode
isn't just about code, it's mostly
about users. Of course, some of them will never learn how to
report a bug, some of them will behave like spoiled
children, and most of them will expect you to work for free
forever. Nevertheless, the time is as valuable as yours.
Whatever they request, there is always something that can
lead to a positive outcome for Org Mode or its community.
The second lesson is that maintenance isn't just about
technical choices. It's also about predictability.
Be very clear and very loud about what users and contributors
can expect of your time, skills, and motivations.
Stick to the robustness principle
by being liberal in what you accept
and strict in what you produce. Thirdly, it's all
about learning. Let's build a culture together where it's
okay to ask stupid questions. No one is born knowing how to
write in English, how to report a bug, or how to maintain a
large piece of code. Remember that old-timers were newbies
and that newbies could become maintainers. We all have a lot
to learn, even if it's just how to respect each other.
Finally, as a maintainer, think about the next one. Who will
be in your shoes next? What kind of maintainer do you want for
a software you will be using for decades? It's also a
responsibility of the Org Mode community. How can we
collectively attract maintainers that want to help us use
and enhance this wonderful little tool? How can we,
as Org Mode users, help Ihor pave the way
for the next maintainers?
Enough said. I'm confident Org Mode is in good hands and I'm a very
happy user. Thanks everyone for all these years of fun and
learning.
Now back to my talk. Let me start by briefly introducing
myself. I'm actually not a programmer. By training, I'm a
material scientist. and I only started using Emacs and
Org Mode and naturally doing programming (because that's
Emacs) when I was doing my PhD and I wanted to use Org Mode to
tame my research work. A couple of years I was just an
ordinary user, until I learned enough and
got enough courage to report my first bug.
Then it all evolved over the years.
I started participating in the mailing list,
I started learning more about Elisp,
I reported more complex bugs,
I eventually got around to go and fix the tricky bugs.
Then I started participating more in Org mailing list
in helping fixing bugs in selected areas of Org
mode and eventually switching to all parts of Org.
At some point I ended up doing the de facto maintenance job
together with Bastien and got an actual maintenance offer
which I accepted recently. The key takeaway I want you to get
from here is that you don't have to be a programmer, you don't
have to know Elisp to contribute, and you don't even have to
be like that to become a maintainer. All it takes is
slow, methodical, persistent learning over the years,
participating in the community, and eventually
submitting your patches upstream. And eventually you can
become a maintainer, or not a maintainer as you wish.
Enough about me.
Let's talk about what I think should happen with
Org Mode and what my goals on Org Mode is. The first and top
priority for me is the basics. The basics, the
code-based stability.
Basically, all the foundations, all the APIs in
Org Mode, all the basic libraries, which everything builds
upon so that we have fewer bugs and we have more
understandable code so that others who want to contribute
have easier time understanding
what is going on in the code base and
contribute without much of a problem or confusion.
Second equally important direction is the Org community,
because a single person, even a couple of people, cannot
really develop such a big project as Org Mode. And we always
want new contributors, which are not coming from nowhere.
We need to have the community of users.
We have a community of
people who participate in discussions, who later submit
patches and code, and that's where we get most of the new
features. So I would like to improve the communication
between Org community and the development. The third
direction I want to pursue is making life easier for
third-party packages. Because we don't only have org-mode
as it's distributed with Emacs. We have a lot of
ELPA packages. We have a lot of MELPA packages.
People who want some specific features
on top of org-mode do develop
these packages which are widely used or not so widely used.
It's not only about Emacs. We have a lot of mobile apps
that can work with Org files, have a lot of parsers which
enable this mobile apps, or in general, programs outside
Elisp to understand Org files.
Fourth direction is the Org markup as a foundation of
org-mode as a major mode, because we have a lot of
functionality in org-mode itself inside Emacs,
but it's all based on the underlying markup,
and markup should have enough features
to support the functions we want to see there.
Last direction, which is somewhat less
important, mostly because I don't have
that much time to focus on everything,
is the new features. I do want to see
certain important features in Org mode,
but I usually cannot spend too much time on them
because of the previous more important parts.
I do rely on the org community and
the contributors to implement these new features.
My idea is that I want to direct which features
and how they should be implemented,
but the actual people who implement
them should come from the community.
Now let's go into the details.
The code base. As a bit of motivation, I would like
to share this email from Richard Stallman, who a couple of
years ago asked about improving Org mode by making it
more modular. That's directly asking about improving the
foundations of the code. That's what I think is
an important direction as well. Here's an example.
We have a lot of really, really large org libraries,
like org.el, which is like one megabyte or something large,
org-agenda, org-table, org-list,
a number of files which are really,
really, really large. What is worse is they are hard to
understand sometimes. I even have an example,
alphapapa complained that part of his motivation
to write org-ql and specifically his agenda part,
agenda-like part, was
because org-agenda is so hard to understand that it's
easier to start from scratch.
I don't want such situations
to happen in future, if possible at all.
One of the projects
I'm currently working on,
it's work in progress now,
is splitting Org libraries into smaller parts,
into more documented parts,
into the APIs which are actually
documented and explained in the code at least,
so that people who just open org-mode code
cannot be scared away and go and
read the comments and understand what is happening there
easier. At this point, I have almost doubled the number of
libraries. It's still work in progress, so there's a lot of
room for improvement in this area.
Another direction which is
somewhat reasonable in relation to splitting things down
is that some libraries are really generic in Org Mode,
because Org Mode often has a functionality
which is really new, and for that,
it had to implement some very generic
functionality that doesn't have to be used
just for Org Mode.
This can be generalized for Emacs in general.
There is a number of libraries
which we may or may not upstream to
Emacs, depending on what Emacs maintain, I think. As one
example is org-capture, because it's a very obvious
example. org-capture started as support from remember.el,
which is still a part of Emacs. It has more features than
remember.el, which we required for Org. But, you know,
these features can be backported. Why not? And then
not only Org-mode, but other Emacs libraries can benefit
from these features we have in Org-mode only.
Similarly, I have a long list of
different libraries that can be shared.
Yeah, for some I'm not sure,
but in general, there is a lot of
work that may be done and may be discussed in the future.
The third part about the basics is making use of the new Emacs
libraries. Org Mode in general is quite well written in terms
of Emacs integration. We do support many of the Emacs
features and libraries which are generic. However, in more
recent Emacs versions, we started getting some new
features, and we do want to make use of them in Org.
For example, recently we contributed yank-media support for
clipboard pasting and drag and drop. Now it is supported
in Org mode already in the released version. Eventually
we want to support transient.el, because now Org uses ad hoc
system. It's of course much better to use existing and
more powerful menus, which are implemented in
transient. It even has some initial work-in-progress
implementation. I hope it can be eventually
extended to the whole Org Mode
There are other things like compatibility, which there is
an excellent Emacs library, compat.el, that provides
backwards compatibility and Org also has something like this
in org-compat. We don't have to write it ourself again.
We can make use of the existing library.
Similarly, there's a very, very new library
track-changes for tracking changes
in real time. Eventually, if you want to support
context-menu mode, maybe touchscreen, Android
support, I don't know, but I hope it can be done by someone.
And some more generic library: select thingatpt.
That's about using external APIs.
Now about the internal Org APIs.
One important, probably one of the most important
parts of Org is the parser, how Org itself understands the
Org files. The situation is that we have two parsers in
Org mode. One is the Org element, the proper parser, which we
use as a reference, but many parts of Org still use regular
expressions, which are approximate. These two parsers
are not exactly consistent, which is really bad, and I hope
to solve this. I already started doing some work by
factoring out some part of abstract syntax tree and working
on real-time parser, incremental parser, which is enabled
by default in Org 9.6, but there are still parts which I need
to work on. Eventually I want to get rid of regular
expression-based parser completely, so that we don't have
any inconsistencies inside Org Mode. One of the examples of
these parts that are still using regular expression is
fontification, which is often simply wrong, especially in
some edge cases, and we really want to use the proper parser
in this area. Maybe even editing org files using the parser
syntax tree, but that might be tricky, although there is an
existing library that implements some ideas for this.
The key point is that org-element-api, the parser, should
eventually be used everywhere
so that everything is consistent.
The second important API is the Org babel.
Currently, Org babel does have some API,
but first, it's not well documented.
Second, it's sometimes awkwardly
designed, especially compared with the exporter.
I do want Org Babel APIs to be more consistent.
Another thing about Org Babel, it's not exactly API, but you
know that documentation for most of the Babel backends are
not even in the Org manual, even though the backends are
built-in. They are on Org Wiki, and we do want to move them to
the manual eventually. That's the important part, and it
should be done. Those are some obstacles, like not all the
features are properly implemented, and that's a bit of an
extra job that should be done. Another small thing which
thanks to Bruno Barbier is being done, in progress: we
should have a more robust asynchronous API for babel.
I hope it can progress further.
For now, it already progressed quite far.
That's all about the basics, the underlying
backbone of the Org codebase. Let's move to the second
important direction which is
the third-party packages and basically the parsers
for mobile apps. I will postpone the community to the end
because I want to have a call for contribution at the end.
For third-party packages, I would like to remind you that
Org mode used to have something called org-contrib as a
part of Org mode, which is a collection of small libraries,
small packages that didn't have a proper
copyright assignment basically,
but more or less a part of Org mode. This is
no longer the case. Now what we did is
we moved a number of very rarely used libraries
from Org core itself to org-contrib,
and now we treat org-contrib as basically the libraries
that we really want someone to take responsibility for.
We want to maintain this for everything that is in
org-contrib, and from me and other Org team, we
do not spend too much time maintaining this package,
just do some most basic bug fixing, and that's all.
If you know, if you see
some libraries from org-contrib and you use them,
and you know Elisp, please volunteer to be
the maintainer, because otherwise
there will be not much progress in these libraries.
As a natural extension of this and inspired by
Tarsius's Emacs Orphanage idea, we also maintain a small
page basically listing the libraries, some others like
packages, Emacs packages that are not really maintained.
If you are a maintainer of a library and you don't have time
to do it, you can write to Org mailing list
and we can add the library to this page
so that we can search for new maintainers
in a more centralized way. If you are an
Elisp hacker and you want to help something for Org Mode,
you can check that page and see where you can help.
Now away from Emacs, or mobile apps.
We have quite a lot of mobile apps at this point.
Unfortunately, it's very hard
for me and many other Elisp contributors to contribute to
these apps because they are not using Elisp naturally. But
these applications heavily rely on Org markup. I do hope
that we can keep Org markup consistent enough and rich
enough so that people don't have to invent extensions to Org
like what happened to Markdown.
I really want to emphasize that
I want to see more Org parsers in different
languages so that they can be used by developers.
For people who are writing these parsers,
I want to share this link. It is the org-syntax reference.
It is the official Org syntax,
which is what we think it should be.
It's described in plain human language.
It's not a code. All details should be listed there.
Please use it as a reference if you are writing a parser.
Eventually, this document will be
submitted to IETF, I hope. In the future
I hope to write a set of tests which will work as benchmarks.
basically we have some existing tests for our internal
parser and I want to factor out these tests so they can be used
by any parser, so that we can compare the performance and
which parts of Org mode are parsed and which parts are not.
I mentioned that we want to submit to IETF, which means that
Org markup will become the actual registered format.
But before we do that and thus fix Org markup in stone,
because it's very hard to change things in the IETF,
it's important to address important problems,
existing problems with Org syntax.
There are some problems, like I mentioned the
inconsistencies between the two existing parsers
in Org mode. There are also some parts,
there are some examples, like there are
problems with numeric priorities, for
example, which are not treated consistently. There are
problems, more general problems with syntax when people
request some edge cases which should be addressed. Like,
it's very hard to do interword markup.
We have zero-width space workaround, but many people dislike it,
so maybe we want to do something about it.
We have some edge cases when we
combine emphasis with links. We have some edge cases when we
have double blank lines inside some source blocks, for
example, and combination in the list. I hope we can
somehow address it. It's not impossible to do it, it just
requires time. One annoying part is the inline task syntax.
It's annoying both from the programming perspective,
internally as implementation, and from the UI
perspective, because there are too many stars. We
probably should redesign it eventually, maybe in
backwards-compatible way, but we will see how it goes.
Another part is not just fixing the edge cases or problems,
it's the completely new syntax features. That's probably
done after we submit to IETF. But there are important things
that people often request, like time zone support in
timestamps. Better repeaters, like more flexible
repeaters, that's really a frequent request also.
Another idea is some custom markup, which is
coming to various requests, like, for example, people
often ask to highlight some words with a color,
for example, or with some other special way
and then export it in a special way,
just as we do with special blocks, basically.
What I want to introduce is the ability to do it
on a macro level or inline.
Of course, a new syntax feature which I wish we could
have is the multi-line cells in tables.
It's very frequently requested as well,
but I really have no clue how to do it.
We had a discussion about this in previous
discussions, but there was no conclusion. We don't see a
good way how to implement it syntax-wise.
Unlike time zones where we decided exactly the syntax,
how it could be and we just need patches to be submitted,
here even the idea of syntax is not clear.
Please do participate in these discussions
if you have ideas.
The last direction is the
new features. In general, I welcome all kinds of good
features, but there are certain things which I explicitly
want to see and I hope to see submitted. If you are
interested, please do submit patches. One, and probably
many people are aware about it, is the asynchronous LaTeX
preview, developed by Timothy and Karthik. I hope it can be
finalized eventually and upstreamed. It's pretty much in
ready state, but on the technical level it should be
discussed further and revised. The second is org-ql by Adam.
I hope it can be upstreamed. It's also a work in progress.
It's just a question of free time for Adam mostly, I think,
and me. That's another important part, new feature. The
third is the so-called multi-page export. The idea is the
same as many packages for blog posts, so that you have a
single Org file and then you can export multiple HTML pages,
for example, or PDF pages, anything like that. This work in
progress by Orm, thanks to him,
although it was a little bit stuck because
I am not exactly sure how to best integrate it
into the existing APIs. If you are a developer of one of the
blogging packages, I would appreciate if you can chime in
and probably share some ideas here. Next are just some
wishes I wish we could have, but
it's not very detailed. One is
the multi-language support, so that we can have Org
documents in multiple languages, or maybe even
translations. The collaborative editing that many
people would wish to have, I think. Things like tracking
changes, adding comments, importing from
some other Org formats with
the comments and changes
so that we can actually participate
with all those Microsoft Word users
and stuff like that.
But that's really too much for me alone to handle.
If you want to see one of these features, please consider
contributing. Just write the mailing list about your
interest and we can start from there.
Now, the important part is, you see,
I keep asking people like,
please contribute, please contribute,
but who should contribute?
So I want to improve communication between the
community and the mailing list. Now, people often
discuss new features or ideas on all kinds of places like
Reddit, Mastodon, like all kinds of Matrix/IRC chats,
even on meetups, some non-English language. That's very
nice that we have this community, but not all the ideas are
visible to the developers. I do wish that the most
important things that people want to see should end up on the
mailing list, one way or another.
I'll later talk about some
ideas how I think it can be done. Another part is
we have org-wiki and I hope that
we can make it more centralized
space for interesting Org mode articles,
for tutorials, for blog posts,
at least linked to blog posts. If you have
some idea about good blog posts, it would be nice if you
submit a patch to work or at least email about this link
to the mailing list.
Of course, not everyone likes to use mailing lists
or don't have a good setup to do it,
or even don't want to read
everything on Org mailing list, because there are things
like bug reports, people don't really want to see that. So
Bastien actually developed a tool that can help with this.
If you want to monitor Org mailing list, but want to see
only the most important discussions, and maybe
participate if you decide to, you don't have to register. We
have Woof, which is basically a web page that monitors our
main list, but not every email. That's the most important
announcements, some blog-like posts, or feature
requests, or some discussions. Then on this webpage,
you can see it as HTML, or you can subscribe as RSS, or even
download in Org or MD format.
Thanks to Sacha Chua, also weekly news about Emacs in
general, but Org Mode as well. It also includes the
new features in Org Mode on the development branch and the
interesting new blog posts and discussions on various
Reddit forums or mailing lists everywhere. I did this
little bit of experimental integration so that many lists
can also be read, kind of announced on the chats.
For example, in #org-mode Matrix room, we have a bot that
connects to Woof RSS so that all the news and discussions are
notified in the chat so that people
can see if they are interested
and maybe, hopefully, participate. I wish
we could also have similar kind of both for Reddit, Mastodon
and maybe IRC. That way we have mailing list connected to
more active and more modern forums and chats.
More people exposed to what is happening.
Another part is that we actually have a web interface to Org
mailing list and you can even reply from there,
but it's not always obvious, unfortunately.
We have this public inbox software
to transform the mailing list into
HTML pages. That's a decent interface, but it could be
improved to look more forum-like, so that people can easily
find the reply button or basically participate without
too much effort, even if they are not subscribed or they are just
casually reading. That would be nice if someone
knowledgeable of CSS could help with this.
Again, I cannot emphasize more that most of the Org ideas of
the new features are coming from people,
but often they are known either by someone
submitting a patch to the mailing list or
submitting an idea to the mailing list. Rarely, it happens when
someone is reading posts from Reddit that don't share to the
mailing list. If you think that there is some important
discussion happening on the forum,
it would be nice that you can go ahead and share it
with Org mailing list. We don't care about
on-topic, off-topic, because unlike
emacs-devel, we don't focus on development.
We can discuss some related to Org mode topics
in open-end list, just like on an ordinary forum.
I would like to end my talk with the call for contributions.
Let me explain a little bit, because you saw now I shared many
many ideas and I do spend a lot of time on Org Mode. In fact,
for the last year, I was spending like
30 hours per week or something.
Until recently, I can spend a bit less, but still
even with that much of commitment,
there is no way I can handle
everything by myself.
We really, really need more people to contribute
so that Org mode keeps moving forward.
Again, I just shared ideas,
but I have so many more.
I have more than a thousand ideas noted down.
Yeah, there's no way I can do it myself.
I do need help for new contributions, for new features.
If someone can help fixing bugs, it would be really great.
If someone knows specialized things,
like for example Open Document Format,
which I don't know very well, it would be
really nice, because things like ODT export requires
knowing that, and if someone already has the knowledge,
it would be much easier
if that person can help with such things.
Although in the coming years,
it's very important to have
regular contributors
because my life is my life
and things may happen,
so it would be nice to have some kind of backup
so that bugs keep being fixed
and things like patches being accepted
and stuff like that,
so that some person can at least
temporarily take on my job.
I would like to emphasize
that the code contributors are the most
important contributions for Org Mode.
All other types are less important, really.
I want to spend a couple of slides
trying to motivate you to contribute.
If you ever considered contributing
or if you ever wished to have some
feature in Org Mode, really don't wait, because
most new features are contributed by people.
They are not contributed by me.
I contribute a few things, but I am alone.
I cannot contribute many things.
Most of the things are contributed by users
who go ahead and submit patches.
And even in very commonly requested features,
it's usually someone who steps up.
If you want something, don't wait.
Just go ahead and write to mailing list.
You don't have to submit a patch immediately.
You can just say, I am interested.
We will start from there.
We'll start because that's my job
and I will guide you through.
If you have problems with Org in general,
I'll explain, because that's what I know.
I will explain how to implement things better,
but I need someone to actually do
the job and write the code.
Even if you don't have 10 years experience with Elisp,
it doesn't matter. You can learn on the way.
There are many examples of this in the
mailing list when we start slowly,
fix things one by one, and
eventually arrive to a good quality
when the person is not experienced.
You just need to be ready to learn things,
Spend your free time (that's as usual, right?),
and have the interest in specific thing
you are contributing.
Don't be afraid to be wrong.
I will be there to assist.
If you don't like emails,
if you don't like patches, it doesn't matter.
Again, you can share GitHub link. It's okay.
You can go and modify some workflow directly.
You submit the modified version. It's also okay.
It's easy for me to create patches if I need to.
If you don't like emails at all, you don't want to
participate in the mailing list, it's not ideal, but I can
still work with this. I am on IRC, I am on Matrix,
you can ping me, it's yantar92.
We also have monthly meetup,
so you can go and ask by voice,
we can just talk in person, discuss your ideas,
it's also fine.
The key point is that we always welcome new contributors.
The more contributions to Org Mode is better.
To increase the motivation,
I'll just try to show some benefits
of contributing to Org Mode and free software in general.
It can be actually useful for your CV if you're a programmer.
It doesn't matter if it's Lisp, because you can have a pet
project that demonstrates your skills, that you can finish
something to a usable state.
A pet project usually demonstrates
that you can work alone, but it doesn't
demonstrate anything about you working in a team, in the
production team. When you contribute to
Libre software, look,
you will work with a number of people
who contribute and comment on your work.
You'll have to learn a new code base.
You will have to follow certain standards.
All these things, by having a public record
of contribution, will be a valid point that proves your
knowledge in your CV. I put a small quote
(which I'm not going to read in the interest of time)
from Rudolf Adamkovič,
who is describing these three points in probably a
more expressive way.
Another benefit is you can actually get money from this.
Thanks to a number of Org Mode users who kindly contribute to
Org development, we have some amount of money
coming in, and we don't hold on this money.
If we get another person who contributes
to Org regularly, we are happy to share this money,
because we do know that
getting some extra money, even a little bit,
does improve motivation.
We are really ready to share this.
If you are serious about contributing,
you can just request this
and we can share a part of the donations to you.
For non-programmers, we also have a lot of work to do.
There's a lot of stuff we can do on the Org wiki,
especially with CSS styles,
with updating articles,
with adding links to tutorials,
or even writing tutorials about Org Mode.
It would be nice to have more screencasts.
It would be nice to improve Org manual,
because that's the most difficult part for me
because I'm too familiar with the code.
Writing the manual when you know things internally
is very hard. I just know too many things.
I can assume that people know too much,
that new users have no idea about certain things.
I can just omit those without being aware.
The final slide I would go to come back is about
donations. Again, most important is contributing code,
but I do hope that donations can
increase the number of contributors.
I don't know. I know for sure because some
people like Timothy, who is participating in this, he did
find the donations helpful, especially for more boring
tasks like bug fixing and to move things
over the long time. If you cannot contribute by other means,
it would be appreciated to contribute money.
We came to the end of my talk.
Thank you for your attention.
If you have any questions, please feel free to ask.
I think we can even discuss further
during the next Org Meetup in the coming week.
There, if you have more detailed questions,
we can continue discussing
apart from what happens after this presentation.