# Org-roam: Technical Presentation
Leo Vivier
[[!template id=vid src="https://mirror.csclub.uwaterloo.ca/emacsconf/2020/emacsconf-2020--18-org-roam-technical-presentation--leo-vivier.webm" subtitles="/2020/subtitles/emacsconf-2020--18-org-roam-technical-presentation--leo-vivier.vtt"]]
[Download compressed .webm video (47.4M)](https://media.emacsconf.org/2020/emacsconf-2020--18-org-roam-technical-presentation--leo-vivier--compressed32.webm)
[Download compressed .webm video (23.5M, highly compressed)](https://mirror.csclub.uwaterloo.ca/emacsconf/2020/smaller/emacsconf-2020--18-org-roam-technical-presentation--leo-vivier--vp9-q56-video-original-audio.webm)
Org-roam is a Roam replica built on top of the all-powerful Org-mode.
Org-roam is a solution for effortless non-hierarchical note-taking
with Org-mode. With Org-roam, notes flow naturally, making
note-taking fun and easy. Org-roam should also work as a
plug-and-play solution for anyone already using Org-mode for their
personal wiki.
Org-roam aims to implement the core features of Roam, leveraging the
mature ecosystem around Org-mode where possible. Eventually, we hope
to further introduce features enabled by the Emacs ecosystem.
The purpose of the talk is to present some technical aspects of
Org-roam. From the very beginning, we wanted Org-roam to scale with
your notes, and this meant that we had to keep a close eye on our
performances. As we iterated, optimisation remained a top-priority,
leading us to constantly peek under Org-mode's hood. Not only has
this made us better developers, but it has also uncovered paths of
optimisation for Org-mode itself.
The talk is targeted at software engineers willing to peek under
Org-mode's hood. A rudimentary understanding of Elisp will be
required.
Points to be covered
- SQL database via emacsql
- Elisp libraries
- Parsing of Org-mode files
- org-elements.e
- Parsing with a background-process
- Ensuring consistency via hooks
- Actual start and end time (EST): Start: 2020-11-28T15.39.41; Q&A:
2020-11-28T15.56.29; End: 2020-11-28T16.01.03
# Questions
## Why not to run a background Emacs for parsing instead of implement a new parser?
Running a background Emacs progress sounds great, but is still
limited. Forwarding all queries to a background Emacs (like org-mode's
exporter does) is only feasible with a (??? zaeph can probably fix the
answer).
## How often does the DB index get updated in order to contain changes within Org files?
Either on save, or on idle-timer.
## Did you ever think of opening up (or designing) the SQL DB as a general Org speedup-tool outside of org-roam so that other libraries that do execute complex queries are able to re-use the summarized data?
FYI, see John Kitchin's work, he uses a SQLite database to index his
Org
files. .
- John's DB approach is great. However, we should not end up using
several DB-index in parallel. ;-)+1
## Obviously with the 'global backlinks' agenda, it would be interesting to combine with the eev stuff from before :-) ()
## About the external program, you could just talk to the PANDOC guys (or Firn [Parses org-files into data structures with Orgize ], Logseq [OCaml & Angstrom, for the document parser ]), they're very helpful and have already a good org-mode parser
## Is it feasible to have this process of parsing org-roam following the LSP protocol? that would allow to be editor agnostic, and it would save the work to define the communication protocol and any other technical details.
# Notes
- "org-roam just wants to create backlinks".
- org-mode has many many files (377 lines in dired… including
.elc files).
- If you want to create an index of all the org files using the native
format, it would be very slow. So org-roam uses a sqlite database.
- ripgrep (written in Rust) is more capable than grep; used by some
Zettelkasten implementations.
- "Is there something we could do to import backlinks into Org mode?"
- "We've always tried to have an experimental ground where we can track backlinks"