# Org-mode and Org-Roam for Scholars and Researchers
Noorah Alhasan
[[!template id=vid src="https://mirror.csclub.uwaterloo.ca/emacsconf/2020/emacsconf-2020--17-org-mode-and-org-roam-for-scholars-and-researchers--noorah-alhasan.webm" subtitles="/2020/subtitles/emacsconf-2020--17-org-mode-and-org-roam-for-scholars-and-researchers--noorah-alhasan.vtt"]]
[Download compressed .webm video (22.2M)](https://mirror.csclub.uwaterloo.ca/emacsconf/2020/smaller/emacsconf-2020--17-org-mode-and-org-roam-for-scholars-and-researchers--noorah-alhasan--vp9-q56-video-original-audio.webm)
[View transcript](#transcript)
Org-mode improved so much over the years, and the use-cases in org-mode are
vast and highly technical. There is something for everyone in org-mode, and
it's important to sift through all of these features and figure out what's
best for a given situation or specific users. Therefore, I will be
targeting academics and scholars that are engaging with literature in the
early stages of a project or their academic careers.
Academics and scholars engage with complex ideas and unstructured research
workflows. I believe that org-mode can add more structure to the madness,
and I will use this talk to clarify a possible solution to reduce such
complexity. I propose a research workflow framework that utilizes
org-mode, its raw form, and its many associated packages. However, the main
package I will be mostly talking about is Org-Roam, and the way its
underlying principles will revolutionize the research workflow.
This presentation will help researchers organize and build their knowledge
database in a streamlined and effective way. The research workflow is
presented in three phases: planning, note-taking, and reference management.
I will talk briefly about the packages and special-use cases for each stage
and learned lessons along the way. Finally, the presentation concludes with
future considerations and possible org-mode features.
- Slides/presentation: <https://github.com/nalhasan/emacsconf2020>
<!-- from the pad --->
- Actual start and end time (EST): Start: 2020-11-28T15.17.33; Q&A:
2020-11-28T15.32.18; End: 2020-11-28T15.39.00
# Questions
## I use org-roam-bibtex to take notes on particular academic papers in conjuction with org-noter. This means all notes for a given paper are in one Org file. However while it is possible to link to headings within a file, there is no functionality to easily search through and link to these subheadings. What do you do to overcome this? I've only superficially looked at org-rifle as a possible method.
## Whats this presentation software? Looks really cool.
beamer (LaTeX)
<https://github.com/nalhasan/emacsconf2020> for the
slides/presentation.
## How does the view for time blocking works?
## have you seen the project Papis? <https://github.com/papis/papis> I think the author is working on an Emacs package, what would be your thoughts? (it's a Zotero alternative)
"Powerful and highly extensible command-line based document and
bibliography manager."
## Did you try using ebib instead of Zotero? If so, is Zotero better in some way?
Zotero has a lot of plugins you can play with and so far it's been
great.
Some people have been using a connector between Emacs & Zotero….
You can create groups for collaborative projects in Zotero and this is
a plus. (thanks for the answers! I'll give it a try!).
- <https://github.com/papis/papis-zotero> maybe useful ^^
## Do you have any suggestions on what subjects/things should be tags/separate org-roam files for cross-linking? I've been struggling with whether making almost every term be a link or only using links for broader subjects.
"Should I be combining ideas together into one…?" So far I've
been using the Org Roam default way.
## Meta question: is there a place where people are collaborating on research "about" Emacs?
Definitely interested, but there is no place (yet!).
# Notes
- org-inlinetasks.
- If you're working on a big Org file that you keep coming back to,
it's better to keep track of todo's related to that file within that
file (e.g. a paper that you're writing).
- <https://github.com/alphapapa/org-sidebar> to keep track of todo's
within a large file.
- Using org-gcal to sync gmail calendar with org-file
<https://github.com/kidd/org-gcal.el/>.
- org-transclusion <https://github.com/nobiot/org-transclusion> to
show (parts of) other files inline and allow editing in a separate
mini-buffer.
- There is a Slack channel for org-roam link/backlink pls?
<!-- transcript: 2020/subtitles/emacsconf-2020--17-org-mode-and-org-roam-for-scholars-and-researchers--noorah-alhasan.vtt -->
<a name="transcript"></a>
# Transcript
Good afternoon or good evening, everyone. Today, my talk is going to
be on Org Mode and org-roam for scholars and researchers. Leo has
talked about the overall picture of org-roam and org-roam-bibtex. I
will be talking more about the research process itself using these
tools.
(00:00:20) All right. So, just to introduce that the research process
is really messy. You're always working in piecemeal tasks and things
move around all the time. There needs to be a system where you can
organize all these tasks, all these ideas in a way that is flexible
and effective. So my motivation is that research is hard and writing
about it is even more difficult. My goal is to add some structure to
this whole madness.
(00:00:51) Here's a list of some of the stuff that I've been using
since I first learned about Emacs in 2019 and what I've found useful
within my research process. I've organized Org Mode for Researchers
and Scholars Within the Writing Process into three modules. First,
there's the planning aspect of it, then you've got the writing and the
reference management, which I will join together by looking at the
example of doing your literature review.
(00:01:21) When we're talking about planning, we're talking about
either task management or time management with task management. You've
got Org Mode's TODOs, tags, and categories. These are really powerful
tools that you could use in your Org files to organize your tasks and
your appointments. There are different types of TODOs that you can
either set globally in your init file or they can be
file-/buffer-specific. That means, based on context, based on the type
of manuscript you're working on, whether it's a literate programming
report or your actual thesis/dissertation.
(00:02:03) Also, these TODOs are either created as a subtree, like
think of them as headings and sections if you use LaTeX, or inline
tasks, which are like Org inline tasks. I like Org inline tasks
because I can add TODOs between two paragraphs. That way, it doesn't
show up in the table of contents when I export into PDF, HTML, or
anything else. So this is an example of buffer-specific TODOs, and
this is an example of a little programming report that I was working
on where I was dealing with data and analysis and all of that stuff. I
needed context-specific TODOs to use them within this buffer. That's
how I would organize it. There's also another example of an Org inline
task where you could see it in the middle between the two headings.
That way, it wouldn't show up in the table of contents, and it would
look neater within the text when you export it. I also added a tag of
:noexport: so it won't show up at all when I export it into either
PDF, which I use all the time.
(00:03:19) Another useful tool for the research and just general
planning is the org-capture. When I first started with Emacs,
actually, it was for org-agenda. I went crazy with my capture
template. I created a template for everything because I was just so
excited. But with time, I was using less and less of them, so I kept
taking them out. Now this is my simplified capture templates that I
use, either for a general TODO, for a regular appointment, a fleeting
note, research tasks (because those are what I focus on, like my bread
and butter), and then finally with meetings, which I find sometimes I
don't use it as much because I would just have the Org file ready
instead of needing to capture, you know, open a capture template.
(00:04:07) Right. Org Agenda. That's how I got into Emacs. I needed to
organize my life. I found Emacs and it's been great ever since. It
populates all your TODOs and appointments into a singular view. So the
default view, I think, is a week view. However, I use
org-super-agenda. Love this package. I set up my agenda as a daily
view with appointments, deadlines, and a habit tracker. A side note,
you guys: I'm still struggling with organizing the perfect agenda, so
it's a process. Take it easy, all right? So this is just an overview
of my daily agenda. As you can see they're just appointments that I
import from Gmail using org-gcal; a simple habit tracker of daily free
writing-- as you can see, there are a lot of times where I'm skipping,
and the asterisk is the one where I've completed that day, so, you
know, it's a process--and then regular deadlines. So what happens is
that I have other TODOs that I have not scheduled or not added a
deadline but they're just tasks that keep piling up.
(00:05:19) When I first started with Emacs and org-agenda, I had
everything in there, and it got overwhelming. Then I decided, no, I'm
not gonna even let them show up. So what I would do at the beginning
of each week or the night before, I would sit down, look at all my
TODOs that I have not assigned yet to a deadline or a schedule or just
a simple timestamp, and I would organize them throughout the week. So,
here's an example of what I did. On that Wednesday, from my Gmail, I
had all these appointments, but one of them is I have a writing group
session. So I looked at my tasks and I thought, okay, then I will just
assign, for example, my Emacs slides or the framework diagram into
that writing session. All I did was just add an active timestamp. That
is all I needed to do, and it went straight into my appointment. Now,
if I miss that, it won't show up on the next day. So if you put in a
deadline, it will show up as an overdue, but if you have no deadline
or schedule, it will not show up in your daily org agenda. So, just a
star.
(00:06:32) All right. Another way of accessing your TODOs is that if
it's Org file-specific, buffer-specific, and so... Like when we talked
about like whether to have a big-ass Org file or like tiny files, it
all depends. This isn't the... you know, the way this depends, because
if you're working on a dissertation, it's a huge manuscript. You need
to work on that Org file all the time. Then yes, my TODOs should be in
that file specifically, because every time, if I'm visiting this Org
file all the time, I should be able to just look at my tasks from
within that buffer. And so I use org-sidebar to keep all these
specific TODOs within that Org file. I find it helpful. Okay.
(00:07:26) Now that we're going into the writing and reference
management... We'll call it a literature review. This is something
I've built as a schema. I think that it works for now. It requires one
outside software, which is Zotero, what I use. It's an open source
reference management software. It's great. But the thing to keep in
mind is that I use two plugins that are really needed for when we work
with org-roam-bibtex, org-roam, Org Mode, and the ZotFile. Better
BibTeX organizes your reference keys in a way, in a fashion that works
for you. For me, all my reference keys are last author and year. With
ZotFile, I let it rename all the PDF files the same way that I have
for my Bib keys, which is last name of author and year. All right.
Once you export your entire library as a Bib file, then you can work
on it within Org Mode and Emacs using the following packages. So with
org-roam-bibtex, it creates an Org file for each Bib entry. You have
the option of templating and doing other stuff with it.
(00:08:52) Then finally, there's this orb-pdf-scrapper. I've used it
briefly but I think the potential with orb-pdf-scrapper is if you're
going to do a bibliometric study or a systematic literature review,
there's something there, but I have to look through it. Anyway, so
once you create your reference file of reference X and you're writing
your notes, you can either go... Like, with going through Org Mode,
you're writing your ideas, you're writing your notes, you're assigning
tasks, and then there's org-transclusion, which I will mention briefly
at the end, and ways to extract. If you're going to go through the
org-roam... Things that you're going to use within org-roam... It's a
great way to build your database. You start making the connections.
You can visualize your notes and how these references are linked to
each other through the org-roam server or org-roam graph. All right.
This is just notes for later.
(00:09:53) So this is an example of an org-roam file that I have. For
example, if I'm working on adaptation policy, I have these hyperlinks
that are linked to other concepts and ideas such as either climate
security, changing global environment, so on and so forth. The
backlinks are other references that talk about this specific concept.
This is really helpful. Then, when you visualize it, the picture on
the left (which I'm sure looks really small), you can see the
connections that it's making with other references. Of course, this is
just like a buffer network. When you look at the entire database
network... It's growing. Okay.
(00:10:38) So going into org-roam-bibtex... It utilizes a combination
of the org-ref package, helm-bibtex, bibtex-completion. It works with
org-roam functionalities and other good stuff. This is an example of
my org-roam-bibtex file. All right. So I've created the template which
I pretty much use, what Leo has produced in his tutorial. I think it's
great. It works well for me. What it does is that it works with your
bib file. So if you're in your bib file, you have a sub entry that's
called keywords, and usually that's within a journal article. The
author would specify these keywords. When it gets imported into
Zotero, it extracts those keywords and then it gets populated as an
Org file with org-roam-bibtex. I always start with the meta
information first, and then I would write my notes after that. This is
an example, though, for reference of a physical book, so I don't have
a pdf file for it. So when I've figured out a new idea for it, if I'm
writing notes on it, I would create a property that says pages. That
way, it's easier for you when you go back to citing certain ideas or
something, that you have the pages prepared there. It's easier that
way. Okay.
(00:12:07) org-noter which is something I use a lot, especially with
journal articles that have PDFs and stuff like that. They're really
helpful if you are going to... If you've just started using Emacs and
org-roam, and you have all these PDFs that have all the annotations
and highlighting and all that stuff, with org-noter you can just use
the org-noter-create-skeleton command and it will populate all your
notes that have already been entered within the PDF file if you're
using an outside software, and creates them as a neat Org file. I
highly recommend.
(00:12:52) Finally, org-transclusion. I think this is still in its
beta phase, but I've been enjoying it so far. I'm guessing people know
what transclusion means, which is like copy-pasting text from one Org
file to another. This is helpful. I think I peeked at a question that
was talking about linking to other Org files. I think org-transclusion
could really work. It's equivalent to the include function within Org
Mode, but I think... So if you have other files that you know which
region that you need in another file, you could use the #+INCLUDE, but
with org-transclusion... It's great. I mean you just have... you're
just linking one part to the other. Sort of... Like, not refiling, but
you know hyperlinking. So this is an example of what org-transclusion
looks like. The highlighted problem statement is from another Org
file. Then what I would do is just link it to there, and there was a
transclusion command. I wish I made another screenshot of it. When you
invoke org-transclusion-mode, it turns... It prints it out like that.
It's in view mode. Then when you want to edit, it will take you back
to that buffer and you can edit the text however you want. All right.
So, thank you so much.
(00:14:22) I wanted to leave room for questions, but special thanks to
all the folks that work on org-roam, org-roam-bibtex, org-roam-server,
org-transclusion, and of course alphapapa on org-super-agenda and
org-sidebar. That's how I got into Emacs. Thank you.
(00:14:43) (Leo: All right. Well, thank you. So yeah, this time I'll
be the one asking the question and not Amin. I'm filling big shoes
right now, so you'll have to bear with me folks. So thank you so much,
Noorah, for your presentation that is incredibly interesting. Would
you mind if I fed you questions from the charts?)
(00:15:00) Noorah: Go ahead. (Leo: Okay. so the first one I've picked
on my end was "Did you try using ebib instead of Zotero, and if so, is
it better than Zotero in some ways?) Noorah: No, I have not used Ebib.
I've only used Mendeley and then they got bought by Elsevier, and so I
was, like, okay I'm done, I'm going to Zotero. There are a lot of
plugins with Zotero that you can play around with. I can't speak for
Ebib, but definitely Zotero has been a good experience so far. (Leo:
Yes, same. I also do research on the side; as I told you, English
major, and yeah I also do Zotero. Some people have been using uh a
connector between Zotero and Emacs which has... they've had great
success with them but personally I haven't touched it already, so
yeah. Oh, go ahead. Sorry.) Noorah: All right. So far, I don't have
any problems with Zotero, but maybe if I run into something, I might
check out Ebib in the future. (Leo: yeah, definitely. I think zotero
is a very solid project. You know, the fact that it's being used by
people outside of Emacs also ensures that there's quite a lot of
backing behind the software, which is reassuring when your livelihood
depends on your research.) Noorah: Right. And then I think one more
thing with Zotero is that you can create groups, so if you're in a
collaborative project, you can create a reference, a library just for
your group, and I think that could help. I'm going to be in a project
next semester that requires that.
(00:16:27) (Leo: Yeah, definitely. I believe the ability to have
folders inside Zotero makes it incredibly useful to manage your
different projects, concurrent projects. So moving on to other
questions, do you have any suggestion on what subjects or things
should be tags or separate org-roam files for cross-linking?)
(00:16:46) Right. So far, now, I'm having trouble with "should I be
combining certain concepts together as one?" This is where the thought
process starts coming to fruit, is that when you start combining ideas
together so you won't need a specific tag and another one that are
like similar in ideas... I'm not sure if that answers the question,
but so far I've been using the org-roam the default way, which is many
small files and then just linking them to my... Like, either if I have
a report to write, or if I have an essay to write... I think you're
muted.
(00:17:35) (Leo: I did two stupid things. The first one was spilling
out my water. The second one was speaking without actually turning on
my microphone. Let's just hope that nothing is going to fry in the
near vicinity of me right now. But yeah, I believe you've answered the
question, so don't worry about it. I'm slightly wet right now, which
is not a very agreeable feeling, but we'll have to carry on, I
suppose. Another question: "is there a place where people are
collaborating on research about Emacs?" So do you want to try to take
this one?)
(00:18:06) Noorah: I don't know, but I'm definitely interested in the
user experience of Emacs, so if anyone wants to work on that, I'm
happy.
(00:18:16) (Leo: Well you do have a a pretty good candidate in front
of you, if I should say so myself. I'm incredibly interested about the
ability to do research in Emacs and about the ability to preach the
FLOSS way to academia and to the academe, especially because I believe
there's really something great to be done. Sorry, I'm just looking at
the puddle of water on the side which is slightly oozing my way, which
is not a very good feeling, really. I believe some work could be done,
and if people are interested in the chat right now, do get in touch
with us. Both Noorah and I are on our Slack channel. Yes, I know,
Slack, the corporate hive mind that is Slack. But we've decided with
org-roam to use Slack. You can find us very easily. If you want to
talk about these topics, by all means, do, and we'll be very
interested to answer your questions.)
(00:19:10) Noorah: I have a question here that says, "How does the
view for time blocking works?"
(00:19:15) I use org-super-agenda, so what happens is that my active
timestamps are only in my Gmail Org file. If you use org-gcal, you
have to specify a certain Org file. When it imports them, it imports
them as active timestamps. I make sure whenever I create a TODO or
even a research task that it doesn't have a timestamp on it, because
what I want to do is go back and then move around these tasks
according to my either weekly schedule, or monthly, or however long
you want to do it. So yeah, only active timestamps or deadline um
appear in your time grid. So that could work. (Leo: That's very good.
Just to interject for a second about this, you know with org-roam
right now, we're mostly focused on optimization, but we're hoping to
move on to UX very soon. So all those matters about having TODOs in
your files, it is something that we've been thinking about with Jethro
Kuan, who is my main co-maintainer for org-roam. We'll be working on
this in the coming months, so don't worry too much about it and stay
tuned.)
(00:20:26) Noorah: Yeah. So I've got the ebib and what else... What
subjects... I think... Okay! What is this question? "Have you seen the
project Papis?"
(00:20:37) I'm not sure what... oh it's a Zotero alternative. Okay.
I'll look into it. Thank you. (Leo: I don't know about it either, so
please look into it and let me know.) Have we covered all the
questions?
(00:20:53) (Leo: I believe we have. We have about two-minute leeway
for me to move into the next talk, so we're right on time.) All right.
Thank you so much. Really appreciate it. Good luck everyone! (Leo:
well thank you, and thank you so much for coming, and allowing me not
to be the only one talking about org-roam today.) Noorah: Sounds good.
All right. (Amin: Thank you both very much.)
