[[!meta title="Managing writing project metadata with org-mode"]] [[!meta copyright="Copyright © 2024 Blaine Mooers"]] [[!inline pages="internal(2024/info/project-nav)" raw="yes"]] # Managing writing project metadata with org-mode Blaine Mooers (he/him) - Pronunciation: Blane Moors, [[!inline pages="internal(2024/info/project-before)" raw="yes"]] The planning and writing of a scientific manuscript is an intricate process that requires focused effort. Scientists must make many decisions about what to include and exclude from the paper, often capturing these decisions in notes in the margins, appended notes, or external files. This ad hoc approach becomes unmanageable when the notes exceed the length of the manuscript, which is often the case. Nonetheless, these notes can be vital when responding to reviewers' critiques. Great scientists like Linus Pauling effectively utilized laboratory notebooks to store metadata on his manuscripts. His cross-referencing system resembled that of Niklas Luhmann in his physical zettelkasten. These paper-based approaches have pros and cons, but they are no longer popular because of the hard work required to make them work well. In comparison, the org-roam-ui view of my zettelkasten provides a garden of endlessly forking paths I can wander in all day. I sought a more focused approach to managing my attention and the metadata for one writing project. I developed a project-specific writing log for this purpose about a decade ago. The writing log helps me manage anxieties about forgetting where I left off on an interrupted project (Fear of Forgetting, FoF). In this talk, I will highlight the features of my writing log template in org-mode. The first section supports gathering the initial thoughts about the project needed to assemble a central hypothesis around which to build the paper. Subsections support listing the experiments required to address the central hypothesis and the key discussion points. These subsections include plans for graphical items like images, data plots, tables, equations, and code blocks. Of course, this section will evolve as the results accumulate. When largely completed, this section supports drafting a quarter to a third of a manuscript on day one of the project. The following two sections support project administration and assessment. The administration section includes plans to apply for funding and approvals for the work. The assessment section supports periodic checks of the project's current state, what holds the manuscript from submission today, and what is missing that makes a larger impact. This section includes a timeline and milestones to finish the project promptly. These can be displayed in tables that org-mode so strongly supports. The central section of the template contains daily accounts of accomplishments, decisions, and correspondence about the project. I read this section after a hiatus to resume work on the project quickly. An open-ended to-do list and a section for collecting ideas for future projects follow the daily log. The last section contains protocols and guidelines for the various tasks involved in completing the project. Here, context switching between the writing log and the manuscript is fine because it usually happens only at the beginning and the end of the writing session. My project-specific approach keeps my mind focused on the project at hand and my FoF under control. I share my writing log template in org-mode on GitHub. About the speaker: Blaine Mooers is an associate professor of Biochemistry and Physiology at the University of Oklahoma. He uses X-ray diffraction to study the molecular structure of proteins and RNAs important in disease. He writes grant applications, progress reports, manuscripts, lectures, seminars, and talks each year in Emacs. To control his fear of forgetting (FoF), he uses an external document, the **writing log**, to store metadata about each writing project. He switched from using LaTeX to Org-mode recently. He will discuss the features of the writing log and the joys of editing it in Org-mode. # Discussion ## Questions and answers - Q: what does 0573 means in your init. file name? - A: Index number used to \"name\"/ describe a project.  My project folders in my home directory are of the format ####stubOfTheProjectsFullName. Maybe i will describe my project numbering system in a future talk. It helps enormously in navigating between projects and stayting organized. I store the project ID number and the folder name in a sqlite database that I pop open with a bash alias to the DB Browser for SQlite. I also use the Project ID to track time spent on each project to the nearest 15 minues increment. I do this time tracking once every several hours 2-4 times a day. If I cannot remember what I was doing, nothing is recorded. This is important for keeping the data accurate. This practive is a pain but the data are valuable for self-assessments and annual reports. - Q: \ What does Zettelkasten mean? - A: \ It\'s a notetaking method \Useful overview at [https://zettelkasten.de/introduction/](https://zettelkasten.de/introduction/){rel="noreferrer noopener"} - A: Pls cf. [https://en.wikipedia.org/wiki/Zettelkasten](https://en.wikipedia.org/wiki/Zettelkasten){rel="noreferrer noopener"} An augmented org-roam is found in the zetteldesk.el project by Vidianos [https://github.com/Vidianos-Giannitsis/zetteldesk.el](https://github.com/Vidianos-Giannitsis/zetteldesk.el){rel="noreferrer noopener"}. See  his talk from EmacsConf 2022:  [https://www.youtube.com/watch?v=dJ859kiFRLg&t=0s](https://www.youtube.com/watch?v=dJ859kiFRLg&t=0s){rel="noreferrer noopener"} - A: A great resource is Sönke Ahrens\'s \"How to Take Smart Notes\" (to get you thinking about the progressive evolution of your note taking; and a solid walk through of Zettlekasten in practice/implementation) . There also is a German edition of the book, it\'s called: \"Das Zettelkasten-Prinzip\". The book \"Antinet Zettelkasten\" by Scott Scheper clarifies several misconceptions about the Zettelkasten Method (see chapter 3). He spent months replicating the analog version that Luhman used and gained a number of important insights. I have seen the method applied in ways that were never intended. I think that these vairant approaches are innovative and can enhance productivity, but they can cause some confusion. The ultimate source is the [https://niklas-luhmann-archiv.de/](https://niklas-luhmann-archiv.de/){rel="noreferrer noopener"}. - Smart notes are linked to one another. They make up a kind of hypertext, similar to a wiki, as a whole. - Org-Roam and Denote are two implementations that can support the slipcase; backlinks are important (and also/perhaps a consideration of the hierarchy) - Q: How many papers are you writing at the same time? - A: Working on 10--15/y, publishing 1--8/y - On a given day, 2 papers/day usually, sometimes up to 5 papers/day - Q: How you capture those ideas when when you are away from Emacs?+3 - A: Digital recorder for recording thoughts, especially during daily commute. I also mentioned caputing thoughts in 750words.com. I was an early user (May 2012) and was grandfathered in with a free subscription for life. This is why I am reluctant to give it up. I started learning to use Emacs in 2000. I have not yet moved all of my workflow to Emacs, but I am getting there. The other site that I use when I exceed the 5000 word limit at 750words.com is WriteHoney [https://app.writehoney.com/write](https://app.writehoney.com/write){rel="noreferrer noopener"}. - Whisper AI to transcribe the recordings. See this site for the post transcription scripts that I apply to clean up the text before I see the trasncript: [https://github.com/MooersLab/bash-whisper-transcription](https://github.com/MooersLab/bash-whisper-transcription){rel="noreferrer noopener"}. I also jot down notes but not so much after I started using the digitial voice recorder (DVR). Dragging the audo file to the folder where I do the transcriptions does take a wee bit of time so I try to be judicious about selecting when I turn on the DVR. I should probably write a Elisp function to transcribe the audio recording from the DVR, save the transcript, and delete the audio recording so that I can streamline this process. There is alawys a way to do things more efficiently with Emacs! - sachac: oooh, I can recommend getting an inexpensive lapel mic if you don't already have one. I'm also experimenting with using keywords to organize my audio braindumps. - Q: What if an ideas does not belong to any current working manuscript? - A: Sandbox area in a log file. The Sandbox area is called \"Future additions and tangents\" in the template log file and is also used to draft sections of the paper that I am uncertain that I will retain. - If not related, kept in an external diary - Q: It looks like you have daily log per project, what do you think about just using single daily notes and cross referencing different projects? - A: I have \> 800 projects per year in 10 categories. Several years ago, I made org files for each category with 1000 headlines, but this proved to be too unwieldy for me at the time as a noivice org-mode user. The log files for the manuscript projects wind up with 50-100 pages of content each so the single-threaded nature of Emacs hampers scrolling through such huge org files. In addition, I want to keep all other projects out of sight and mind while focused on the project at hand. - Q: \ If there were one habit from your process (referencing your extensive flow chart) that you want active learners/professional researchers to adopt, which would it be and why? - Keeping a daily diary is essential for overcoming the fears of forgetting and losing momentum. These fears are the maiin barriers to switching between projects on a given day. The in-grained preference is to do binge writing, which I still succumb to doing sometimes, but I am trying to avoid binge writing with this system. - Q: \ I see that you use checkboxes for tasks. Did you know you can format headings to support tracking them? Like this: TODO Heading \[0/4\] - A: Yes, I convert some of my most important todo items into \*\*\* TODO headline to be picked up by arg agenda. I add the writing-project log file to the list of org files that org agenda searches. I use a Elisp function to add new \*\*\*TODO items to the list when they come to mind. I will post this function shortly on the AAAreadme of [https://github.com/MooersLab/writingLogTemplateInOrg](https://github.com/MooersLab/writingLogTemplateInOrg){rel="noreferrer noopener"}. This eliminates the need to refile the idea, which I am too undisciplined to do. - Q: I actually love this one! But it creates as much separated log as there is project. You limit yourself to work only on how many project per day? How do you explore them back (if you ever)?  I assume your org files is growing exponantially over time. You said about \"valuable when receiving critiques?\" (Donovan)   - A: Thank you for the opportunity to make clarifications. I try to work on two writing projects a day on average terms of doing generative writing tasks. Yes, there is one writing-project log file per project. They tend to grow to a length of 50-100 pages by the time the paper is published. Because of the ability to fold sections, I have not run into trouble with the longer lengths.  I use the template on GitHub to start the log file for a new writing probject. I could consider retaining a log file for a sequel paper to retain the history of decisions made in the first paper but I would duplicate it and rename the copy with the new project number. I will go through an old log file for a published paper from years ago to recover details on how I carried out certain procedures. The history of decisions in the daily log is invaluable for responding to probing questions by reviewers about why we decided to do certain procedures. - \<\_viz\> Q: Sorry, not really related to the talk\'s topic but can you share the pymol project that shown in the writing log slide earlier in the talk? I am interested in the paper/project. - A: Thank you for your interest on the PyMOL project. I have not published that particular project yet, but a number of related PyMOL projects are on my GitHub site. The most popular is MooersLab/PyMOLshortcuts [https://github.com/MooersLab/pymolshortcuts](https://github.com/MooersLab/pymolshortcuts){rel="noreferrer noopener"}. The associated paper is here [https://onlinelibrary.wiley.com/doi/10.1002/pro.3781](https://onlinelibrary.wiley.com/doi/10.1002/pro.3781){rel="noreferrer noopener"}. I have run PyMOL in Jupyter [https://github.com/MooersLab/jupyterlabpymolpysnips](https://github.com/MooersLab/jupyterlabpymolpysnips){rel="noreferrer noopener"}  [https://pubmed.ncbi.nlm.nih.gov/33967632/](https://pubmed.ncbi.nlm.nih.gov/33967632/){rel="noreferrer noopener"} ,  Colab   [https://github.com/MooersLab/colabOpenSourcePyMOLpySnips](https://github.com/MooersLab/colabOpenSourcePyMOLpySnips){rel="noreferrer noopener"}  [https://proceedings.scipy.org/articles/majora-1b6fd038-002](https://proceedings.scipy.org/articles/majora-1b6fd038-002){rel="noreferrer noopener"}, RStudio  [https://github.com/MooersLab/rstudiopymolpysnips](https://github.com/MooersLab/rstudiopymolpysnips){rel="noreferrer noopener"}, and Org-mode [https://github.com/MooersLab/orgpymolpysnips](https://github.com/MooersLab/orgpymolpysnips){rel="noreferrer noopener"}. I gave a talk about the later at emacsconf 2021 [https://emacsconf.org/2021/talks/molecular/](https://emacsconf.org/2021/talks/molecular/){rel="noreferrer noopener"}.  - \ Q: Have you tweaked your Emacs configuration at all \-- with cus keybindings, etc \-- to support the process you demonstrated? Thank for the good question because my talk was focused on a template org document. I use org-ref heavily. I use it to add Bibtex entries, download PDFs of research articles by their DOIs, and rename them with the citekey. I made several Elisp functions that do minor tasks to support my worflow. I will post them github.  - \ I like to Seperate my ZettelKasten and GTD  with Daily and global, Zettelkasten daily would be a journal, The reason to keep them seperated is daily is to start with a black sheet and can be thrown ignored if not done well, If good enough goes into zettel etc. That is a good suggestion. Thank you for sharing it. ## Notes - Hi, This is Blaine. I am checkin in! - Hey :) - Hello from Madagascar :) Hello from Brazil (there\'s an old song about madagascar: [https://www.youtube.com/watch?v=k206JIkXJbk](https://www.youtube.com/watch?v=k206JIkXJbk){rel="noreferrer noopener"}) :) Hello from Uganda Hello from ISS\^\^ The International Space Station? 0_0 Hello from US Hello world and outer space!!!! - It\'s so awesome seeing these talks that are not strictly software development relatedThank you! - It\'s amazing to see how much people do with emacs Thank you! - \ \[MooersLab/writingLogTemplateInOrg: Template for writing.log in org-mode\]([https://github.com/MooersLab/writingLogTemplateInOrg](https://github.com/MooersLab/writingLogTemplateInOrg){rel="noreferrer noopener"}) - \ it\'s really impressive. it also makes me realise that people are a lot more organised than could ever be. I have been studying time management for decades; it is still a stuggle for me. - \ That chart looks complicated. It can get pretty complicated as its deeply personal and has seen several revisions.  (Sounds like the chart is descriptive and not prescriptive) Right, is just descriptive. Yes, that concept map it a bit overwhelming. I plann to make a slimed down version. Thank you for the feedback. - \ I love the fact. that \"shower\" is called out as an Ideation \"feed\" in the writing process chart :-) Takes shower thoughts to a whole new level\...It is a feature of our insane 24-7 culture.  - \ The man is trying to treat cancer, I expected nothing less complicated than this gg Thank you! Yes, our work has added years of additional life to many cancer patients.  - \ There is a package \-- I think it is \'org-tracktable \-- that records word counts by session in an inline block within org files  [https://github.com/tty-tourist/org-tracktable](https://github.com/tty-tourist/org-tracktable){rel="noreferrer noopener"} i will check it out. Thank you for the link! My time tracking databse has 10,848 rows after 2.5 years. It may be over 100,000 rows long by the time that I expire. I decided that a org-mode table would be not appropriate. I adandoned the Google Sheet Workbook over a year ago because it is reduant with entries in my timespent.db. I did not like the duplicated effort.  I am now testing the use of a heat map for a similar writing accountability purpose. It tallies my writing effort per day from my timespent.db file. I enter \`hmj\` for Heat Map of Journal article writing to effort.  This is a long bash alias to to script and commands that generate the heatmap and post it on Github. The GitHub repo for the script is here [https://github.com/MooersLab/writing-time-spent-heatmap](https://github.com/MooersLab/writing-time-spent-heatmap){rel="noreferrer noopener"}. I have similar aliases for makng heatmps for grant writing, teaching, and service, but my daily focus is on article writing. - \<\_viz\> oohhh using a drawer for stashing these details away is a nice idea! Thank you! Org-mode has so many niffty feature like drawers. - \ I\'m impressed by how systematic he is about everything.Thank you! - \ I\'m thankful for people like the presenter that dedicate so much of their time to researching treatments for the horrible diseases that people suffer. \[10:09\] Thank you! [[!inline pages="internal(2024/info/project-after)" raw="yes"]] [[!inline pages="internal(2024/info/project-nav)" raw="yes"]]