1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
|
[[!meta title="Literate Programming for the 21st Century"]]
[[!meta copyright="Copyright © 2024 Howard Abrams"]]
[[!inline pages="internal(2024/info/literate-nav)" raw="yes"]]
<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->
# Literate Programming for the 21st Century
Howard Abrams (he/him) - @howard@emacs.ch , <https://www.howardism.org>
[[!inline pages="internal(2024/info/literate-before)" raw="yes"]]
Donald Knuth’s idea of *literate programming* in the 80’s and 90’s was
interesting, but he didn’t realize what Emacs and Org can do in this
century. In this talk, I would like to go back your initial *dabblings*
with Org `src` blocks to show how you can program *literately* as quickly
as you can in any other mode.
Some of the tips and tricks include:
- Automatically keeping your lit code sync’d
- Easier code generation
- Jumping to Org headers to help organize code
- Jumping to code definitions with the xref interface
At the end of this talk, I hope to inspire you to try it again, as my
personal “go to” is *programming literately*.
I will be following the format and outline in my essay:
<https://howardism.org/Technical/Emacs/literate-writ-large.html>
About the speaker:
About ten years ago, I gave a talk I called *literate devops* and people
still ask me if I still use those techniques. For all my personal
projects, I do. Even projects that I share with others, I often *start*
programming with an Org file.
I will admit, programming within Org blocks has some burrs, but over
the years, I’ve filed them off with helper functions, snippets and
other features. Thought I would share these.
# Discussion
## Questions and answers
- Q: Apropos large literate programs: what's the largest code base
you've ever tackled with the literate approach (esp. Emacs +
Org-mode)?
- A: The largest is the one I mentioned in the talk \... about
8000 lines of "code" and another "10000" lines of prose. I
think I came to 15,000 max (in code blocks only).
- Q: You touched on it briefly, but how do you handle things like
"C-h f" helpful info not being tied back to the defuns in src
block code when you "C-c C-c" them in the org buffers instead of
re-tangling it to the files, and other such things? Did you create
wrappers for jumping back and forth atop org's built-in mechanisms
to go back and forth between org/tangled files?
- A:
- Q: Have you ever used org-transclusion
([https://github.com/nobiot/org-transclusion](https://github.com/nobiot/org-transclusion){rel="noreferrer noopener"})?
- A: Nope \... but I will
- I tried it out once, and had one hour of work deleted 🥲,
but it was from an issue already reported:
[https://github.com/nobiot/org-transclusion/issues/177](https://github.com/nobiot/org-transclusion/issues/177){rel="noreferrer noopener"}
and
[https://github.com/nobiot/org-transclusion/issues/257](https://github.com/nobiot/org-transclusion/issues/257){rel="noreferrer noopener"}
- Q: What is your usage of dynamic blocks in such workflows? Any
interesting use cases and custom ones?
- A: Not yet, will report back
- Q:Is the minibuffer being deliberately hidden in this video? (first
noticed this in the section previous to "Navigating by Function
Names")
- A: Not intentionally :) You may notice the minibuffer comes and
goes, sorry about that; not intentional (didn't quite "fix"
all of them) (Thanks for the answer, no worries.)
- Q:What's your take on Emacs+Org vs. Jupyter notebooks (for
interactive programming)?
- A: Not something I use right now. Tend to include things from
jupyter/python (e.g. numpy) that has been the biggest challenge
(not knowing that stuff all that well), things like matrix
multiplications are easy in jupyter not such much in org. May
make sense to stay where you are comfortable. Curious what the
community can do to make this transition easier
- You can't work with Jupyter in \> 1 language either (I think).
It's Py + SQL or R + SQL etc. Org allows 45+ languages in one
document (I often mix languages).
- Q: Do you think any programming language is more suited to literate
programming than another?
- A: R, C are my favorites (for literate programs). C (and C++)
have got great support. There are some great books implemented
in literate programming I think. The two that come to mind are
Physically Based Rendering: From Theory to Implementation and C
Interfaces and Implementations. The first is C++ and the
second C. Ty.
- A: \<gs-101\> Personally, when working with Org files, I have a
better time dealing with interpreted languages, such as Python,
because you can initialize a session and the code is all
conected. You can divide blocks however you seem fit.
- Q: related to above, do you use inline org function calls and org
babel library and such?
- A: usta-use more org-babel inline functions, found sound bugs
(maybe) 8yrs ago, right now my literate dev-ops is calling a lot
of backend programs so the org-babel has limited help in that
regard while in the emacs session things are "just available"
so that hasn't help much either
- Q: How do you handle the cases where org markup may sometimes
interfere with some of the code, in places where you can't use
"escapes" (\~ or = or \| \<- vertical bar), doubly so if you use
modes to not show these but the styled text instead, and so on?
- I think an example is in C when you assign to a pointer \*p = &i;
(In Org, you need to write (\*p) or ,\*p = &i; or it will be
mistaken for a headline \*
- A: (clarifying) when I'm making pros and I'm talking about a
function I've written somewhere else I'll use tildes and look
for those things so I can strip them off. Is that the spirit of
the question. (confirmed). Yes, I'll strip that off after
finding the function name, so I can still mark it correctly.
- Q: clarifying: when in code inside an org buffer, you don't get
to use \~ or = (verbatim/etc), and any font-locking interferes
with the proper display in the src blocks, that kind of
interference.
- Q: You said at the start that literate didn't catch on in corporate
DevOps - why not?
- A: I guess the big thing is not everyone is using Emacs and org
is needed to make it work really well.
- Q: I gotta ask: why not that full stack on Markdown, I'm sure it's
crossed your mind at least a few times how the same setup on
Markdown would be more interop-friendly with colleagues and such?
- A: It's a real good idea.
- Q: How does your management of "TODOs" (projects/tasks) interact
with this literate mindset, any insightful things you do on that
front?
- A:
- Q: \<Donovan\> Do you LP also on larger projects? (More files &
nested directories)
- A: I haven't done nested directories, but I can now. Now that
i've realized I have the feature where I can just jump to any
projects and all the org-files and all the headings just show
up, that works in nested directories, that that's fun.
- Q: Have you used Cucumber/Gherkin/BDD and do you think it has a
strong overlap to what you talked about here?
- A: I tend to put the tests right next to the function, I like
tangle it out to different files; keeping things together is
nice. Many frameworks assume we'll have things seperated out
in a way that isn't useful to me. I like to go old-school on
that?
- Q: What granularity are you looking for re your org files and
contents, with respect to a codebase that it tangles to, or in
non-coding contexts?
- A: Great questions, really subjective. I change that all
time. I have an idea, I start to refine it. My goal at one
point was to have an emacs config that was really small and
simple and that just really doesn't happen, it's full of ideas
and things that are half-baked and i pull them out and polish
them up bit by bit so it ends up being like any code-base it
just keeps getting refined. Sub-trees, archiving are useful.
- I've found it useful to prune the init file back to minimal
every once in a while (actually, AI has been surprisingly
helpful - perhaps it helps that Emacs is ancient and hence there
is a lot of doc out there and much of it \... correct?)
## Notes
- My literate programming code extensions:
[https://www.howardabrams.com/git/howard/hamacs/src/branch/main/ha-org-literate.org](https://www.howardabrams.com/git/howard/hamacs/src/branch/main/ha-org-literate.org){rel="noreferrer noopener"}
- My Emacs configuration written in a literate style:
[https://github.com/howardabrams/hamacs](https://github.com/howardabrams/hamacs){rel="noreferrer noopener"}
- See it rendered here:
[https://howardabrams.com/hamacs/](https://howardabrams.com/hamacs/){rel="noreferrer noopener"}
- My JOPS (Jump to Project Sections) code that "searches Org
headers":
[https://www.howardabrams.com/git/howard/jops](https://www.howardabrams.com/git/howard/jops){rel="noreferrer noopener"}
\... temporary location?
- snippet on \<ssl: Cool one.
- \<gs-101\> 0_0 I need to do this.
- Your way of delivering is inspiring.
- \<gs-101\> This (evaluating babel blocks) is also possible with the
Avy + Embark combo developed by karthik:
([https://karthinks.com/software/avy-can-do-anything/#avy-plus-embark-any-action-anywhere](https://karthinks.com/software/avy-can-do-anything/#avy-plus-embark-any-action-anywhere){rel="noreferrer noopener"}),
just jump to a block and then "RET".
- \<NullNix\> god I wish I was that good a presenter
- \<ericsfraga\> Really good talk. I need to find out how to extend
xref to handle org files!
- \<lounge-081\> ericsfraga: same here, I asked a long-winded
question that was about that (before he touched a bit on it),
but feel there's more in terms of wrappers and such
- \<aschmidt\> excellent presentation indeed
- \<lounge-965\> Thank you for the marvelous talk!!
- \<lounge-350\> A legend! \... loved the Ironsworn presentation from
previous year.
- \<ElfOfPi\> Denote has some pretty good use of dynamic blocks I
think
([https://protesilaos.com/emacs/denote#h:8b542c50-dcc9-4bca-8037-a36599b22779](https://protesilaos.com/emacs/denote#h:8b542c50-dcc9-4bca-8037-a36599b22779){rel="noreferrer noopener"})
- There's also the dynamic blocks from org-nursery:
[https://github.com/chrisbarrett/nursery?tab=readme-ov-file#org-roam-dblocks-incubating](https://github.com/chrisbarrett/nursery?tab=readme-ov-file#org-roam-dblocks-incubating){rel="noreferrer noopener"}
- \<ellis\> there is an RFC in for 'cargo-script' which allows
building single-file crates - i think that will be quite useful in
ob-rust
- \<gs-101\> ellis: There's a ob-rust already and it uses
rust-script: [https://github.com/micanzhang/ob-rust](https://github.com/micanzhang/ob-rust){rel="noreferrer noopener"},
but the developer wanted to use rustc instead.
- cargo-script RFC issue: [https://github.com/rust-lang/cargo/issues/12207](https://github.com/rust-lang/cargo/issues/12207){rel="noreferrer noopener"}
- Not a Q, just a comment that we need more of your insightful
posts and videos! :) (sic)
[[!inline pages="internal(2024/info/literate-after)" raw="yes"]]
[[!inline pages="internal(2024/info/literate-nav)" raw="yes"]]
|
