Hi, I'm Sacha Chua. This presentation is a quick tour
of some of the things we do to run EmacsConf.
Since 2019, we've run it as an entirely online conference,
and we do as much of the organization as possible
within Emacs itself.
I have three reasons for making this presentation.
The first is entirely selfish: I need to figure out
all the stuff I built for last year's EmacsConf,
since it was a bit of a crazy scramble.
The second is that I want to show people
the process of thinking about a complex project,
looking for little things to automate in Emacs,
and building things up from small pieces.
Maybe you'll get some ideas
and start building tools for yourself, too.
The third is that you find any of these little tools interesting,
I want to point you to blog posts and source code
where you can find out more.
That way, you don't need to try
to read and understand everything quickly.
You can find this presentation and other links
on the talk page at emacsconf.org/2023/talks/emacsconf.
There are a lot of different parts,
so I'll try to use this map to help make sense of it all.
There's so much information to work with,
so it probably doesn't surprise you that we use Org Mode a lot.
Most of the conference coordination happens over e-mail,
which I can quickly search with notmuch.
Some of the information is private,
like emergency contact numbers.
We store the talk information in a private Org file.
I try to put as much as possible
into our public organizers' notebook
so that processes and decisions are documented.
We need a public website.
We use Ikiwiki to make the webpages
because we can work with plain text files
in a Git repository.
We also make a few static HTML pages
for things where Ikiwiki is a little awkward.
We post announcements to mailing lists.
We also receive submissions in a private mailing list
so that a number of people can review them.
We have a backstage area
for sharing files with volunteers and speakers.
We share those files publicly when the talk goes live.
And there's all the other stuff that goes into running EmacsConf,
like shell scripts and configuration files.
First, speakers propose a talk by sending an e-mail.
We take the info from that e-mail and store it in Org properties
so that we can work with it later.
Every talk is identified with an ID,
but since `:ID:` and `:CUSTOM_ID:` have special meanings for Org,
I use `:SLUG:` as the keyword.
Speakers' names go into the `:NAME:` property,
and a short version goes into `:NAME_SHORT:`
so that we can include that in a greeting.
If people follow the template closely...
...we can even automatically fill in the Org subtree for their talk.
We can use regular expressions to recognize the text
and extract the properties.
Other properties need to be set by hand.
I often mess things up when I retype them.
To avoid typos, I have a function that sets a property
based on the current region. I bind that to `C-c C-x p`.
That makes it much easier to set properties
that couldn't automatically be recognized.
Sometimes it makes sense to dynamically generate a property
and then edit it, like with filenames.
We like to name all the talk files the same way,
but sometimes special characters in talk titles or speaker names
need a little tweaking. I'll put that in a `:FILE_PREFIX:` property
so I can edit it.
An Org property match can map over all the talk entries
that don't have `:FILE_PREFIX:` defined.
We can use that `:FILE_PREFIX:` to rename files from Emacs.
With that property, we can then rename files using that prefix,
some extra text, and the file extension.
Sometimes it's easier to work with the data outside Emacs,
like when I want to rename files with a shell script.
If I export a subset of the data as JSON
or JavaScript Object Notation, using `json-encode`...
... then I can extract the data with `jq`
and use it in shell scripts.
Another example of semi-structured information
is speaker availability.
We have speakers from all over the world,
so we try to schedule live Q&A sessions when they're around.
That means working with timezones.
Completion makes it much easier to set the timezone property
without worrying about typos.
We can take advantage of the timezone list from the tzc package,
which works with Unix timezone definitions.
Then we can convert times using Emacs.
Using a standard format to encode the availability
makes it easier to parse.
I can use those availability constraints to report errors
when I'm experimenting with the schedule.
Now that I have the availability information,
I can think about scheduling.
When we were planning EmacsConf 2022, the schedule was so full,
I wanted to see if we could make it more manageable
by splitting it up into two tracks.
It was hard to think about times with just a table.
I was able to turn the schedule information
into an SVG to convince the other organizers
to get on board with this crazy plan.
And the nice thing about SVGs is that
they can even be clickable on the wiki.
Being able to quickly make SVGs of different schedules
also helped me test scheduling ideas and think out loud.
I could change the time between talks, the order of the talks,
and even what tracks the talks were in.
This was helpful when I needed to include
some late submissions or availability changes
and I wanted to ask speakers what they thought.
They could see the different schedule options themselves.
It's really nice to have Emacs Lisp support for working with SVGs.
I also love how I can have an Emacs Lisp block
in an Org Mode document that updates an SVG
that I can view right there in my text editor.
Setting the timezone lets me automatically translate times
to the speaker's local timezone when I e-mail them.
That's mostly a matter of using `format-time-string` with a timezone.
There's a lot of text to work with,
which means templates are super handy.
There are a number of templating functions for Emacs Lisp,
like the built-in `tempo.el` or `s-lex-format` from `s.el`.
I ended up writing something
that works with property lists (plists) instead,
since we use plists all over the emacsconf-el library.
All it does is replace `${variable}`
with the value from a property list.
I use this mostly because I have a hard time
keeping track of which `%s` is which when I use `format`,
and it's hard to get an overall view if I just use `concat`.
The code looks for the properties and replaces them with the values.
I just find it a little easier to think about sometimes.
Getting all the information is just a matter of going over
all the talk entries using `org-map-entries`.
This builds the talk info by running a bunch of functions.
Some functions get the information from the Org file.
Other functions use the info already collected.
This can take a while to do again and again.
It's useful to `memoize` this function
when I know I'll be using it a lot,
like when I export the organizers notebook.
Memoize caches recent values.
We combine this templating function
with the talk information
to fill in the conference wiki,
since that's a matter of writing templated strings to files.
The talk pages are generated once
and then left alone for manual editing,
while the navigation is regenerated
every time we change the details.
Here are some examples
of how we fill in the conference wiki.
We put in the format of the talk, how Q&A works,
and what the status is.
Once the talk is live, we include the video
and the links to the files, too.
The code is a little bit long,
but the important part is that
we fill in a plist with the values we calculate,
and then we can use `emacsconf-replace-plist-in-string`
to put that all together.
The schedule is a little more complicated.
I wrote an Ikiwiki directive
so that the markup is more manageable,
and the Emacs Lisp function uses that.
The Ikiwiki directive takes all the data and turns it into HTML...
...so we can use Emacs Lisp to iterate over
a slightly smaller property list
and put them into the format Ikiwiki expects.
It's nice to be able to navigate between talks
without going back to the schedule page each time.
This is handled by keeping two extra copies of the list:
one with the first talk popped off,
and one with an extra element added to the beginning.
Then we can use the heads of those lists
for next/previous links.
Links to the next talks are also handy
on the collaborative Etherpad documents
that we use for collecting questions, answers, and notes
during each talk.
Etherpad has an API...
...so I can start the pads off with a template
before the conference.
I don't want to accidentally overwrite a pad
that has been manually edited.
We can save the timestamp of the last modification
and then compare it before overwriting.
Templates are also very handy when it comes to e-mail.
Sometimes we send e-mails one at a time,
like when we let a speaker know
that we've received their proposal.
That's mostly a matter of plugging the talk's properties
into the right places in the template.
Sometimes we send e-mails to lots of speakers at the same time,
like when we send them instructions for uploading their files.
Instead of sending one e-mail and Bcc-ing everyone,
or sending people multiple e-mails
because they have multiple talks,
I like to draft these as individual e-mails
to each speaker (or group of speakers,
if more than one person is associated with a talk).
That gives me an opportunity to personalize it further.
Many speakers answer questions live
in BigBlueButton web conference rooms.
Setting up one room per group of speakers
makes it easy to give the speakers the details
and associate the recorded video with the talk afterwards.
For EmacsConf 2023,
I used Spookfox to control Mozilla Firefox from Emacs
so that I could automate creating the rooms
and adding the URLs to the talk properties in my Org file.
Then I can use mail merge to send each speaker
the check-in instructions for their specific room.
Some speakers will take questions by e-mail
after the conference instead of attending live,
so we send them shorter instructions
just in case they want to drop by.
[Live Q&A sessions]: After the first rush of questions,
we can open it up for other people to join.
This is handled by changing the public page
from one that just refreshes in a loop
to one that redirects to the actual web conference room.
Just in case, we also
generate static copies of those redirects
so that we can copy them if needed.
That way, I don't have to count on Emacs being able to
publish them over TRAMP.
During the conference, I'm often jumping from talk to talk.
Instead of going to the Org file
and then searching for the talk,
I've made a little Hydra with keyboard shortcuts.
One of these shortcuts lets me
jump to a talk with completion
so that I can just type in part of the talk ID,
title, or speaker name.
I've also defined some Embark actions
so that I can act on a talk right from the completion menu.
For example, I might want to jump to the wiki page
or e-mail the speaker.
I can also add notes to a talk while looking at an email,
like when a speaker lets me know
that their video will be late.
Making it easy to add a note turns Emacs into
a very basic contact relationship management system, or CRM.
The way this works is that we have a function
that lists all the email addresses associated with a talk.
We can then map that over the list of talks,
look up the author of the current email,
prompt the user for the talk to add the note to, and add the note.
On to captions.
We've been doing captions for the last couple of years,
and now we have a small army of volunteer captioners.
They get early access to the recorded talks
and fix up misrecognized words, format keyboard shortcuts
to follow Emacs conventions, spell names correctly,
and do all sorts of other wonderful things.
One of our evil plans with EmacsConf
is to get cool stuff out of people's heads into videos
and also make captions so that those videos can be searched.
To make that possible, we first need a backstage area
where volunteers can get the files.
This is just a simple password-protected directory
with a static HTML page that lists the talks by status
and shows the files related to each talk.
As a talk moves through the process, I update its TODO state
and republish this index.
Talks that are ready to be captioned show up in that section,
and volunteers can call dibs on the talk they're interested in.
That's all done with a function that formats the information
and uses TRAMP to save the file directly to the server.
You can find more details on our captioning process
at emacsconf.org/captioning.
I like using subed to edit subtitles within Emacs.
Let's talk about actually playing the talks.
For EmacsConf 2022, we tried using Emacs timers
to run the talks.
It turns out that you can't call TRAMP from a timer
when you're already using TRAMP from another timer
at the same time.
I thought about just tweaking the schedule
so that we always start things at different times,
but I figured there's probably a more elegant way to do this.
This year, I'm planning to experiment with using cron
to start talks on autopilot.
The shell scripts will take care of playing the videos...
... figuring out the appropriate Q&A...
... and joining the web conference if needed.
We just need to format the information...
...and install it as the track's crontab.
It's useful to be able to switch tracks
to manual mode independently,
just in case things go haywire.
Then we can start everything manually.
I can also manually update a talk's status,
like when the host tells me that it's okay to open up the Q&A.
The shell scripts we run from the crontab
can also update the talk status themselves.
Then a bunch of things automatically happen based on
the talk status changes.
This uses `org-after-todo-state-change-hook`.
We get the talk information
and pass it to a list of functions.
Internet Relay Chat or IRC is an easy way for people
to join the conversation around EmacsConf.
We announce a talk whenever it changes state.
For example, when a talk starts,
we post the URLs to the talk webpage
and the Etherpad for questions. We change the topic as well,
so anyone can see the current talk's information
even if they're a little late.
This is easy to do with a little bit of Emacs Lisp
because (of course!) Emacs has an IRC client.
In fact, it has several.
It seems like a lot of automation and Emacs Lisp,
but really, all of this was just built up little by little.
And tinkering with this is *fun*, you know?
It's like always being able to ask,
"Hey, wouldn't it be cool if..."
and then actually being able to go and do it.
Sometimes it feels like EmacsConf is an excuse
for me to play with Emacs.
It's pretty amazing what you can do
by combining a bunch of pieces.
A way to store slightly-structured information.
A way to get it out again. Templates.
TRAMP, for working with remote files
and running remote commands.
A way to talk to a web browser.
A way to work with SVGs.
An email client. A chat client.
You can smoosh them all together
in a way that you couldn't if they were all separate things.
The code is in the emacsconf-el repository.
It's a bit of a tangle because it's accumulating organically
and I haven't really had the brainspace
to step back and clean it up.
But if you spotted anything interesting in this presentation,
you can go check it out and see what you can scavenge.
The link and this presentation are available
from this talk's webpage at emacsconf.org/2023/talks/emacsconf .
Let's figure out how to make Emacsconf even awesomer next year!