add upload to organizers notebookHEADmaster

2 files changed, 499 insertions, 51 deletions

diff --git a/organizers-notebook.md b/organizers-notebook.md
index ef153a5f..2ff20b0d 100644
--- a/organizers-notebook.md
+++ b/organizers-notebook.md
@@ -20,43 +20,48 @@ This document is the general organizers' notebook that can be used as a starting
     -   [Create the public organizers’ notebook in the wiki](#starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki)
 -   [Phases and lessons learned](#phases-and-lessons-learned)
     -   [Draft CFP](#phases-and-lessons-learned-draft-cfp)
-    -   [Announce](#org897f0c8)
-    -   [Process proposals](#orgd12f3b1)
-    -   [Accept proposals](#orgcbfca65)
+    -   [Announce](#phases-and-lessons-learned-announce)
+    -   [Process proposals](#phases-and-lessons-learned-process-proposals)
+    -   [Accept proposals](#phases-and-lessons-learned-accept-proposals)
     -   [Draft schedule](#draft-schedule)
-    -   [Set up backstage and upload](#orgdbb77f2)
-    -   [Post the schedule](#org9456d25)
-    -   [Process uploaded files](#org75e87bb)
-    -   [Generate assets](#org125bd75)
-    -   [Prepare the infrastructure](#org9759aa9)
-    -   [Getting ready for captions](#org9f978d1)
+    -   [Set up backstage and upload](#phases-and-lessons-learned-set-up-backstage-and-upload)
+    -   [Post the schedule](#phases-and-lessons-learned-post-the-schedule)
+    -   [Process uploaded files](#phases-and-lessons-learned-process-uploaded-files)
+    -   [Generate assets](#phases-and-lessons-learned-generate-assets)
+    -   [Prepare the infrastructure](#phases-and-lessons-learned-prepare-the-infrastructure)
+    -   [Getting ready for captions](#phases-and-lessons-learned-getting-ready-for-captions)
     -   [Getting ready for the conference](#pre-conf)
     -   [During the conference](#conf)
--   [General infrastructure](#org85c6f49)
+    -   [After the conference](#phases-and-lessons-learned-after-the-conference)
+-   [General infrastructure](#general-infrastructure)
+    -   [Linode instance sizes](#general-infrastructure-linode-instance-sizes)
+    -   [File uploads](#upload)
     -   [Icecast](#icecast)
     -   [Watching pages](#watch)
     -   [VNC](#vnc)
     -   [Streaming with OBS](#obs)
-    -   [Shell scripts](#orgab7108a)
+    -   [Shell scripts](#general-infrastructure-shell-scripts)
     -   [Backstage](#backstage)
-    -   [IRC web client](#org5545d42)
-    -   [Etherpad](#orgd7fd9ee)
+    -   [IRC web client](#general-infrastructure-irc-web-client)
+    -   [Etherpad](#general-infrastructure-etherpad)
     -   [Publishing media to the server and to the wiki](#media)
-    -   [Automated IRC announcements](#org75cad9b)
+    -   [Automated IRC announcements](#general-infrastructure-automated-irc-announcements)
     -   [BigBlueButton](#bbb)
-    -   [Manual IRC announcements](#org5a3d818)
-    -   [Low-res stream](#orge08d62d)
-    -   [Upload talks to YouTube](#org7f2e95e)
-    -   [Restream to YouTube](#org0cc19c3)
+    -   [Manual IRC announcements](#general-infrastructure-manual-irc-announcements)
+    -   [Low-res stream](#general-infrastructure-low-res-stream)
+    -   [Upload talks to YouTube](#general-infrastructure-upload-talks-to-youtube)
+    -   [Restream to YouTube](#general-infrastructure-restream-to-youtube)
 -   [Other process notes](#other)
     -   [Adding another user to BBB](#bbb-user)
-    -   [When a talk is added after the schedule has already been drafted](#org4efd774)
+    -   [When a talk is added after the schedule has already been drafted](#other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted)
     -   [When a talk is cancelled](#cancel-talk)
+    -   [When a talk changes title](#other-process-notes-when-a-talk-changes-title)
     -   [When a talk Q&A changes method](#change-talk-qa)
-    -   [When a system is down](#org17cc58c)
-    -   [When we need to bring on a last-minute volunteer](#org4636e4e)
-    -   [Checking people in](#org811d67a)
-    -   [Ansible](#org5d78595)
+    -   [To play the other stream](#other-process-notes-to-play-the-other-stream)
+    -   [When a system is down](#other-process-notes-when-a-system-is-down)
+    -   [When we need to bring on a last-minute volunteer](#other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer)
+    -   [Checking people in](#other-process-notes-checking-people-in)
+    -   [Ansible](#other-process-notes-ansible)
 
 
 <a id="starting-up-the-conference-planning-process"></a>
@@ -157,7 +162,7 @@ Put inside double square brackets: `!template id=pagedraft`
 -   Put the speaker introduction right after the talk description for easier web page copying.
 
 
-<a id="org897f0c8"></a>
+<a id="phases-and-lessons-learned-announce"></a>
 
 ## Announce
 
@@ -165,7 +170,25 @@ Put inside double square brackets: `!template id=pagedraft`
 -   Submit to  <https://foss.events/>
 
 
-<a id="orgd12f3b1"></a>
+### Promote the conference
+
+Submit to <https://foss.events/>
+
+Maybe it would be good to get in touch with podcasts like
+
+-   [System Crafters](https://systemcrafters.net/)
+-   [This Week in Linux](https://www.youtube.com/playlist?list=PLbFVcOQ-YH_LRP687N0YeN78YZmBp5wqF)
+-   [Linux Unplugged](https://linuxunplugged.com/)
+-   [Ask Noah](http://asknoahshow.com/)
+-   [Linux After Dark](https://linuxafterdark.net/)
+-   [Lispy Gopher Show](https://anonradio.net/)
+
+etc to give a heads up on EmacsConf before it
+happens and also mention to them when videos are
+available to spark more interest.
+
+
+<a id="phases-and-lessons-learned-process-proposals"></a>
 
 ## Process proposals
 
@@ -194,7 +217,7 @@ When proposals come in:
     PUBLIC\_EMAIL
 
 
-<a id="orgcbfca65"></a>
+<a id="phases-and-lessons-learned-accept-proposals"></a>
 
 ## Accept proposals
 
@@ -227,7 +250,7 @@ For each talk:
 -   Add missing items to plan
 
 
-<a id="orgdbb77f2"></a>
+<a id="phases-and-lessons-learned-set-up-backstage-and-upload"></a>
 
 ## Set up backstage and upload
 
@@ -238,7 +261,7 @@ Lessons learned:
 -   Next year, let&rsquo;s get upload.emacsconf.org in place as the domain name so that people don&rsquo;t try to FTP files to it.
 
 
-<a id="org9456d25"></a>
+<a id="phases-and-lessons-learned-post-the-schedule"></a>
 
 ## Post the schedule
 
@@ -249,7 +272,7 @@ Lessons learned:
 -   Update talks.md to refer to watch.md as well
 
 
-<a id="org75e87bb"></a>
+<a id="phases-and-lessons-learned-process-uploaded-files"></a>
 
 ## Process uploaded files
 
@@ -274,7 +297,7 @@ Process:
     3.  <elisp:emacsconf-publish-backstage-index>
 
 
-<a id="org125bd75"></a>
+<a id="phases-and-lessons-learned-generate-assets"></a>
 
 ## Generate assets
 
@@ -292,7 +315,7 @@ Process:
 -   Splice together into a file that will be processed with subed-record-compile-video.
 
 
-<a id="org9759aa9"></a>
+<a id="phases-and-lessons-learned-prepare-the-infrastructure"></a>
 
 ## Prepare the infrastructure
 
@@ -314,7 +337,7 @@ Checklist:
 -   [ ] Icecast
 
 
-<a id="org9f978d1"></a>
+<a id="phases-and-lessons-learned-getting-ready-for-captions"></a>
 
 ## Getting ready for captions
 
@@ -344,6 +367,9 @@ Go through the :tminustwo: tags.
 
 Go through the :tminusone: tags.
 
+-   Set `emacsconf-publishing-phase` to `conference`.
+-   Set emacsconf-publish-include-pads to t and republish the info pages.
+
 
 <a id="conf"></a>
 
@@ -352,11 +378,198 @@ Go through the :tminusone: tags.
 <elisp:emacsconf-pad-open-shift-hyperlist>
 
 
-<a id="org85c6f49"></a>
+<a id="phases-and-lessons-learned-after-the-conference"></a>
+
+## After the conference
+
+
+### Capture stats
+
+2024 stats
+
+Saturday:
+gen: 177 peak + 14 peak lowres
+dev: 226 peak + 79 peak lowres
+
+Sunday:
+gen: 89 peak + 10 peak lowres
+
+meet peak 409% CPU (100% is 1 CPU), average 69.4%
+front peak 70.66% CPU (100% is 1 CPU)
+live peak 552% CPU (100% is 1 CPU) average 144%
+res peak 81.54% total CPU (each OBS ~250%), mem 7GB used
+
+so far we have used 1.2TB of transfer this month
+
+
+### Back up the data
+
+
+#### Icecast
+
+rsync -avze ssh live:&rsquo;/data/emacsconf-2024-\*&rsquo; media:~/2024
+
+rsync -avze ssh live:&rsquo;/data/emacsconf-2024-\*&rsquo; .
+
+
+#### BigBlueButton
+
+rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/
+
+ssh root@66.175.208.243 &rsquo;dd if=/dev/sda bs=5M &rsquo; | dd of=bbb-img-2024-12-08.img status=progress
+
+The img can be loaded with
+sudo losetup &#x2013;find &#x2013;show bbb-img-2024-12-08.img
+sudo mount /dev/loop10 test
+cd test
+
+
+##### Reviewing all the Q&A durations and chat messages
+
+    (let ((default-directory "/home/sacha/proj/emacsconf/2024/backups/bigbluebutton/published/video"))
+    	(mapconcat
+    	 (lambda (dir)
+    		 (let ((chat (xml-parse-file
+    									(expand-file-name "video.xml" dir)))
+    					 (metadata (xml-parse-file
+    											(expand-file-name "metadata.xml" dir))))
+    			 (concat
+    				"- "
+    				(dom-text (car (dom-by-tag metadata 'meetingName))) " ("
+    				(format-seconds  "%.2h:%z%.2m:%.2s"
+    												 (/ (string-to-number (dom-text (dom-by-tag metadata 'duration)))
+    														1000))
+    				")"
+    				"\n"
+    				(if (dom-by-tag chat 'chattimeline)
+    						(mapconcat (lambda (node)
+    												 (concat "  - " (dom-attr node 'name) ": "
+    																 (dom-attr node 'message) "\n"))
+    											 (dom-by-tag chat 'chattimeline)
+    											 "")
+    					"")
+    				"")
+    			 )
+    		 )
+    	 (directory-files "." nil "-"))
+    	)
+
+-   Copy IRC logs to cache and therefore to backstage: (ex: 2024-12-07-emacsconf-gen.txt)
+
+
+### Combine the IRC logs and the pads, and add them to the wiki
+
+(did we e-mail the speakers before or after?)
+
+
+### Process the Q&A recordings and live presentations
+
+-   Save answers videos as &#x2013;answers.webm.
+-   From the pad file: emacsconf-extract-copy-pad-to-wiki
+-   update cache
+-   Set `QA_PUBLIC` property to t as each Q&A gets processed.
+-   change status to `TO_INDEX_QA`
+-   emacsconf-publish-media-files-on-change
+-   emacsconf-publish-info-pages-for-talk
+
+Indexing
+
+-   emacsconf-extract-insert-note-with-question-heading
+-   emacsconf-subed-make-chapter-file-based-on-comments
+-   change status to `TO_CAPTION_QA`
+
+Also copy the YouTube comments and IRC comments
+emacsconf-extract-irc-anonymize-log
+
+
+### Send thank-you emails to the speakers
+
+
+### Notify emacs-discuss that the Q&A is up
+
+<https://lists.gnu.org/archive/html/emacsconf-discuss/2024-12/msg00000.html>
+
+    Hello, everyone!
+    
+    https://emacsconf.org/2024/talks and https://media.emacsconf.org/2024
+    should have the talks and Q&A sessions now, hooray, including chapter
+    indices for the Q&A. I've also copied over notes from Etherpad and IRC
+    on the talk pages, and I've e-mailed them to the speakers so they can
+    update the wiki directly or send in additional answers.
+    
+    BigBlueButton audio mixing is kinda iffy, so I'm holding off on
+    uploading the Q&A videos (and the live talk for secrets) to
+    YouTube/Toobnix until Leo and other volunteers have a listen to see
+    whether the audio needs to be tweaked, and maybe even to do some kind of
+    normalization. Unfortunately, recordings with separate audio tracks per
+    participant aren't available--it would be cool to figure that out next
+    year--so some talks might need some manual processing. (Please feel free
+    to volunteer if that's up your alley!) In the meantime, you can check
+    out the Q&A videos from the talk pages on the wiki. Please let us know
+    if there's anything that needs tweaking.
+    
+    Enjoy!
+    
+    Sacha
+
+
+### Write a report
+
+Sample: <https://emacsconf.org/2023/report/>
+
+
+### Send thank-you emails to the volunteers
+
+emacsconf-mail-template-ask-volunteer-for-mailing-address
+
+
+### Process the lessons learned
+
+
+<a id="general-infrastructure"></a>
 
 # General infrastructure
 
 
+<a id="general-infrastructure-linode-instance-sizes"></a>
+
+## Linode instance sizes
+
+
+### Production
+
+-   front0 shared Linode 32GB
+-   live0 shared Linode 64GB
+-   meet Dedicated CPU 16GB
+
+
+### Dormant
+
+Resize disk to 25GB, copying large files to media if needed
+
+-   nanode front0
+-   nanode live0
+-   delete meet after downloading all the recordings, or resize down to nanode
+
+
+<a id="upload"></a>
+
+## File uploads
+
+[psi-4ward/psitransfer: Simple open source self-hosted file sharing solution](https://github.com/psi-4ward/psitransfer)
+
+Before the conference:
+
+1.  Set `upload_enabled` to `true` in `prod-vars.yml` and run `ansible-playbook -i inventory.yml prod-playbook.yml --tags upload`.
+2.  Confirm that <https://upload.emacsconf.org> has the upload interface and works with the password.
+3.  Use `emacsconf-mail-template-to-all-groups` and `emacsconf-mail-upload-and-backstage-info` to e-mail the upload information to all speakers.
+
+After the conference:
+
+1.  Set `upload_enabled` to `false` in `prod-vars.yml` and run `ansible-playbook -i inventory.yml prod-playbook.yml --tags upload`.
+2.  Confirm that <https://upload.emacsconf.org> shows the in-between page.
+
+
 <a id="icecast"></a>
 
 ## Icecast
@@ -516,7 +729,7 @@ so that the year is updated in the configuration.
 This is on live.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`.
 
 
-<a id="orgab7108a"></a>
+<a id="general-infrastructure-shell-scripts"></a>
 
 ## Shell scripts
 
@@ -630,7 +843,7 @@ Explanation of files:
 Other files might also have been uploaded by the speaker, such as slides or notes.
 
 
-<a id="org5545d42"></a>
+<a id="general-infrastructure-irc-web-client"></a>
 
 ## IRC web client
 
@@ -652,7 +865,7 @@ connection exhaustion errors. Is that something we can plan now or would
 you like me to bring it up a few days before the conference?
 
 
-<a id="orgd7fd9ee"></a>
+<a id="general-infrastructure-etherpad"></a>
 
 ## Etherpad
 
@@ -661,6 +874,10 @@ Success:
 -   [X] pad.emacsconf.org redirects to this year&rsquo;s entry
 -   [X] Each talk has its own Etherpad
 
+Lessons learned:
+
+-   After 2024: Added pronouns and pronunciation to the pad template since that&rsquo;s what the hosts will be looking at
+
 
 <a id="media"></a>
 
@@ -688,7 +905,7 @@ Switching it back to `TO_STREAM` and calling
 You can generate the index with `emacsconf-publish-update-media`.
 
 
-<a id="org75cad9b"></a>
+<a id="general-infrastructure-automated-irc-announcements"></a>
 
 ## Automated IRC announcements
 
@@ -712,8 +929,6 @@ ssh root@bbb
 cd greenlight-v3
 docker compose restart
 
-Does not seem to work in Firefox private browsing; documented at <https://emacsconf.org/2024/watch/>.
-
 
 ### System audio limitations
 
@@ -728,7 +943,41 @@ BBB says sharing system audio works only if you use Chrome on Mac OS X or Micros
 Related: <https://github.com/bigbluebutton/bigbluebutton/issues/8632>
 
 
-<a id="org5a3d818"></a>
+### Creating talk BBB rooms
+
+-   `docker exec -it greenlight-v3 /bin/bash`
+-   `bundle exec rails console`
+-   `user_id = User.find_by_email("sacha@sachachua.com").id`
+
+    (mapconcat (lambda (group)
+    					(format
+    					 "Room.create(user_id: user_id, name: \"%s - %s\")\n"
+    					 (plist-get (cadr group) :speakers)
+    					 (string-join (mapcar (lambda (talk) (plist-get talk :slug))
+    																			 (cdr group)))))
+    				(emacsconf-mail-groups (emacsconf-active-talks (emacsconf-get-talk-info)))
+    				"")
+
+-   <elisp:emacsconf-publish-bbb-static-redirects>
+
+Print out the room IDs with
+
+    Room.all.each { |x| puts x.friendly_id + " " + x.name }; nil
+
+
+### Setting up moderator access codes
+
+
+### Backing up BBB     :backup:
+
+rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/
+
+ssh root@bbb &rsquo;tar zcvf - /var/bigbluebutton /etc/bigbluebutton /root/greenlight-v3 /usr/local/bigbluebutton /usr/share/bbb-web&rsquo; > bbb-backup-$(date &ldquo;+%Y-%m-%d&rdquo;).tar.gz
+
+ssh root@bbb &rsquo;dd if=/dev/sda bs=5M &rsquo; | dd of=bbb-img-$(date &ldquo;+%Y-%m-%d&rdquo;).img status=progress
+
+
+<a id="general-infrastructure-manual-irc-announcements"></a>
 
 ## Manual IRC announcements
 
@@ -737,7 +986,7 @@ Success:
 -   [X] You can /opall, /conftopic, and /broadcast
 
 
-<a id="orge08d62d"></a>
+<a id="general-infrastructure-low-res-stream"></a>
 
 ## Low-res stream
 
@@ -748,7 +997,7 @@ backup on live:
 -   /usr/local/bin/emacsconf-lowres-dev-on-connect
 
 
-<a id="org7f2e95e"></a>
+<a id="general-infrastructure-upload-talks-to-youtube"></a>
 
 ## Upload talks to YouTube
 
@@ -784,7 +1033,7 @@ backup on live:
      (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))
 
 
-<a id="org0cc19c3"></a>
+<a id="general-infrastructure-restream-to-youtube"></a>
 
 ## Restream to YouTube
 
@@ -802,7 +1051,7 @@ backup on live:
     docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"]
 
 
-<a id="org4efd774"></a>
+<a id="other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted"></a>
 
 ## When a talk is added after the schedule has already been drafted
 
@@ -826,12 +1075,28 @@ backup on live:
 
 ## When a talk is cancelled
 
-1.  Update the status to CANCELLED.
+1.  Update the status to CANCELLED. Update that specific talk with <elisp:emacsconf-publish-info-pages-for-talk>.
 2.  Rejig the schedule if needed, and update with <elisp:emacsconf-schedule-update-from-info>
 3.  Update the published schedule with <elisp:emacsconf-update-schedule> and push the wiki.
 4.  Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file.
-5.  Sync the new intros to the /data/emacsconf/shared/$YEAR/assets/intros directory.
-6.  Regenerate the pads: <elisp:emacsconf-pad-prepopulate-all-talks>
+5.  Resync the cache directory.
+6.  Consider updating the screenshots for sat-open and sun-open.
+
+
+<a id="other-process-notes-when-a-talk-changes-title"></a>
+
+## When a talk changes title
+
+1.  Update the title in conf.org.
+2.  Delete the FILE\_PREFIX property.
+3.  Call <elisp:emacsconf-set-file-prefix-if-needed>.
+4.  Rename existing files with <elisp:emacsconf-update-file-prefixes>.
+5.  Update the intro VTT file with the new title.
+6.  Update the wiki page for the talk.
+7.  Regenerate the in-between images with <elisp:emacsconf-stream-generate-in-between-pages>
+8.  Rerecord and recompile the intro, and recompile the intro for the talk after it.
+9.  <elisp:emacsconf-update-schedule>
+10. <elisp:emacsconf-pad-prepopulate-hyperlists>
 
 
 <a id="change-talk-qa"></a>
@@ -846,7 +1111,15 @@ backup on live:
 6.  Regenerate talks.json: <elisp:emacsconf-publish-talks-json-to-files>
 
 
-<a id="org17cc58c"></a>
+<a id="other-process-notes-to-play-the-other-stream"></a>
+
+## To play the other stream
+
+-   mpv &#x2013;profile=full <https://live0.emacsconf.org/gen.webm>
+-   Change the channel topic to note that discussion can be in the other channel
+
+
+<a id="other-process-notes-when-a-system-is-down"></a>
 
 ## When a system is down
 
@@ -859,7 +1132,7 @@ Update the status page:
 /broadcast <message> may also be helpful
 
 
-<a id="org4636e4e"></a>
+<a id="other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer"></a>
 
 ## When we need to bring on a last-minute volunteer
 
@@ -868,7 +1141,7 @@ Update the status page:
 will give them the backstage credentials and links to the index
 
 
-<a id="org811d67a"></a>
+<a id="other-process-notes-checking-people-in"></a>
 
 ## Checking people in
 
@@ -876,7 +1149,7 @@ will give them the backstage credentials and links to the index
 /room nick   (should be automatic if live, but there just in case)
 
 
-<a id="org5d78595"></a>
+<a id="other-process-notes-ansible"></a>
 
 ## Ansible
 
diff --git a/organizers-notebook/index.org b/organizers-notebook/index.org
index fe8bcc89..891218b0 100644
--- a/organizers-notebook/index.org
+++ b/organizers-notebook/index.org
@@ -76,9 +76,15 @@ Create linked pages
 - [ ] submit page
 
 *** How to mark pages as drafts
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-draft-cfp-how-to-mark-pages-as-drafts
+:END:
 
 Put inside double square brackets: =!template id=pagedraft=
 *** Previous years
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-draft-cfp-previous-years
+:END:
 
 [[file:~/proj/emacsconf/wiki/2023/cfp.org][2023]]
 
@@ -121,11 +127,17 @@ Put inside double square brackets: =!template id=pagedraft=
   European organizers and volunteers, though.
 - Put the speaker introduction right after the talk description for easier web page copying.
 ** Announce
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-announce
+:END:
 
 - Post CFP to emacsconf-discuss, reddit.com/r/emacs, and in Emacs News
 - Submit to  https://foss.events/
 
 *** Promote the conference
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-announce-promote-the-conference
+:END:
 
 Submit to https://foss.events/
 
@@ -143,6 +155,9 @@ happens and also mention to them when videos are
 available to spark more interest.
 
 ** Process proposals
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-process-proposals
+:END:
 
 1. In the private repository, create a conf.org.
 2. Set ~emacsconf-org-file~ to its location.
@@ -170,6 +185,9 @@ When proposals come in:
      TRACK, TIMEZONE, CATEGORY, DATE_SUBMITTED,
      PUBLIC_EMAIL
 ** Accept proposals
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-accept-proposals
+:END:
 
 Setup:
 
@@ -197,6 +215,9 @@ For each talk:
 - Create draft-schedule section in the public organizers notebook for the year
 - Add missing items to plan
 ** Set up backstage and upload
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-set-up-backstage-and-upload
+:END:
 
 See [[emacsconf-ansible:README.org]].
 
@@ -205,6 +226,9 @@ Lessons learned:
 - Next year, let's get upload.emacsconf.org in place as the domain name so that people don't try to FTP files to it.
 
 ** Post the schedule
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-post-the-schedule
+:END:
 
 - [[elisp:(setq emacsconf-publishing-phase 'schedule)]]
 - elisp:emacsconf-publish-info-pages
@@ -212,6 +236,9 @@ Lessons learned:
 - elisp:emacsconf-ical-generate-all
 - Update talks.md to refer to watch.md as well
 ** Process uploaded files
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-process-uploaded-files
+:END:
 
 Basic setup:
 
@@ -233,14 +260,26 @@ Process:
    2. elisp:emacsconf-publish-cache-video-data or elisp:emacsconf-cache-all-video-data
    3. elisp:emacsconf-publish-backstage-index
 ** Generate assets
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-generate-assets
+:END:
 *** In-between pages
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-generate-assets-in-between-pages
+:END:
 - Make the $year/assets/in-between directory and copy the template.svg from the previous year.
 - elisp:emacsconf-stream-generate-in-between-pages
 *** Intros
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-generate-assets-intros
+:END:
 - In an empty VTT file, elisp:emacsconf-subed-intro-subtitles
 - Record the audio and get captions
 - Splice together into a file that will be processed with subed-record-compile-video.
 ** Prepare the infrastructure
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-prepare-the-infrastructure
+:END:
 
 Checklist:
 
@@ -260,6 +299,9 @@ Checklist:
 - [ ] Icecast
 
 ** Getting ready for captions
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-captions
+:END:
 
 Onboarding:
 
@@ -276,13 +318,20 @@ elisp:emacsconf-pad-prepopulate-shift-hyperlists
 SCHEDULED: <2024-11-30 Sat>
 :PROPERTIES:
 :CREATED:  [2024-11-27 Wed 17:34]
+:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-send-check-in-details-schedule-update-etc
 :END:
 
 *** Two days before the conference
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-two-days-before-the-conference
+:END:
 
 Go through the :tminustwo: tags.
 
 *** One day before the conference
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-one-day-before-the-conference
+:END:
 
 Go through the :tminusone: tags.
 
@@ -295,7 +344,13 @@ Go through the :tminusone: tags.
 :END:
 elisp:emacsconf-pad-open-shift-hyperlist
 ** After the conference
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference
+:END:
 *** Capture stats
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-capture-stats
+:END:
 2024 stats
 
 Saturday:
@@ -312,13 +367,22 @@ res peak 81.54% total CPU (each OBS ~250%), mem 7GB used
 
 so far we have used 1.2TB of transfer this month
 *** Back up the data
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data
+:END:
 **** Icecast
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-icecast
+:END:
 
 rsync -avze ssh live:'/data/emacsconf-2024-*' media:~/2024
 
 rsync -avze ssh live:'/data/emacsconf-2024-*' .
 
 **** BigBlueButton
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-bigbluebutton
+:END:
 
 rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/
 
@@ -329,6 +393,9 @@ sudo losetup --find --show bbb-img-2024-12-08.img
 sudo mount /dev/loop10 test
 cd test
 ***** Reviewing all the Q&A durations and chat messages
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-bigbluebutton-reviewing-all-the-q-a-durations-and-chat-messages
+:END:
 #+begin_src emacs-lisp :eval no
 (let ((default-directory "/home/sacha/proj/emacsconf/2024/backups/bigbluebutton/published/video"))
 	(mapconcat
@@ -362,8 +429,14 @@ cd test
 - Copy IRC logs to cache and therefore to backstage: (ex: 2024-12-07-emacsconf-gen.txt)
 
 *** Combine the IRC logs and the pads, and add them to the wiki
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-combine-the-irc-logs-and-the-pads-and-add-them-to-the-wiki
+:END:
 (did we e-mail the speakers before or after?)
 *** Process the Q&A recordings and live presentations
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-process-the-q-a-recordings-and-live-presentations
+:END:
 
 - Save answers videos as --answers.webm.
 - From the pad file: emacsconf-extract-copy-pad-to-wiki
@@ -383,7 +456,13 @@ Also copy the YouTube comments and IRC comments
 emacsconf-extract-irc-anonymize-log
 
 *** Send thank-you emails to the speakers
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-send-thank-you-emails-to-the-speakers
+:END:
 *** Notify emacs-discuss that the Q&A is up
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-notify-emacs-discuss-that-the-q-a-is-up
+:END:
 
 https://lists.gnu.org/archive/html/emacsconf-discuss/2024-12/msg00000.html
 
@@ -413,25 +492,62 @@ Sacha
 #+end_example
 
 *** Write a report
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-write-a-report
+:END:
 Sample: https://emacsconf.org/2023/report/
 *** Send thank-you emails to the volunteers
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-send-thank-you-emails-to-the-volunteers
+:END:
 
 emacsconf-mail-template-ask-volunteer-for-mailing-address
 
 *** Process the lessons learned
+:PROPERTIES:
+:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-process-the-lessons-learned
+:END:
 * General infrastructure
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure
+:END:
 ** Linode instance sizes
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-linode-instance-sizes
+:END:
 *** Production
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-linode-instance-sizes-production
+:END:
 - front0 shared Linode 32GB
 - live0 shared Linode 64GB
 - meet Dedicated CPU 16GB
 *** Dormant
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-linode-instance-sizes-dormant
+:END:
 Resize disk to 25GB, copying large files to media if needed
 
 - nanode front0
 - nanode live0
 - delete meet after downloading all the recordings, or resize down to nanode
+** File uploads
+:PROPERTIES:
+:CUSTOM_ID: upload
+:END:
+
+[[https://github.com/psi-4ward/psitransfer][psi-4ward/psitransfer: Simple open source self-hosted file sharing solution]]
+
+Before the conference:
 
+1. Set ~upload_enabled~ to ~true~ in ~prod-vars.yml~ and run ~ansible-playbook -i inventory.yml prod-playbook.yml --tags upload~.
+2. Confirm that https://upload.emacsconf.org has the upload interface and works with the password.
+3. Use ~emacsconf-mail-template-to-all-groups~ and ~emacsconf-mail-upload-and-backstage-info~ to e-mail the upload information to all speakers.
+
+After the conference:
+
+1. Set ~upload_enabled~ to ~false~ in ~prod-vars.yml~ and run ~ansible-playbook -i inventory.yml prod-playbook.yml --tags upload~.
+2. Confirm that https://upload.emacsconf.org shows the in-between page.
 
 ** Icecast
 :PROPERTIES:
@@ -602,6 +718,9 @@ so that the year is updated in the configuration.
 
 This is on live.emacsconf.org and can be restarted with =/etc/init.d/emacsconf restart=.
 ** Shell scripts
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-shell-scripts
+:END:
 
 Success: From the commandline, hosts can:
 - [X] play just the intro: intro $SLUG
@@ -648,6 +767,9 @@ Explanation of files:
 Other files might also have been uploaded by the speaker, such as slides or notes.
 
 ** IRC web client
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-irc-web-client
+:END:
 
 Success:
 - [X] You can join from https://chat.emacsconf.org
@@ -665,11 +787,18 @@ allowed from chat.emacsconf.org during conference days in order to avoid
 connection exhaustion errors. Is that something we can plan now or would
 you like me to bring it up a few days before the conference?
 ** Etherpad
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-etherpad
+:END:
 
 Success:
 
 - [X] pad.emacsconf.org redirects to this year's entry
 - [X] Each talk has its own Etherpad
+
+Lessons learned:
+
+- After 2024: Added pronouns and pronunciation to the pad template since that's what the hosts will be looking at
 ** Publishing media to the server and to the wiki
 :PROPERTIES:
 :CUSTOM_ID: media
@@ -688,6 +817,9 @@ Switching it back to =TO_STREAM= and calling
 elisp:emacsconf-publish-media-files-on-change should remove it.
 
 *** TODO Switch public media to unprotected root before the conference  :tminustwo:
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-publishing-media-to-the-server-and-to-the-wiki-switch-public-media-to-unprotected-root-before-the-conference
+:END:
 
 1. Clear public media directory.
 2. Set =media_protect_root= to false in Ansible =group_vars/all.yml=.
@@ -695,6 +827,9 @@ elisp:emacsconf-publish-media-files-on-change should remove it.
 
 You can generate the index with =emacsconf-publish-update-media=.
 ** Automated IRC announcements
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-automated-irc-announcements
+:END:
 
 Success:
 
@@ -733,6 +868,9 @@ Related: https://github.com/bigbluebutton/bigbluebutton/issues/8632
 
 
 *** Creating talk BBB rooms
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-bigbluebutton-creating-talk-bbb-rooms
+:END:
 
 - =docker exec -it greenlight-v3 /bin/bash=
 - =bundle exec rails console=
@@ -758,7 +896,13 @@ Room.all.each { |x| puts x.friendly_id + " " + x.name }; nil
 #+end_src
 
 *** Setting up moderator access codes
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-bigbluebutton-setting-up-moderator-access-codes
+:END:
 *** Backing up BBB                                                 :backup:
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-bigbluebutton-backing-up-bbb
+:END:
 
 rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/
 
@@ -767,12 +911,18 @@ ssh root@bbb 'tar zcvf - /var/bigbluebutton /etc/bigbluebutton /root/greenlight-
 ssh root@bbb 'dd if=/dev/sda bs=5M ' | dd of=bbb-img-$(date "+%Y-%m-%d").img status=progress
 
 ** Manual IRC announcements
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-manual-irc-announcements
+:END:
 
 Success:
 
 - [X] You can /opall, /conftopic, and /broadcast
 
 ** Low-res stream
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-low-res-stream
+:END:
 Should be automatic, handled by Icecast
 backup on live:
 - /usr/local/bin/emacsconf-lowres-gen-on-connect
@@ -781,6 +931,7 @@ backup on live:
 SCHEDULED: <2024-11-30 Sat>
 :PROPERTIES:
 :CREATED:  [2024-11-27 Wed 17:33]
+:CUSTOM_ID: general-infrastructure-upload-talks-to-youtube
 :END:
 
 [[https://studio.youtube.com/channel/UCwuyodzTl_KdEKNuJmeo99A/videos/upload?filter=%5B%5D&sort=%7B%22columnType%22%3A%22date%22%2C%22sortOrder%22%3A%22DESCENDING%22%7D][Channel videos]]
@@ -830,6 +981,9 @@ SCHEDULED: <2024-11-30 Sat>
 :end:
 
 ** Restream to YouTube
+:PROPERTIES:
+:CUSTOM_ID: general-infrastructure-restream-to-youtube
+:END:
 
 
 * Other process notes
@@ -846,6 +1000,9 @@ ssh bbb.emacsverse.org
 docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"]
 #+end_src
 ** When a talk is added after the schedule has already been drafted
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted
+:END:
 - [X] Add the talk to conf.org
 - [X] Add the talk ID to the draft schedule
 - [X] Run the draft schedule block
@@ -874,6 +1031,9 @@ docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","P
 6. Consider updating the screenshots for sat-open and sun-open.
 
 ** When a talk changes title
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-when-a-talk-changes-title
+:END:
 
 1. Update the title in conf.org.
 2. Delete the FILE_PREFIX property.
@@ -897,11 +1057,17 @@ docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","P
 5. Regenerate the pads: elisp:emacsconf-pad-prepopulate-all-talks
 6. Regenerate talks.json: elisp:emacsconf-publish-talks-json-to-files
 ** To play the other stream
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-to-play-the-other-stream
+:END:
 
 - mpv --profile=full https://live0.emacsconf.org/gen.webm
 - Change the channel topic to note that discussion can be in the other channel
 
 ** When a system is down
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-when-a-system-is-down
+:END:
 
 
 Let people know in #emacsconf and #emacsconf-org
@@ -912,17 +1078,26 @@ Update the status page:
 
 /broadcast <message> may also be helpful
 ** When we need to bring on a last-minute volunteer
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer
+:END:
 
 /backstage nick
 
 will give them the backstage credentials and links to the index
 
 ** Checking people in
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-checking-people-in
+:END:
 
 /checkin nick
 /room nick   (should be automatic if live, but there just in case)
 
 ** Ansible
+:PROPERTIES:
+:CUSTOM_ID: other-process-notes-ansible
+:END:
 
 #+begin_src sh :eval no
 git clone git@git.emacsconf.org:pub/emacsconf-ansible