diff options
Diffstat (limited to '')
| -rw-r--r-- | organizers-notebook.md | 810 |
1 files changed, 738 insertions, 72 deletions
diff --git a/organizers-notebook.md b/organizers-notebook.md index bcaa74f7..a336f9fd 100644 --- a/organizers-notebook.md +++ b/organizers-notebook.md @@ -20,36 +20,49 @@ This document is the general organizers' notebook that can be used as a starting - [Create the public organizers’ notebook in the wiki](#starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki) - [Phases and lessons learned](#phases-and-lessons-learned) - [Draft CFP](#phases-and-lessons-learned-draft-cfp) - - [Announce](#org0e21b8c) - - [Process proposals](#org0b948ff) - - [Accept proposals](#org709c243) + - [Announce](#phases-and-lessons-learned-announce) + - [Process proposals](#phases-and-lessons-learned-process-proposals) + - [Accept proposals](#phases-and-lessons-learned-accept-proposals) - [Draft schedule](#draft-schedule) - - [Set up backstage and upload](#orgdc12c78) - - [Post the schedule](#orgce4ff13) - - [Process uploaded files](#org6cdaa28) - - [Generate assets](#org3c7eeb9) - - [Prepare the infrastructure](#orgf502d28) - - [Getting ready for captions](#org00f1270) -- [General infrastructure](#orga2f2794) + - [Set up backstage and upload](#phases-and-lessons-learned-set-up-backstage-and-upload) + - [Post the schedule](#phases-and-lessons-learned-post-the-schedule) + - [Process uploaded files](#phases-and-lessons-learned-process-uploaded-files) + - [Generate assets](#phases-and-lessons-learned-generate-assets) + - [Prepare the infrastructure](#phases-and-lessons-learned-prepare-the-infrastructure) + - [Getting ready for captions](#phases-and-lessons-learned-getting-ready-for-captions) + - [Getting ready for the conference](#pre-conf) + - [During the conference](#conf) + - [After the conference](#phases-and-lessons-learned-after-the-conference) +- [Ideas for improvement](#ideas) +- [General infrastructure](#general-infrastructure) + - [Linode instance sizes](#general-infrastructure-linode-instance-sizes) + - [File uploads](#upload) - [Icecast](#icecast) - [Watching pages](#watch) - [VNC](#vnc) - [Streaming with OBS](#obs) - - [Shell scripts](#org93c38b4) + - [Shell scripts](#general-infrastructure-shell-scripts) - [Backstage](#backstage) - - [IRC web client](#org75fb0fd) - - [Etherpad](#orgadd3307) + - [IRC web client](#general-infrastructure-irc-web-client) + - [Etherpad](#general-infrastructure-etherpad) - [Publishing media to the server and to the wiki](#media) - - [Automated IRC announcements](#org0b45c02) - - [Manual IRC announcements](#orgf31a707) - - [Low-res stream](#org2b9c281) - - [Restream to YouTube](#org00999f1) - - [Restream to Toobnix](#org1d92e71) + - [Automated IRC announcements](#general-infrastructure-automated-irc-announcements) + - [BigBlueButton](#bbb) + - [Manual IRC announcements](#general-infrastructure-manual-irc-announcements) + - [Low-res stream](#general-infrastructure-low-res-stream) + - [Upload talks to YouTube](#general-infrastructure-upload-talks-to-youtube) + - [Restream to YouTube](#general-infrastructure-restream-to-youtube) - [Other process notes](#other) - [Adding another user to BBB](#bbb-user) + - [When a talk is added after the schedule has already been drafted](#other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted) - [When a talk is cancelled](#cancel-talk) + - [When a talk changes title](#other-process-notes-when-a-talk-changes-title) - [When a talk Q&A changes method](#change-talk-qa) - - [Ansible](#orge2b5368) + - [To play the other stream](#other-process-notes-to-play-the-other-stream) + - [When a system is down](#other-process-notes-when-a-system-is-down) + - [When we need to bring on a last-minute volunteer](#other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer) + - [Checking people in](#other-process-notes-checking-people-in) + - [Ansible](#other-process-notes-ansible) <a id="starting-up-the-conference-planning-process"></a> @@ -67,24 +80,25 @@ This repository will store submissions, contact information, etc. 1. Create the repository on `front0`. Ex: + export YEAR=2025 sudo su - git cd repositories - mkdir emacsconf-2024-private.git - cd emacsconf-2024-private.git + mkdir emacsconf-$YEAR-private.git + cd emacsconf-$YEAR-private.git git init --bare -2. Edit `gitolite-admin/config/gitolite-conf`. Add lines for the group and the repo. Commit and push. Ex: +2. In the `gitolite-admin` repo, edit `conf/gitolite-conf`. Add lines for the group and the repo. Commit and push. Ex: @emacsconf-2024-org = bandali sachac ... repo emacsconf-2024-private - RW+ = @admins @emacsconf-2024-org orga -3. Clone the private repository into the appropriate location. `git clone git@git.emacsconf.org:emacsconf-2024-private private` + RW+ = @admins @emacsconf-2024-org orga +3. Clone the private repository into the appropriate location. `git clone git@git.emacsconf.org:emacsconf-2025-private private` <a id="starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki"></a> ## Create the public organizers’ notebook in the wiki -Ex: 2024/organizers-notebook/index.org +Ex: <../2024/organizers-notebook/index.md> <a id="phases-and-lessons-learned"></a> @@ -98,14 +112,67 @@ Ex: 2024/organizers-notebook/index.org Create linked pages -- [ ] year index -- [ ] volunteer page -- [ ] submit page +- [ ] year index: ex: <../2024.md> +- [ ] volunteer page: ex: <../volunteer.md> +- [ ] submit page: ex: <../2024/submit.md> ### How to mark pages as drafts -Put inside double square brackets: `!template id=pagedraft` +Have a Markdown export block with the following contents. Replace { and } with [ and ]. + +\#+begin\_export md +{{!template id=pagedraft}} +\#+end\_export md + + +### Wiki template for year index + +Replace { and } with [ and ]. + + {{!meta copyright="Copyright © 2021-2022 Amin Bandali; 2023-2025 Sacha Chua"}} + + <p class="center">EmacsConf 2025 | Online Conference<br /> + <b>December 6 and 7, 2024 (Sat-Sun)</b></p> + + <p class="center">{{!img /i/emacsconf-logo1-256.png + alt="EmacsConf logo"}}</p> + + <p class="center">{{<b>Call for Participation</b>|cfp}} | {{Ideas}} | + {{Submit a Proposal|submit}} | {{Volunteer}} | {{Guidelines for Conduct|conduct}}</p> + + <p class="center">EmacsConf is the conference about the joy of + <a href="https://www.gnu.org/software/emacs/">GNU Emacs</a> and + Emacs Lisp.</p> + + We are busy putting things together for EmacsConf 2025, and we would + love to have *your* help to make EmacsConf 2025 amazing, much like the + previous EmacsConfs. See our {{**Call for Participation**|cfp}}, get + involved, and help spread the word! + + We are holding EmacsConf 2025 as an online conference again this year. + We remain fully committed to freedom, and we will continue using our + infrastructure and streaming setup consisting entirely of {free + software}{freesw}, much like previous EmacsConf conferences. + + For general EmacsConf discussions, join the + {emacsconf-discuss}(https://lists.gnu.org/mailman/listinfo/emacsconf-discuss) + mailing list. For discussions related to organizing EmacsConf, join + the + {emacsconf-org}(https://lists.gnu.org/mailman/listinfo/emacsconf-org) + mailing list. You can email us publicly at + <mailto:emacsconf-org@gnu.org> or privately at + <mailto:emacsconf-org-private@gnu.org>. + + Come hang out with us in the `#emacsconf` channel on `irc.libera.chat` + ({Libera.Chat}{libera} IRC network). You can join the chat using + {your favourite IRC client}{libera-emacsconf}, or by visiting + {chat.emacsconf.org}{chat} in your web browser. + + {freesw}: https://www.gnu.org/philosophy/free-sw.html + {libera}: https://libera.chat + {libera-emacsconf}: ircs://irc.libera.chat:6697/emacsconf + {chat}: https://chat.emacsconf.org ### Previous years @@ -113,6 +180,21 @@ Put inside double square brackets: `!template id=pagedraft` [2023](file:///home/sacha/proj/emacsconf/wiki/2023/cfp.md) +### Update dates + +<file:///home/sacha/proj/emacsconf/lisp/emacsconf.el> +<file:///home/sacha/proj/emacsconf/lisp/emacsconf-erc.el> + + +### Update the IRC topic + + (erc-cmd-OPALL) + (erc-cmd-CONFTOPIC (emacsconf-replace-plist-in-string + `(:year ,emacsconf-year :cfp-deadline ,emacsconf-cfp-deadline) + "Getting ready for EmacsConf ${year}! Call for proposals: https://emacsconf.org/${year}/cfp by ${cfp-deadline}, volunteer: https://emacsconf.org/volunteer/")) + (erc-cmd-DEOPALL) + + ### Lessons learned - Maybe incentivize proper timezone specification by saying we can translate times to their local time? @@ -150,7 +232,7 @@ Put inside double square brackets: `!template id=pagedraft` - Put the speaker introduction right after the talk description for easier web page copying. -<a id="org0e21b8c"></a> +<a id="phases-and-lessons-learned-announce"></a> ## Announce @@ -158,7 +240,25 @@ Put inside double square brackets: `!template id=pagedraft` - Submit to <https://foss.events/> -<a id="org0b948ff"></a> +### Promote the conference + +Submit to <https://foss.events/> + +Maybe it would be good to get in touch with podcasts like + +- [System Crafters](https://systemcrafters.net/) +- [This Week in Linux](https://www.youtube.com/playlist?list=PLbFVcOQ-YH_LRP687N0YeN78YZmBp5wqF) +- [Linux Unplugged](https://linuxunplugged.com/) +- [Ask Noah](http://asknoahshow.com/) +- [Linux After Dark](https://linuxafterdark.net/) +- [Lispy Gopher Show](https://anonradio.net/) + +etc to give a heads up on EmacsConf before it +happens and also mention to them when videos are +available to spark more interest. + + +<a id="phases-and-lessons-learned-process-proposals"></a> ## Process proposals @@ -187,7 +287,7 @@ When proposals come in: PUBLIC\_EMAIL -<a id="org709c243"></a> +<a id="phases-and-lessons-learned-accept-proposals"></a> ## Accept proposals @@ -220,7 +320,7 @@ For each talk: - Add missing items to plan -<a id="orgdc12c78"></a> +<a id="phases-and-lessons-learned-set-up-backstage-and-upload"></a> ## Set up backstage and upload @@ -231,7 +331,7 @@ Lessons learned: - Next year, let’s get upload.emacsconf.org in place as the domain name so that people don’t try to FTP files to it. -<a id="orgce4ff13"></a> +<a id="phases-and-lessons-learned-post-the-schedule"></a> ## Post the schedule @@ -242,7 +342,7 @@ Lessons learned: - Update talks.md to refer to watch.md as well -<a id="org6cdaa28"></a> +<a id="phases-and-lessons-learned-process-uploaded-files"></a> ## Process uploaded files @@ -267,7 +367,7 @@ Process: 3. <elisp:emacsconf-publish-backstage-index> -<a id="org3c7eeb9"></a> +<a id="phases-and-lessons-learned-generate-assets"></a> ## Generate assets @@ -285,7 +385,7 @@ Process: - Splice together into a file that will be processed with subed-record-compile-video. -<a id="orgf502d28"></a> +<a id="phases-and-lessons-learned-prepare-the-infrastructure"></a> ## Prepare the infrastructure @@ -307,7 +407,7 @@ Checklist: - [ ] Icecast -<a id="org00f1270"></a> +<a id="phases-and-lessons-learned-getting-ready-for-captions"></a> ## Getting ready for captions @@ -318,11 +418,294 @@ Onboarding: - Send backstage information <elisp:emacsconf-mail-backstage-info-to-volunteer> . (Can be sent to all captioning volunteers with <elisp:emacsconf-mail-backstage-info-to-captioning-volunteers>) -<a id="orga2f2794"></a> +<a id="pre-conf"></a> + +## Getting ready for the conference + +<elisp:emacsconf-pad-prepopulate-shift-hyperlists> + + +### TODO Send check-in details, schedule update, etc. + + +### Two days before the conference + +Go through the :tminustwo: tags. + + +### One day before the conference + +Go through the :tminusone: tags. + +- Set `emacsconf-publishing-phase` to `conference`. +- Set emacsconf-publish-include-pads to t and republish the info pages. + + +<a id="conf"></a> + +## During the conference + +<elisp:emacsconf-pad-open-shift-hyperlist> + + +<a id="phases-and-lessons-learned-after-the-conference"></a> + +## After the conference + + +### Capture stats + +2024 stats + +Saturday: +gen: 177 peak + 14 peak lowres +dev: 226 peak + 79 peak lowres + +Sunday: +gen: 89 peak + 10 peak lowres + +meet peak 409% CPU (100% is 1 CPU), average 69.4% +front peak 70.66% CPU (100% is 1 CPU) +live peak 552% CPU (100% is 1 CPU) average 144% +res peak 81.54% total CPU (each OBS ~250%), mem 7GB used + +so far we have used 1.2TB of transfer this month + + +### Back up the data + + +#### Icecast + +rsync -avze ssh live:’/data/emacsconf-2024-\*’ media:~/2024 + +rsync -avze ssh live:’/data/emacsconf-2024-\*’ . + + +#### BigBlueButton + +rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/ + +ssh root@66.175.208.243 ’dd if=/dev/sda bs=5M ’ | dd of=bbb-img-2024-12-08.img status=progress + +The img can be loaded with +sudo losetup –find –show bbb-img-2024-12-08.img +sudo mount /dev/loop10 test +cd test + + +##### Reviewing all the Q&A durations and chat messages + + (let ((default-directory "/home/sacha/proj/emacsconf/2024/backups/bigbluebutton/published/video")) + (mapconcat + (lambda (dir) + (let ((chat (xml-parse-file + (expand-file-name "video.xml" dir))) + (metadata (xml-parse-file + (expand-file-name "metadata.xml" dir)))) + (concat + "- " + (dom-text (car (dom-by-tag metadata 'meetingName))) " (" + (format-seconds "%.2h:%z%.2m:%.2s" + (/ (string-to-number (dom-text (dom-by-tag metadata 'duration))) + 1000)) + ")" + "\n" + (if (dom-by-tag chat 'chattimeline) + (mapconcat (lambda (node) + (concat " - " (dom-attr node 'name) ": " + (dom-attr node 'message) "\n")) + (dom-by-tag chat 'chattimeline) + "") + "") + "") + ) + ) + (directory-files "." nil "-")) + ) + +- Copy IRC logs to cache and therefore to backstage: (ex: 2024-12-07-emacsconf-gen.txt) + + +### Combine the IRC logs and the pads, and add them to the wiki + +(did we e-mail the speakers before or after?) + + +### Process the Q&A recordings and live presentations + +- Save answers videos as –answers.webm. +- From the pad file: emacsconf-extract-copy-pad-to-wiki +- update cache +- Set `QA_PUBLIC` property to t as each Q&A gets processed. +- change status to `TO_INDEX_QA` +- emacsconf-publish-media-files-on-change +- emacsconf-publish-info-pages-for-talk + +Indexing + +- emacsconf-extract-insert-note-with-question-heading +- emacsconf-subed-make-chapter-file-based-on-comments +- change status to `TO_CAPTION_QA` + +Also copy the YouTube comments and IRC comments +emacsconf-extract-irc-anonymize-log + + +### Send thank-you emails to the speakers + + +### Notify emacs-discuss that the Q&A is up + +<https://lists.gnu.org/archive/html/emacsconf-discuss/2024-12/msg00000.html> + + Hello, everyone! + + https://emacsconf.org/2024/talks and https://media.emacsconf.org/2024 + should have the talks and Q&A sessions now, hooray, including chapter + indices for the Q&A. I've also copied over notes from Etherpad and IRC + on the talk pages, and I've e-mailed them to the speakers so they can + update the wiki directly or send in additional answers. + + BigBlueButton audio mixing is kinda iffy, so I'm holding off on + uploading the Q&A videos (and the live talk for secrets) to + YouTube/Toobnix until Leo and other volunteers have a listen to see + whether the audio needs to be tweaked, and maybe even to do some kind of + normalization. Unfortunately, recordings with separate audio tracks per + participant aren't available--it would be cool to figure that out next + year--so some talks might need some manual processing. (Please feel free + to volunteer if that's up your alley!) In the meantime, you can check + out the Q&A videos from the talk pages on the wiki. Please let us know + if there's anything that needs tweaking. + + Enjoy! + + Sacha + + +### Write a report + +Sample: <https://emacsconf.org/2023/report/> + + +### Send thank-you emails to the volunteers + +emacsconf-mail-template-ask-volunteer-for-mailing-address + + +### Process the lessons learned + + +<a id="ideas"></a> + +# Ideas for improvement + +How can we make EmacsConf better for participants? Speakers? Volunteers? People who come across the videos afterwards? + +- Emacs: + - view Etherpad (read-only): might be able to query something like <https://pad.emacsconf.org/2024/export/html> or <https://pad.emacsconf.org/2024/export/txt> + - view upcoming talks, countdown? + - resources, past talks + - ??: Collaborative real-time editing? Not sure crdt.el or collab-mode can scale up as far as we like; how would we test this? +- HTML/CSS/JS: + - general design improvements? + - [This is what the talks page looks like when there’s no schedule yet](https://web.archive.org/web/20240806134541/https://emacsconf.org/2024/talks/) + - [This is what the schedule page looks like before the conference](https://web.archive.org/web/20241108132608/https://emacsconf.org/2024/talks/) + - [This is what a talk page looks like before the conference](https://web.archive.org/web/20241208042046/https://emacsconf.org/2024/talks/literate/) + - [This is the watch page for the general track](https://web.archive.org/web/20241208034535/https://emacsconf.org/2024/watch/gen/) + - [This is the same talk page after the conference, with the videos, notes, and captions](https://web.archive.org/web/20241212145433/https://emacsconf.org/2024/talks/literate) + - [This is what the talks page looks like after the conference](https://emacsconf.org/2024/talks/) - list of resources + - timezone translations in the schedule SVG, on the schedule page, etc. + - countdown on talk pages? + - prettier viewers for videos, captions + - make it easier to see the talk/pad/IRC at the same time? + - easy timestamp jumping +- Volunteers: + - better documentation + - share to OBS from local? Maybe vdo.ninja? + - how do we manage the fallback/switchover in case we need to go back to shared VNC? + - meeting layout? + - countdown to switch over to the next talk, or easier manual control; web or command-line interface to control it? + - make it easier for people to join as hosts or check-in volunteers? + - improve backstage experience? + - automate or cross-train more people in what Sacha does (coordination, infrastructure) to reduce bus factor and increase awesomeness +- Processing: + - audio normalization: better workflow, more people helping + - noise removal (many speakers don’t have external mics) + - Makefile improvements + - captioning: + - simplify common edits + - reflowing? check out [sub-seg](https://gitlab.com/jun8git/sub-seg)? + - handle last-minute uploads + - validation: make sure audio, video, and caption times agree +- Infrastructure: + - add read-only non-JS links to pads; make the pads viewable through the web instead of being downloaded + - upgrade to latest versions + - Etherpad (deployed 1.9.7) + - DONE - BigBlueButton (deployed 3.0.12, which is the latest as of 2025-08-02) + - Consider 3.1 beta? new interface, probably new bugs; see if it’s worth it? + - Icecast? alternative? + - customize BigBlueButton welcome, slides, title, branding + - change the default presentation (maybe someone can make a PDF?) <https://docs.bigbluebutton.org/administration/customize/#change-the-default-presentation> + - modify the default landing page <https://docs.bigbluebutton.org/administration/customize/#modify-the-default-landing-page> + - change the default welcome messages <https://docs.bigbluebutton.org/administration/customize/#change-the-default-welcome-message> + - change the favicon <https://docs.bigbluebutton.org/administration/customize/#change-favicon> + - change the default logo <https://docs.bigbluebutton.org/administration/customize/#changing-the-default-logo> + - check out BigBlueButton live captions? + - figure out if we can connect to or reuse the etherpad in BigBlueButton? + - figure out how to record separate audio streams in BigBlueButton because the audio mixing is sometimes terrible (some people’s audio streams end up much louder than others). Related: <https://github.com/bigbluebutton/bigbluebutton/issues/12302> , but not sure how to actually do it… + - replicate BBB setup in a virtual machine so that it’s easy to re-deploy + - replicate streaming setup (OBS+VNC) in a virtual machine so that it’s easy to re-deploy + - replicate video processing workflow on my computer and/or a virtual machine +- Other: + - panel? some thoughts: <https://pad.emacsconf.org/emacsconf-panel> + - unconference? + + +<a id="general-infrastructure"></a> # General infrastructure +<a id="general-infrastructure-linode-instance-sizes"></a> + +## Linode instance sizes + + +### Production + +- front0 shared Linode 32GB +- live0 shared Linode 64GB +- meet Dedicated CPU 16GB + + +### Dormant + +Resize disk to 25GB, copying large files to media if needed + +- nanode front0 +- nanode live0 +- delete meet after downloading all the recordings, or resize down to nanode + + +<a id="upload"></a> + +## File uploads + +[psi-4ward/psitransfer: Simple open source self-hosted file sharing solution](https://github.com/psi-4ward/psitransfer) + +Before the conference: + +1. Set `upload_enabled` to `true` in `prod-vars.yml` and run `ansible-playbook -i inventory.yml prod-playbook.yml --tags upload`. +2. Confirm that <https://upload.emacsconf.org> has the upload interface and works with the password. +3. Use `emacsconf-mail-template-to-all-groups` and `emacsconf-mail-upload-and-backstage-info` to e-mail the upload information to all speakers. + +After the conference: + +1. Set `upload_enabled` to `false` in `prod-vars.yml` and run `ansible-playbook -i inventory.yml prod-playbook.yml --tags upload`. +2. Confirm that <https://upload.emacsconf.org> shows the in-between page. + + <a id="icecast"></a> ## Icecast @@ -373,7 +756,7 @@ Success: You can watch at <https://live.emacsconf.org>. New year: -1. ansible-playbook -i inventory.yml prod-playbook.yml –tags live +1. `ansible-playbook -i inventory.yml prod-playbook.yml --tags live` 2. Republish with <elisp:emacsconf-publish-watch-pages>. 3. Update front:/var/www/live.emacsconf.org. @@ -482,7 +865,7 @@ so that the year is updated in the configuration. This is on live.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`. -<a id="org93c38b4"></a> +<a id="general-infrastructure-shell-scripts"></a> ## Shell scripts @@ -527,67 +910,67 @@ Explanation of files: </colgroup> <tbody> <tr> -<td class="org-left">–original.mp4/mov/webm</td> +<td class="org-left"><code>--original.mp4/mov/webm</code></td> <td class="org-left">original file as uploaded by the speaker</td> </tr> <tr> -<td class="org-left">–silence.mp4/mov/webm</td> +<td class="org-left"><code>--silence.mp4/mov/webm</code></td> <td class="org-left">silence sample if uploaded by speaker</td> </tr> <tr> -<td class="org-left">–reencoded.webm</td> +<td class="org-left"><code>--reencoded.webm</code></td> <td class="org-left">converted to .webm format and compressed</td> </tr> <tr> -<td class="org-left">–reencoded.opus</td> +<td class="org-left"><code>--reencoded.opus</code></td> <td class="org-left">audio only</td> </tr> <tr> -<td class="org-left">–normalized.opus</td> +<td class="org-left"><code>--normalized.opus</code></td> <td class="org-left">normalized audio</td> </tr> <tr> -<td class="org-left">–reencoded.vtt</td> +<td class="org-left"><code>--reencoded.vtt</code></td> <td class="org-left">WhisperX captions, WebVTT format</td> </tr> <tr> -<td class="org-left">–reencoded.json</td> +<td class="org-left"><code>--reencoded.json</code></td> <td class="org-left">WhisperX word data</td> </tr> <tr> -<td class="org-left">–reencoded.srt</td> +<td class="org-left"><code>--reencoded.srt</code></td> <td class="org-left">WhisperX captions, SRT format</td> </tr> <tr> -<td class="org-left">–reencoded.txt</td> +<td class="org-left"><code>--reencoded.txt</code></td> <td class="org-left">WhisperX captions, plain text</td> </tr> <tr> -<td class="org-left">–main.vtt</td> +<td class="org-left"><code>--main.vtt</code></td> <td class="org-left">edited captions</td> </tr> <tr> -<td class="org-left">–main–chapters.vtt</td> +<td class="org-left"><code>--main--chapters.vtt</code></td> <td class="org-left">chapter markers</td> </tr> <tr> -<td class="org-left">–main.txt</td> +<td class="org-left"><code>--main.txt</code></td> <td class="org-left">captions as text (optional)</td> </tr> <tr> -<td class="org-left">–main.webm</td> +<td class="org-left"><code>--main.webm</code></td> <td class="org-left">version for streaming based on the most recent resources</td> </tr> </tbody> @@ -596,7 +979,7 @@ Explanation of files: Other files might also have been uploaded by the speaker, such as slides or notes. -<a id="org75fb0fd"></a> +<a id="general-infrastructure-irc-web-client"></a> ## IRC web client @@ -618,7 +1001,7 @@ connection exhaustion errors. Is that something we can plan now or would you like me to bring it up a few days before the conference? -<a id="orgadd3307"></a> +<a id="general-infrastructure-etherpad"></a> ## Etherpad @@ -627,6 +1010,17 @@ Success: - [X] pad.emacsconf.org redirects to this year’s entry - [X] Each talk has its own Etherpad +Lessons learned: + +- After 2024: Added pronouns and pronunciation to the pad template since that’s what the hosts will be looking at + +The pad is at front0.emacsconf.org + +Non-JS way to get plain text or HTML of a pad: ex: + +- <https://pad.emacsconf.org/orgmeetup/export/txt> +- <https://pad.emacsconf.org/orgmeetup/export/html> + <a id="media"></a> @@ -645,7 +1039,7 @@ Switching it back to `TO_STREAM` and calling <elisp:emacsconf-publish-media-files-on-change> should remove it. -### TODO Switch public media to unprotected root before the conference +### TODO Switch public media to unprotected root before the conference :tminustwo: 1. Clear public media directory. 2. Set `media_protect_root` to false in Ansible `group_vars/all.yml`. @@ -654,7 +1048,7 @@ Switching it back to `TO_STREAM` and calling You can generate the index with `emacsconf-publish-update-media`. -<a id="org0b45c02"></a> +<a id="general-infrastructure-automated-irc-announcements"></a> ## Automated IRC announcements @@ -663,7 +1057,168 @@ Success: - [ ] When a talk starts playing, it is announced in the relevant channel -<a id="orgf31a707"></a> +<a id="bbb"></a> + +## BigBlueButton + +ssh orga@media.emacsconf.org “~/bin/bbb-before sat-open” +<https://media.emacsconf.org/2024/current/bbb-sat-open.html> + +ssh orga@media.emacsconf.org “~/bin/bbb-open sat-open” + +After restarting, and in case of “Firefox can’t establish a connection to the server at wss://bbb.emacsverse.org/graphql.” + +ssh root@bbb +cd greenlight-v3 +docker compose restart + +Sizes: + +- dormant: 1 GB nanode +- testing: 4 core 8 GB shared CPU +- production: 8 core 16 GB dedicated CPU (roughly half CPU load for 107 simultaneous users, 2024) + + +### Installing BigBlueButton + +1. Create the instance. + - Linode 4core 8GB shared CPU + - Image: Ubuntu 22.04 +2. Update the DNS for bbb.emacsverse.org with its IP address. +3. Download the BBB installer and make it executable. +4. ufw allow 16384:32768/udp +5. Run the BBB installer: `./bbb-install.sh -v jammy-300 -s bbb.emacsverse.org -e emacsconf@sachachua.com -g` +6. `sudo apt install bbb-playback-video` and then follow the instructions at [Server Customization | BigBlueButton](https://docs.bigbluebutton.org/3.0/administration/customize/#install-additional-recording-processing-formats) . + - /usr/local/bigbluebutton/core/scripts/bigbluebutton.yml: + + steps: + archive: 'sanity' + sanity: 'captions' + captions: + - 'process:presentation' + - 'process:video' + 'process:presentation': 'publish:presentation' + 'process:video': 'publish:video' + - systemctl restart bbb-rap-resque-worker.service +7. docker exec -it greenlight-v3 bundle exec rake admin:create[’name’,’email’,’password’] + + +### System audio limitations + +Sharing system audio by changing the audio device to “Built-in Audio Analog Stereo” is very choppy, only gets captured in the recording if I’m using Firefox (not Chrome) AND there’s at least one listener, and doesn’t allow simultaneous narration from the speaker. + +Using pulseaudio to change the app’s recording device to the audio monitor results in smoother audio (because it’s pretending to be the microphone), but still doesn’t allow simultaneous narration from the speaker. + +Combining microphone + system audio into a new virtual microphone source is probably more complicated than we want to walk speakers through. + +BBB says sharing system audio works only if you use Chrome on Mac OS X or Microsoft Windows (<https://support.bigbluebutton.org/hc/en-us/articles/1500005315642-Can-I-play-a-video-using-screen-sharing>). I tested it on Linux and sharing system audio is not an option. If you happen to know of any more elegant way to do this, I’d love to hear it. + +Related: <https://github.com/bigbluebutton/bigbluebutton/issues/8632> + + +### Creating talk BBB rooms + +- `docker exec -it greenlight-v3 /bin/bash` +- `bundle exec rails console` +- `user_id = User.find_by_email("sacha@sachachua.com").id` + + (mapconcat (lambda (group) + (format + "Room.create(user_id: user_id, name: \"%s - %s\")\n" + (plist-get (cadr group) :speakers) + (string-join (mapcar (lambda (talk) (plist-get talk :slug)) + (cdr group))))) + (emacsconf-mail-groups (emacsconf-active-talks (emacsconf-get-talk-info))) + "") + +- <elisp:emacsconf-publish-bbb-static-redirects> + +Print out the room IDs with + + Room.all.each { |x| puts x.friendly_id + " " + x.name }; nil + +It’s possible to change the friendly\_id and then use `x.save!`. + + +### Setting up moderator access codes + + (dolist (talk (seq-filter (lambda (o) + (and (plist-get o :bbb-room) + (not (plist-get o :bbb-mod-code)))) + (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) + (spookfox-js-injection-eval-in-active-tab + (format "window.location.href = \"%s\"" + (replace-regexp-in-string "/join" "" (plist-get talk :bbb-room))) + t) + (sleep-for 3) + (spookfox-js-injection-eval-in-active-tab + "document.querySelector('button[data-rr-ui-event-key=\"settings\"]').click()" t) + (spookfox-js-injection-eval-in-active-tab + "document.querySelector('input#glAnyoneCanStart').checked = true") + (spookfox-js-injection-eval-in-active-tab + "document.querySelector('input#muteOnStart').checked = true") + (spookfox-js-injection-eval-in-active-tab + "document.querySelectorAll('.border-end button')[2].click()" t) + (let ((code (spookfox-js-injection-eval-in-active-tab + "document.querySelector('.access-code-input input').value" t))) + (message "Setting %s to %s" (plist-get talk :slug) code) + (emacsconf-set-property-from-slug + talk "BBB_MOD_CODE" + code) + (sit-for 2))) + + (dolist (talk (seq-filter (lambda (o) + (plist-get o :bbb-room)) + (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) + (spookfox-js-injection-eval-in-active-tab + (format "window.location.href = \"%s\"" + (replace-regexp-in-string "/join" "" (plist-get talk :bbb-room))) + t) + (sleep-for 3) + (spookfox-js-injection-eval-in-active-tab + "document.querySelector('button[data-rr-ui-event-key=\"settings\"]').click()" t) + (sleep-for 3)) + + +### Backing up BBB :backup: + +rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/ + +ssh root@bbb ’tar zcvf - /var/bigbluebutton /etc/bigbluebutton /root/greenlight-v3 /usr/local/bigbluebutton /usr/share/bbb-web’ > bbb-backup-$(date “+%Y-%m-%d”).tar.gz + +ssh root@bbb ’dd if=/dev/sda bs=5M ’ | dd of=bbb-img-$(date “+%Y-%m-%d”).img status=progress + + +### Spinning BBB up again + +<span class="timestamp-wrapper"><span class="timestamp">[2025-01-22 Wed]</span></span> … actually, this ran into some issues with the certificate, so I’m just going to do a reinstall. + +Previous notes + +1. Create a Nanode. +2. Update the DNS for bbb.emacsverse.org with its IP address. (dns.he.net) +3. Reboot into Rescue mode. In Lish: + + passwd # set interactively + sed -i -e 's/#PasswordAuthentication yes/PasswordAuthentication yes/g' /etc/ssh/sshd_config + echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config + service ssh start +4. From the backup directory: + gunzip -c bbb-img-2024-12-08.img.gz | ssh root@bbb.emacsconf.org “dd of=/dev/sda bs=4M status=progress oflag=direct iflag=fullblock” +5. After I resized to 4core 8GB RAM, I couldn’t log in with SSH, so I used the Linode shell. `sshd -t` said no host keys available. Fix: + + ssh-keygen -A + service ssh --full-restart +6. ./bbb-install.sh -v jammy-300 -s bbb.emacsverse.org -e emacsconf@sachachua.com + +Resources: + +- <https://overto.eu/posts/gunzip-into-dd/> +- <https://www.linode.com/community/questions/20386/how-do-i-ssh-to-linode-from-the-finnix-boot-in-order-to-copy-my-local-vm-over-to> +- <https://github.com/bigbluebutton/bigbluebutton/issues/9485> + + +<a id="general-infrastructure-manual-irc-announcements"></a> ## Manual IRC announcements @@ -672,21 +1227,58 @@ Success: - [X] You can /opall, /conftopic, and /broadcast -<a id="org2b9c281"></a> +<a id="general-infrastructure-low-res-stream"></a> ## Low-res stream - -<a id="org00999f1"></a> +Should be automatic, handled by Icecast +backup on live: + +- /usr/local/bin/emacsconf-lowres-gen-on-connect +- /usr/local/bin/emacsconf-lowres-dev-on-connect + + +<a id="general-infrastructure-upload-talks-to-youtube"></a> + +## Upload talks to YouTube + +[Channel videos](https://studio.youtube.com/channel/UCwuyodzTl_KdEKNuJmeo99A/videos/upload?filter=%5B%5D&sort=%7B%22columnType%22%3A%22date%22%2C%22sortOrder%22%3A%22DESCENDING%22%7D) + + (seq-keep + (lambda (o) + (when (and (string= (plist-get o :status) "TO_STREAM") + (not (plist-get o :youtube))) + (list + (plist-get o :slug) + (org-link-make-string + (concat "copy:" + ) + "video file") + (org-link-make-string + (format "elisp:(emacsconf-publish-video-description \"%s\" t)" + (plist-get o :slug)) + "desc") + (if (emacsconf-talk-file o "--main.vtt") + (org-link-make-string + (concat "copy:" + (emacsconf-talk-file o "--main.vtt")) + "sub file") + "") + (org-link-make-string + (concat "elisp:(emacsconf-set-property-from-slug \"" + (plist-get o :slug) + "\" \"YOUTUBE\" (read-string \"YouTube URL: \"))" + ) + "save") + (plist-get o :scheduled)))) + (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))) + + +<a id="general-infrastructure-restream-to-youtube"></a> ## Restream to YouTube -<a id="org1d92e71"></a> - -## Restream to Toobnix - - <a id="other"></a> # Other process notes @@ -700,16 +1292,52 @@ Success: docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"] +<a id="other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted"></a> + +## When a talk is added after the schedule has already been drafted + +- [X] Add the talk to conf.org +- [X] Add the talk ID to the draft schedule +- [X] Run the draft schedule block +- [X] <elisp:emacsconf-schedule-update-from-info> +- [X] <elisp:emacsconf-publish-talks-json-to-files> +- [X] Generate pads with `ansible-playbook -i inventory.yml prod-playbook.yml --tags pad` +- [X] <elisp:emacsconf-set-file-prefixes> +- [X] <elisp:emacsconf-update-schedule> +- [X] <elisp:emacsconf-stream-generate-overlays> +- [X] <elisp:emacsconf-stream-generate-test-videos> +- [X] `rsync -avze ssh ~/proj/emacsconf/2024/assets/ res:/data/emacsconf/shared/2024/assets/` +- [X] Send the upload instructions +- [X] Generate the intro text +- [ ] Record the intro + + <a id="cancel-talk"></a> ## When a talk is cancelled -1. Update the status to CANCELLED. +1. Update the status to CANCELLED. Update that specific talk with <elisp:emacsconf-publish-info-pages-for-talk>. 2. Rejig the schedule if needed, and update with <elisp:emacsconf-schedule-update-from-info> -3. Update the published schedule with <elisp:emacsconf-update-schedule> +3. Update the published schedule with <elisp:emacsconf-update-schedule> and push the wiki. 4. Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file. -5. Sync the new intros to the /data/emacsconf/shared/$YEAR/assets/intros directory. -6. Regenerate the pads: <elisp:emacsconf-pad-prepopulate-all-talks> +5. Resync the cache directory. +6. Consider updating the screenshots for sat-open and sun-open. + + +<a id="other-process-notes-when-a-talk-changes-title"></a> + +## When a talk changes title + +1. Update the title in conf.org. +2. Delete the FILE\_PREFIX property. +3. Call <elisp:emacsconf-set-file-prefix-if-needed>. +4. Rename existing files with <elisp:emacsconf-update-file-prefixes>. +5. Update the intro VTT file with the new title. +6. Update the wiki page for the talk. +7. Regenerate the in-between images with <elisp:emacsconf-stream-generate-in-between-pages> +8. Rerecord and recompile the intro, and recompile the intro for the talk after it. +9. <elisp:emacsconf-update-schedule> +10. <elisp:emacsconf-pad-prepopulate-hyperlists> <a id="change-talk-qa"></a> @@ -724,7 +1352,45 @@ Success: 6. Regenerate talks.json: <elisp:emacsconf-publish-talks-json-to-files> -<a id="orge2b5368"></a> +<a id="other-process-notes-to-play-the-other-stream"></a> + +## To play the other stream + +- `mpv --profile=full https://live0.emacsconf.org/gen.webm` +- Change the channel topic to note that discussion can be in the other channel + + +<a id="other-process-notes-when-a-system-is-down"></a> + +## When a system is down + +Let people know in #emacsconf and #emacsconf-org + +Update the status page: + +<file://ssh:orga@front0.emacsconf.org:/var/www/status.emacsconf.org/index.html> + +/broadcast <message> may also be helpful + + +<a id="other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer"></a> + +## When we need to bring on a last-minute volunteer + +/backstage nick + +will give them the backstage credentials and links to the index + + +<a id="other-process-notes-checking-people-in"></a> + +## Checking people in + +/checkin nick +/room nick (should be automatic if live, but there just in case) + + +<a id="other-process-notes-ansible"></a> ## Ansible |