diff options
Diffstat (limited to 'organizers-notebook.md')
| -rw-r--r-- | organizers-notebook.md | 639 |
1 files changed, 506 insertions, 133 deletions
diff --git a/organizers-notebook.md b/organizers-notebook.md index 7a663664..3cdcda26 100644 --- a/organizers-notebook.md +++ b/organizers-notebook.md @@ -22,40 +22,48 @@ This document is the general organizers' notebook that can be used as a starting - [Draft CFP](#phases-and-lessons-learned-draft-cfp) - [Announce](#phases-and-lessons-learned-announce) - [Process proposals](#phases-and-lessons-learned-process-proposals) - - [Accept proposals](#phases-and-lessons-learned-accept-proposals) + - [Accept proposals](#phases-and-lessons-learned-accept-proposals):email: - [Draft schedule](#draft-schedule) - [Set up backstage and upload](#phases-and-lessons-learned-set-up-backstage-and-upload) - [Post the schedule](#phases-and-lessons-learned-post-the-schedule) - [Process uploaded files](#phases-and-lessons-learned-process-uploaded-files) - [Generate assets](#phases-and-lessons-learned-generate-assets) + - [Record pronunciations](#phases-and-lessons-learned-record-pronunciations) - [Prepare the infrastructure](#phases-and-lessons-learned-prepare-the-infrastructure) - [Getting ready for captions](#phases-and-lessons-learned-getting-ready-for-captions) - [Getting ready for the conference](#pre-conf) - [During the conference](#conf) - [After the conference](#phases-and-lessons-learned-after-the-conference) +- [Ideas for improvement](#ideas) - [General infrastructure](#general-infrastructure) - [Linode instance sizes](#general-infrastructure-linode-instance-sizes) - [File uploads](#upload) + - [Media processing](#general-infrastructure-media-processing) + - [Wiki](#org7f83d48) - [Icecast](#icecast) - [Watching pages](#watch) - [VNC](#vnc) - [Streaming with OBS](#obs) + - [Mumble](#org09cd092) - [Shell scripts](#general-infrastructure-shell-scripts) - [Backstage](#backstage) - - [IRC web client](#general-infrastructure-irc-web-client) + - [Thelounge IRC web client](#general-infrastructure-irc-web-client) - [Etherpad](#general-infrastructure-etherpad) - [Publishing media to the server and to the wiki](#media) + - [Automated publishing to the media server](#orgc40bf50) - [Automated IRC announcements](#general-infrastructure-automated-irc-announcements) - [BigBlueButton](#bbb) - [Manual IRC announcements](#general-infrastructure-manual-irc-announcements) + - [Check test crontab](#orgc7d6d2a) + - [Confirm automated IRC announcements from res](#general-infrastructure-confirm-automated-irc-announcements-from-res) - [Low-res stream](#general-infrastructure-low-res-stream) - [Upload talks to YouTube](#general-infrastructure-upload-talks-to-youtube) - [Restream to YouTube](#general-infrastructure-restream-to-youtube) + - [Restream to Toobnix](#orgdc911e7) + - [i3 window manager](#org692ea98) - [Other process notes](#other) - [Adding another user to BBB](#bbb-user) - - [When a talk is added after the schedule has already been drafted](#other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted) - - [When a talk is cancelled](#cancel-talk) - - [When a talk changes title](#other-process-notes-when-a-talk-changes-title) + - [Schedule changes](#org5f8f6b0) - [When a talk Q&A changes method](#change-talk-qa) - [To play the other stream](#other-process-notes-to-play-the-other-stream) - [When a system is down](#other-process-notes-when-a-system-is-down) @@ -179,6 +187,21 @@ Replace { and } with [ and ]. [2023](file:///home/sacha/proj/emacsconf/wiki/2023/cfp.md) +### Update dates + +<file:///home/sacha/proj/emacsconf/lisp/emacsconf.el> +<file:///home/sacha/proj/emacsconf/lisp/emacsconf-erc.el> + + +### Update the IRC topic + + (erc-cmd-OPALL) + (erc-cmd-CONFTOPIC (emacsconf-replace-plist-in-string + `(:year ,emacsconf-year :cfp-deadline ,emacsconf-cfp-deadline) + "Getting ready for EmacsConf ${year}! Call for proposals: https://emacsconf.org/${year}/cfp by ${cfp-deadline}, volunteer: https://emacsconf.org/volunteer/")) + (erc-cmd-DEOPALL) + + ### Lessons learned - Maybe incentivize proper timezone specification by saying we can translate times to their local time? @@ -273,7 +296,7 @@ When proposals come in: <a id="phases-and-lessons-learned-accept-proposals"></a> -## Accept proposals +## Accept proposals :email: Setup: @@ -287,6 +310,18 @@ For each talk: 4. Use `emacsconf-publish-info-pages` and `emacsconf-publish-schedule` as needed. 5. Send acceptance e-mail with `emacsconf-mail-accept-talk` +After the CFP ends: + +1. Change the yyyy.md page to remove the CFP and submit links. +2. Update the yyyy/sidebar.md page to remove the CFP and submit links. +3. Update the yyyy/talks.md page to remove the CFP. +4. Change the IRC channel topics. + 1. `/opall`, + 2. ex: `/conftopic Getting ready for EmacsConf 2025! Volunteer: https://emacsconf.org/volunteer` + 3. `/deopall` + +[Example git commit](https://git.emacsconf.org/emacsconf-wiki/commit/?id=9b2c5a6f86d9e4c3faf4c7f028eabb758583fa97) + <a id="draft-schedule"></a> @@ -304,6 +339,14 @@ For each talk: - Add missing items to plan +### TODO Send draft schedule to speakers :email: + +1. Evaluate the draft schedule we want to use. +2. `emacsconf-schedule-update-from-info` +3. `C-u M-x emacsconf-mail-template-to-all-groups RET emacsconf-mail-draft-schedule RET` +4. If that looks good, `M-x emacsconf-mail-template-to-all-groups`. You can use `M-x emacsconf-mail-merge-send-all` to send all the drafts, or send them one by one. + + <a id="phases-and-lessons-learned-set-up-backstage-and-upload"></a> ## Set up backstage and upload @@ -315,6 +358,9 @@ Lessons learned: - Next year, let’s get upload.emacsconf.org in place as the domain name so that people don’t try to FTP files to it. +### DONE Send backstage and upload instructions :email: + + <a id="phases-and-lessons-learned-post-the-schedule"></a> ## Post the schedule @@ -341,6 +387,7 @@ Process: 1. Review uploaded files with <elisp:emacsconf-upload-dired> 2. Open the JSON for the file you want to process and use <elisp:emacsconf-upload-copy-from-json> . If more than one file uses the same extension, provide a suffix to disambiguate them. + If emacsconf-upload-copy-from-json might take a long time, you can also scp the file to your local 3. On res: 1. Create a directory ~/current/files/$slug. 2. Copy the uploaded file from ~/cache as `$video_slug--original.$extension`, or use `rename-original.sh $slug $file`. @@ -353,7 +400,7 @@ Process: <a id="phases-and-lessons-learned-generate-assets"></a> -## Generate assets +## TODO Generate assets ### In-between pages @@ -369,6 +416,31 @@ Process: - Splice together into a file that will be processed with subed-record-compile-video. +### TODO Test videos + +<elisp:emacsconf-stream-generate-test-videos> + + +<a id="phases-and-lessons-learned-record-pronunciations"></a> + +## Record pronunciations + + +### DONE Record pronunciations + +1. mkdir -p ~/proj/emacsconf/2025/assets/intros ~/proj/emacsconf/2025/assets/in-between +2. cp ~/proj/emacsconf/2024/assets/in-between/template.svg ~/proj/emacsconf/2025/assets/in-between/template.svg +3. <elisp:emacsconf-stream-generate-in-between-pages> +4. Create ~/proj/emacsconf/2025/assets/intros/intro.vtt +5. <elisp:emacsconf-subed-intro-subtitles> +6. Record the audio, possibly with subed-record, left, and right. +7. Split the lines for nicer subtitles. +8. Use <elisp:subed-record-compile> + + +### DONE E-mail speakers asking them to confirm the pronunciations + + <a id="phases-and-lessons-learned-prepare-the-infrastructure"></a> ## Prepare the infrastructure @@ -409,20 +481,78 @@ Onboarding: <elisp:emacsconf-pad-prepopulate-shift-hyperlists> -### TODO Send check-in details, schedule update, etc. +### DONE Check in with speakers for cancellations, adjustments +emacsconf-mail-intro-to-all -### Two days before the conference -Go through the :tminustwo: tags. +### TODO Record opening remarks -### One day before the conference +### TODO Send check-in details, schedule update, etc. :email: -Go through the :tminusone: tags. -- Set `emacsconf-publishing-phase` to `conference`. -- Set emacsconf-publish-include-pads to t and republish the info pages. +### One day before the conference + +- [X] Set `emacsconf-publishing-phase` to `conference`. +- [X] Set emacsconf-publish-include-pads to t. +- [X] Republish the schedule and the info pages. +- [X] Update the conf.org and the wiki on res. (~/emacsconf-2025-private, ~/emacsconf-wiki) +- [X] Resize the BBB server to production levels with bbb-prod: 16 GB $0.1440/hour 6 CPUs +- [X] Send check-in details and last-minute schedule updates. +- [X] Start thelounge on front0: systemctl start thelounge. +- [X] Confirm I can connect to Mumble. +- [X] Resize front0 to production: front0 shared Linode 32GB 8 CPUs 640GB $0.2880/hour +- [X] Resize live0 to production: shared Linode 64GB 16 CPUs $0.5760/hour +- [X] Start the VNC servers. + + #!/bin/bash + # SSH to both + cd ~/proj/emacsconf/emacsconf-ansible + + # Gen + export TRACK=gen; TRACK_PORT=5905; export SSH_PORT=46668 + ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc + sleep 5 + ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT & + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & + + # Dev + export TRACK=dev; export TRACK_PORT=5906; export SSH_PORT=46668 + ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc + sleep 5 + ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT & + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & +- [X] Confirm that we can use “Start Recording” to stream to gen. + - [X] play-with-intro sat-open + - [X] <https://live.emacsconf.org/2025/watch/gen/> + - [X] mpv <https://live0.emacsconf.org/gen.webm> + - [X] play-with-intro sun-open (to check the overlays) + - [X] <https://live.emacsconf.org/2025/watch/gen-480p/> + - [X] mpv <https://live0.emacsconf.org/gen-480p.webm> +- [X] Confirm that we can stream to dev. + - [X] play-with-intro sat-open + - [X] <https://live.emacsconf.org/2025/watch/dev/> + - [X] mpv <https://live0.emacsconf.org/dev.webm> + - [X] play-with-intro sun-open (to check the overlays) + - [X] <https://live.emacsconf.org/2025/watch/dev-480p/> + - [X] mpv <https://live0.emacsconf.org/dev-480p.webm> +- [X] Confirm that I can restream to YouTube. + - Settings - Stream, paste in the RTMP key for the relevant event, then Start Streaming. + - [X] gen + - [X] dev +- [X] Confirm that I can restream to Toobnix + - [X] live0: restream-gen-toobnix + - [X] live0: restream-dev-toobnix +- [X] Check automated announcements and publishing + - [X] orga@res: talk sat-open TO\_STREAM + - [X] <https://emacsconf.org/2025/talks/sat-open/> doesn’t have video + - [X] <https://media.emacsconf.org/2025> doesn’t have it + - orga@res: talk sat-open PLAYING + - [X] <https://emacsconf.org/2025/talks/sat-open/> has video and transcript + - [X] <https://media.emacsconf.org/2025> has talk files + - [X] IRC announcement in #emacsconf-gen +- [X] Confirm icecast stats are available: <http://live0.emacsconf.org/admin/stats> (username: admin, password is in prod-vars.yml) <a id="conf"></a> @@ -448,6 +578,8 @@ dev: 226 peak + 79 peak lowres Sunday: gen: 89 peak + 10 peak lowres +<http://live0.emacsconf.org/admin/stats> (username: admin, password is in prod-vars.yml) + meet peak 409% CPU (100% is 1 CPU), average 69.4% front peak 70.66% CPU (100% is 1 CPU) live peak 552% CPU (100% is 1 CPU) average 144% @@ -473,7 +605,7 @@ rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/ ssh root@66.175.208.243 ’dd if=/dev/sda bs=5M ’ | dd of=bbb-img-2024-12-08.img status=progress The img can be loaded with -sudo losetup –find –show bbb-img-2024-12-08.img +sudo losetup –find –show bbb-img-2024-12-08.img sudo mount /dev/loop10 test cd test @@ -518,7 +650,7 @@ cd test ### Process the Q&A recordings and live presentations -- Save answers videos as –answers.webm. +- Save answers videos as –answers.webm. - From the pad file: emacsconf-extract-copy-pad-to-wiki - update cache - Set `QA_PUBLIC` property to t as each Q&A gets processed. @@ -536,7 +668,7 @@ Also copy the YouTube comments and IRC comments emacsconf-extract-irc-anonymize-log -### Send thank-you emails to the speakers +### Send thank-you emails to the speakers :email: ### Notify emacs-discuss that the Q&A is up @@ -572,7 +704,7 @@ emacsconf-extract-irc-anonymize-log Sample: <https://emacsconf.org/2023/report/> -### Send thank-you emails to the volunteers +### Send thank-you emails to the volunteers :email: emacsconf-mail-template-ask-volunteer-for-mailing-address @@ -580,6 +712,94 @@ emacsconf-mail-template-ask-volunteer-for-mailing-address ### Process the lessons learned +<a id="ideas"></a> + +# Ideas for improvement + +How can we make EmacsConf better for participants? Speakers? Volunteers? People who come across the videos afterwards? + +- Emacs: + - view Etherpad (read-only): might be able to query something like <https://pad.emacsconf.org/2024/export/html> or <https://pad.emacsconf.org/2024/export/txt> + - view upcoming talks, countdown? + - resources, past talks + - ??: Collaborative real-time editing? Not sure crdt.el or collab-mode can scale up as far as we like; how would we test this? +- HTML/CSS/JS: + - general design improvements? + - [This is what the talks page looks like when there’s no schedule yet](https://web.archive.org/web/20240806134541/https://emacsconf.org/2024/talks/) + - [This is what the schedule page looks like before the conference](https://web.archive.org/web/20241108132608/https://emacsconf.org/2024/talks/) + - [This is what a talk page looks like before the conference](https://web.archive.org/web/20241208042046/https://emacsconf.org/2024/talks/literate/) + - [This is the watch page for the general track](https://web.archive.org/web/20241208034535/https://emacsconf.org/2024/watch/gen/) + - [This is the same talk page after the conference, with the videos, notes, and captions](https://web.archive.org/web/20241212145433/https://emacsconf.org/2024/talks/literate) + - [This is what the talks page looks like after the conference](https://emacsconf.org/2024/talks/) - list of resources + - timezone translations in the schedule SVG, on the schedule page, etc. + - countdown on talk pages? + - prettier viewers for videos, captions + - make it easier to see the talk/pad/IRC at the same time? + - easy timestamp jumping +- Volunteers: + - better documentation + - share to OBS from local? Maybe vdo.ninja? + - how do we manage the fallback/switchover in case we need to go back to shared VNC? + - meeting layout? + - countdown to switch over to the next talk, or easier manual control; web or command-line interface to control it? + - make it easier for people to join as hosts or check-in volunteers? + - improve backstage experience? + - automate or cross-train more people in what Sacha does (coordination, infrastructure) to reduce bus factor and increase awesomeness +- Processing: + - audio normalization: better workflow, more people helping + - noise removal (many speakers don’t have external mics) + - Makefile improvements + - captioning: + - simplify common edits + - reflowing? check out [sub-seg](https://gitlab.com/jun8git/sub-seg)? + - handle last-minute uploads + - validation: make sure audio, video, and caption times agree +- Infrastructure: + - document media processing pipeline + - make media processing pipeline more hands-off; ideally, just copy the video to the directory and that kicks off everything including eventually republishing the backstage index + - add read-only non-JS links to pads; make the pads viewable through the web instead of being downloaded (…/export/html) + - upgrade to latest versions + - front0 and live0: currently Debian 10.13 + - <span class="timestamp-wrapper"><time class="timestamp" datetime="2025-09-21">[2025-09-21 Sun]</time></span> bandali will look into upgrades, cleaning up + - <span class="timestamp-wrapper"><time class="timestamp" datetime="2025-09-19">[2025-09-19 Fri]</time></span> sachac: e-mailed bandali to ask if I can do the upgrades or if he wants to + - ffmpeg 6.0.1, latest is 8 + - 7 has demuxing, decoding, filtering, encoding, and muxing in the ffmpeg CLI now all run in parallel + - OBS on res? Currently 29.1.2 + - probably no must-haves up to version 32, okay to stay with current version + - Do we want to go back to having hosts stream from their local computers? + - local: better performance + - via VNC: + - easier to switch in a different host or manage multiple streams + - easier to move files around + - consistent setup for profiles, overlays + - DONE Etherpad (upgrade 1.9.7 to 2.5.0) + - DONE BigBlueButton (deployed 3.0.12, which is the latest as of 2025-08-02) + - Consider 3.1 beta? new interface, probably new bugs; see if it’s worth it? + - DONE Icecast (deployed 2.4.4-1) - decided to stick with current version + - <span class="timestamp-wrapper"><time class="timestamp" datetime="2025-09-21">[2025-09-21 Sun]</time></span> Decision: stick with Icecast + - Leaning towards sticking with [Icecast](https://icecast.org/) + - 2.5.0-rc1? health reporting, bugfixes; probably not essential + - [Ant Media Server](https://antmedia.io/) community edition, to enable adaptive bitrate streaming, lower latency, and HLS for wider support? + - [Owncast](https://owncast.online/)? Automatic downscaling, Fediverse support; might need two instances to support two tracks + - [SRS](https://github.com/ossrs/srs)? record, replay? more of a building block? + - Nginx RTMP? More of a building block + - customize BigBlueButton welcome, slides, title, branding + - change the default presentation (maybe someone can make a PDF?) <https://docs.bigbluebutton.org/administration/customize/#change-the-default-presentation> + - modify the default landing page <https://docs.bigbluebutton.org/administration/customize/#modify-the-default-landing-page> + - change the default welcome messages <https://docs.bigbluebutton.org/administration/customize/#change-the-default-welcome-message> + - change the favicon <https://docs.bigbluebutton.org/administration/customize/#change-favicon> + - change the default logo <https://docs.bigbluebutton.org/administration/customize/#changing-the-default-logo> + - check out BigBlueButton live captions? + - figure out if we can connect to or reuse the etherpad in BigBlueButton? + - figure out how to record separate audio streams in BigBlueButton because the audio mixing is sometimes terrible (some people’s audio streams end up much louder than others). Related: <https://github.com/bigbluebutton/bigbluebutton/issues/12302> , but not sure how to actually do it… + - replicate BBB setup in a virtual machine so that it’s easy to re-deploy + - replicate streaming setup (OBS+VNC) in a virtual machine so that it’s easy to re-deploy + - replicate video processing workflow on my computer and/or a virtual machine +- Other: + - panel? some thoughts: <https://pad.emacsconf.org/emacsconf-panel> + - unconference? + + <a id="general-infrastructure"></a> # General infrastructure @@ -615,7 +835,7 @@ Resize disk to 25GB, copying large files to media if needed Before the conference: 1. Set `upload_enabled` to `true` in `prod-vars.yml` and run `ansible-playbook -i inventory.yml prod-playbook.yml --tags upload`. -2. Confirm that <https://upload.emacsconf.org> has the upload interface and works with the password. +2. Confirm that <https://upload.emacsconf.org> has the upload interface and works with the password. If it’s not up yet, use `sudo service upload start`. 3. Use `emacsconf-mail-template-to-all-groups` and `emacsconf-mail-upload-and-backstage-info` to e-mail the upload information to all speakers. After the conference: @@ -624,17 +844,65 @@ After the conference: 2. Confirm that <https://upload.emacsconf.org> shows the in-between page. +<a id="general-infrastructure-media-processing"></a> + +## Media processing + +1. Make sure `group_vars/all.yml` has the correct setting for `emacsconf_year`. +2. `ansible-playbook -i inventory.yml prod-playbook.yml --tags prerec` +3. Set up the `update-cache` script. + + #!/bin/bash + # export CONF_PUBLIC=1 + rsync -avu ~/proj/emacsconf/2025/assets/intros/*--intro.* ~/proj/emacsconf/2025/cache/ + rsync -avzue ssh --exclude ffmpeg2pass* --exclude .lock-\* --exclude 'partial--*' res:~/cache/ ~/proj/emacsconf/2025/cache/ --exclude .git + chmod ugo+r . -R + rsync -avzue ssh --exclude ffmpeg2pass* --exclude 'partial--*' ~/proj/emacsconf/2025/cache/ res:~/cache/ $* --exclude .git + rsync -avzue ssh --exclude ffmpeg2pass* --exclude 'partial--*' ~/proj/emacsconf/2025/cache/ orga@media:/var/www/media.emacsconf.org/2025/backstage/ $* --exclude .git + ssh orga@media "cd ~/backstage; chmod ugo+r -- *" + if [ ! -z "$CONF_PUBLIC" ]; then + echo "Updating public media" + rsync -avzue ssh --exclude \*--original\* --exclude ffmpeg2pass* --exclude \*--reencoded\* ~/proj/emacsconf/2025/cache/ orga@media:/var/www/media.emacsconf.org/2025/ 3--exclude .git + fi + rsync -avzue ssh ~/proj/emacsconf/2025/assets/ res:~/current/assets/ + +When a talk comes in: + +1. Update the talk status to `PROCESSING`. Re-evaluate the schedule draft. +2. Set the file prefix with `emacsconf-set-file-prefix-if-needed` or use `emacsconf-set-file-prefixes`. +3. Download the talk’s files to your local cache (as specified by `emacsconf-cache-dir`). +4. Rename the files with `emacsconf-rename-files`. +5. Use `../update-cache` to update res and backstage. +6. SSH to res. Go to the `~/cache` directory. + 1. Start or reuse a `screen` session. + 2. Use `make` to convert the file to WEBM and caption the file. +7. On your local computer, use `../update-cache` to move files around. +8. `emacsconf-publish-cache-video-data` to save the file size and length. +9. `emacsconf-publish-backstage-index` + + +<a id="org7f83d48"></a> + +## Wiki + +Stylesheet: <file://ssh:front|sudo:ikiwiki@front:/var/www/emacsconf.org/style.css> + +See also Ansible notes + + <a id="icecast"></a> -## Icecast +## DONE Icecast -Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv. +Success: You can send a stream to Icecast and the results can be viewed by mpv. - [X] Gen - [X] Dev Also, recordings are available in /data. +See also: [Use OBS+VNC to record](#obs-instructions) + New year: 1. Reprovision with @@ -647,8 +915,8 @@ New year: 3. To test, you can use FFmpeg like this for the gen stream: ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \ - -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ - -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/gen.webm" + -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ + -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/gen.webm" For dev: @@ -659,11 +927,18 @@ New year: and then view it with mpv https://live0.emacsconf.org/gen.webm + mpv https://live0.emacsconf.org/dev.webm + + +### Cooldown + +ssh live0.emacsconf.org +sudo /etc/init.d/emacsconf stop <a id="watch"></a> -## Watching pages +## DONE Watching pages Success: You can watch at <https://live.emacsconf.org>. @@ -674,7 +949,7 @@ Success: You can watch at <https://live.emacsconf.org>. New year: -1. ansible-playbook -i inventory.yml prod-playbook.yml –tags live +1. `ansible-playbook -i inventory.yml prod-playbook.yml --tags live` 2. Republish with <elisp:emacsconf-publish-watch-pages>. 3. Update front:/var/www/live.emacsconf.org. @@ -683,7 +958,7 @@ Set up test streams (see ffmpeg instructions from [Icecast](#icecast)) and then: <a id="vnc"></a> -## VNC +## DONE VNC We use VNC to connect to the X servers on res.emacsconf.org so that we can stream from it. @@ -692,10 +967,6 @@ Success: - [X] Confirm that you can connect to emacsconf-gen via VNC - [X] Confirm that you can connect to emacsconf-dev via VNC -Setting up - -- <elisp:emacsconf-publish-res-index> - <a id="vnc-instructions"></a> @@ -738,21 +1009,29 @@ and then connect with: xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & +### Cooldown + +ssh sachac@res +sudo killall Xtigervnc + + <a id="obs"></a> -## Streaming with OBS +## DONE Streaming with OBS -Success: Confirm that you can stream +Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv. - [X] gen - [X] dev New year: reprovision with - ansible-playbook -i inventory.yml prod-playbook.yml --tags obs + ansible-playbook -i inventory.yml prod-playbook.yml --tags obs,stream so that the year is updated in the shell scripts. +icecast is on live0.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`. + <a id="obs-instructions"></a> @@ -766,21 +1045,31 @@ so that the year is updated in the shell scripts. mpv https://live0.emacsconf.org/$TRACK.webm & +so that the year is updated in the configuration. -### Icecast -Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv. +<a id="org09cd092"></a> -- [ ] gen -- [ ] dev +## DONE Mumble -New year: reprovision with +Success: - ansible-playbook -i inventory.yml prod-playbook.yml --tags stream +- [X] You can connect to Mumble on your preferred device(s). +- [X] You can broadcast to the emacsconf-gen channel. +- [X] You can broadcast to the emacsconf-dev channel. -so that the year is updated in the configuration. +mumble.emacsconf.org is hosted on front. Start it with: -This is on live.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`. + sudo /etc/init.d/mumble-server start + +It listens on port 64738 + +Logs are in `/var/log/mumble-server`. + +To add a new account: + +- After they log in, register their user. +- Right-click on the server name. Edit - Groups. Add them to the org group. <a id="general-infrastructure-shell-scripts"></a> @@ -828,67 +1117,67 @@ Explanation of files: </colgroup> <tbody> <tr> -<td class="org-left">–original.mp4/mov/webm</td> +<td class="org-left"><code>--original.mp4/mov/webm</code></td> <td class="org-left">original file as uploaded by the speaker</td> </tr> <tr> -<td class="org-left">–silence.mp4/mov/webm</td> +<td class="org-left"><code>--silence.mp4/mov/webm</code></td> <td class="org-left">silence sample if uploaded by speaker</td> </tr> <tr> -<td class="org-left">–reencoded.webm</td> +<td class="org-left"><code>--reencoded.webm</code></td> <td class="org-left">converted to .webm format and compressed</td> </tr> <tr> -<td class="org-left">–reencoded.opus</td> +<td class="org-left"><code>--reencoded.opus</code></td> <td class="org-left">audio only</td> </tr> <tr> -<td class="org-left">–normalized.opus</td> +<td class="org-left"><code>--normalized.opus</code></td> <td class="org-left">normalized audio</td> </tr> <tr> -<td class="org-left">–reencoded.vtt</td> +<td class="org-left"><code>--reencoded.vtt</code></td> <td class="org-left">WhisperX captions, WebVTT format</td> </tr> <tr> -<td class="org-left">–reencoded.json</td> +<td class="org-left"><code>--reencoded.json</code></td> <td class="org-left">WhisperX word data</td> </tr> <tr> -<td class="org-left">–reencoded.srt</td> +<td class="org-left"><code>--reencoded.srt</code></td> <td class="org-left">WhisperX captions, SRT format</td> </tr> <tr> -<td class="org-left">–reencoded.txt</td> +<td class="org-left"><code>--reencoded.txt</code></td> <td class="org-left">WhisperX captions, plain text</td> </tr> <tr> -<td class="org-left">–main.vtt</td> +<td class="org-left"><code>--main.vtt</code></td> <td class="org-left">edited captions</td> </tr> <tr> -<td class="org-left">–main–chapters.vtt</td> +<td class="org-left"><code>--main--chapters.vtt</code></td> <td class="org-left">chapter markers</td> </tr> <tr> -<td class="org-left">–main.txt</td> +<td class="org-left"><code>--main.txt</code></td> <td class="org-left">captions as text (optional)</td> </tr> <tr> -<td class="org-left">–main.webm</td> +<td class="org-left"><code>--main.webm</code></td> <td class="org-left">version for streaming based on the most recent resources</td> </tr> </tbody> @@ -899,24 +1188,29 @@ Other files might also have been uploaded by the speaker, such as slides or note <a id="general-infrastructure-irc-web-client"></a> -## IRC web client +## Thelounge IRC web client Success: +- [X] When the service is not running, the error page is displayed - [X] You can join from <https://chat.emacsconf.org> - [X] You can join from <https://live.emacsconf.org> and the channels end up as the latest ones +ansible: lounge tag + +On front: + +- To start: sudo systemctl start thelounge +- To stop: sudo systemctl stop thelounge + + +### DONE Ask libera.chat to increase IRC limits + Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors. Message I’ve posted to support@libera.chat: -Thank you for running libera.chat. We’re looking forward to using IRC -(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming -EmacsConf (Dec 7-8, <https://emacsconf.org>). In the past, we’ve needed to -coordinate with Libera staff to increase the number of connections -allowed from chat.emacsconf.org during conference days in order to avoid -connection exhaustion errors. Is that something we can plan now or would -you like me to bring it up a few days before the conference? +Thank you for running libera.chat. We’re looking forward to using IRC (#emacsconf, #emacsconf-gen, #emacsconf-dev, #emacsconf-org) for the discussions for the upcoming EmacsConf (Dec 6-7, <https://emacsconf.org>). In the past, we’ve needed to coordinate with Libera staff to increase the number of connections allowed from chat.emacsconf.org during conference days in order to avoid connection exhaustion errors. Is that something we can plan now or would you like me to bring it up a few days before the conference? <a id="general-infrastructure-etherpad"></a> @@ -932,6 +1226,13 @@ Lessons learned: - After 2024: Added pronouns and pronunciation to the pad template since that’s what the hosts will be looking at +The pad is at front0.emacsconf.org + +Non-JS way to get plain text or HTML of a pad: ex: + +- <https://pad.emacsconf.org/orgmeetup/export/txt> +- <https://pad.emacsconf.org/orgmeetup/export/html> + <a id="media"></a> @@ -943,14 +1244,13 @@ Start of year: 2. `ansible-playbook -i inventory.yml prod-playbook.yml --tags media` 3. Generate the index with `emacsconf-publish-update-media` -Confirm by setting a submitted talk to `PLAYING` and testing with -<elisp:emacsconf-publish-media-files-on-change> . The public media +Confirm by using `talk sat-open PLAYING` on res. The public media directory should have the files and the entry should be in the index. Switching it back to `TO_STREAM` and calling <elisp:emacsconf-publish-media-files-on-change> should remove it. -### TODO Switch public media to unprotected root before the conference :tminustwo: +### DONE Switch public media to unprotected root before the conference :tminustwo: 1. Clear public media directory. 2. Set `media_protect_root` to false in Ansible `group_vars/all.yml`. @@ -959,14 +1259,31 @@ Switching it back to `TO_STREAM` and calling You can generate the index with `emacsconf-publish-update-media`. +<a id="orgc40bf50"></a> + +## Automated publishing to the media server + +`ansible-playbook -i inventory.yml prod-playbook.yml --tags publish,edit` + +On orga@res, in the screen session with the emacs server, make sure emacsconf-year and emacsconf-cache-dir refer to the current ones. + + <a id="general-infrastructure-automated-irc-announcements"></a> -## Automated IRC announcements +## DONE Automated IRC announcements Success: - [ ] When a talk starts playing, it is announced in the relevant channel +How it works: On res: + +/usr/local/bin/handle-session sudos to the orga user and calls the `/usr/local/bin/talk` shell script, + +You will need the private repository checked out to `orga/emacsconf-2025-private`, which is set up by the ansible playbook `publish` tag. + +This is by `/usr/local/bin/talk` on res, which connects to a running Emacs with emacsclient. Make sure the emacs thing has the right year. + <a id="bbb"></a> @@ -988,7 +1305,10 @@ Sizes: - dormant: 1 GB nanode - testing: 4 core 8 GB shared CPU - production: 8 core 16 GB dedicated CPU (roughly half CPU load for 107 simultaneous users, 2024) + - maybe 8 core 16 GB shared CPU next? + +<a id="general-infrastructure-bigbluebutton-installing-bigbluebutton"></a> ### Installing BigBlueButton @@ -1027,68 +1347,39 @@ BBB says sharing system audio works only if you use Chrome on Mac OS X or Micros Related: <https://github.com/bigbluebutton/bigbluebutton/issues/8632> -### Creating talk BBB rooms +### DONE Create meeting rooms for each speaker -- `docker exec -it greenlight-v3 /bin/bash` -- `bundle exec rails console` -- `user_id = User.find_by_email("sacha@sachachua.com").id` + docker exec -it greenlight-v3 /bin/bash -c "bundle exec rails console" - (mapconcat (lambda (group) - (format - "Room.create(user_id: user_id, name: \"%s - %s\")\n" - (plist-get (cadr group) :speakers) - (string-join (mapcar (lambda (talk) (plist-get talk :slug)) - (cdr group))))) - (emacsconf-mail-groups (emacsconf-active-talks (emacsconf-get-talk-info))) - "") + user_id = User.find_by_email("emacsconf@sachachua.com").id -- <elisp:emacsconf-publish-bbb-static-redirects> +1. <elisp:emacsconf-bbb-create-rooms> +2. <elisp:emacsconf-bbb-load-rooms>: set the ROOM properties. +3. <elisp:emacsconf-publish-bbb-static-redirects> -Print out the room IDs with +Print out the room IDs with: Room.all.each { |x| puts x.friendly_id + " " + x.name }; nil It’s possible to change the friendly\_id and then use `x.save!`. -### Setting up moderator access codes - - (dolist (talk (seq-filter (lambda (o) - (and (plist-get o :bbb-room) - (not (plist-get o :bbb-mod-code)))) - (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) - (spookfox-js-injection-eval-in-active-tab - (format "window.location.href = \"%s\"" - (replace-regexp-in-string "/join" "" (plist-get talk :bbb-room))) - t) - (sleep-for 3) - (spookfox-js-injection-eval-in-active-tab - "document.querySelector('button[data-rr-ui-event-key=\"settings\"]').click()" t) - (spookfox-js-injection-eval-in-active-tab - "document.querySelector('input#glAnyoneCanStart').checked = true") - (spookfox-js-injection-eval-in-active-tab - "document.querySelector('input#muteOnStart').checked = true") - (spookfox-js-injection-eval-in-active-tab - "document.querySelectorAll('.border-end button')[2].click()" t) - (let ((code (spookfox-js-injection-eval-in-active-tab - "document.querySelector('.access-code-input input').value" t))) - (message "Setting %s to %s" (plist-get talk :slug) code) - (emacsconf-set-property-from-slug - talk "BBB_MOD_CODE" - code) - (sit-for 2))) - - (dolist (talk (seq-filter (lambda (o) - (plist-get o :bbb-room)) - (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info)))) - (spookfox-js-injection-eval-in-active-tab - (format "window.location.href = \"%s\"" - (replace-regexp-in-string "/join" "" (plist-get talk :bbb-room))) - t) - (sleep-for 3) - (spookfox-js-injection-eval-in-active-tab - "document.querySelector('button[data-rr-ui-event-key=\"settings\"]').click()" t) - (sleep-for 3)) +### DONE Set up moderator access codes + +- <elisp:emacsconf-bbb-spookfox-set-moderator-codes> +- <elisp:emacsconf-bbb-spookfox-confirm-settings> + +Then you can use <elisp:emacsconf-mail-checkin-instructions-for-attending-speakers> +or <elisp:emacsconf-mail-checkin-instructions-to-all> to send the info. + + +### Processing BBB videos + +`bbb-record --list` shows all the recordings. You can use `bbb-record --rebuild` to regenerate the meeting, and `bbb-record --watch` to monitor progress. Presentation is a web-based interface, video is a single video file that contains the webcam, the screenshare, and the audio. + +If you don’t see the video format, doublecheck that [the video format is enabled](#general-infrastructure-bigbluebutton-installing-bigbluebutton). + +bbb-record seems to call `ruby /usr/local/bigbluebutton/core/scripts/process/video.rb -m ID`, which might be a useful backup. ### Backing up BBB :backup: @@ -1100,11 +1391,17 @@ ssh root@bbb ’tar zcvf - /var/bigbluebutton /etc/bigbluebutton /root/green ssh root@bbb ’dd if=/dev/sda bs=5M ’ | dd of=bbb-img-$(date “+%Y-%m-%d”).img status=progress -### Spinning BBB up again +### Scaling BBB up and down -<span class="timestamp-wrapper"><span class="timestamp">[2025-01-22 Wed] </span></span> … actually, this ran into some issues with the certificate, so I’m just going to do a reinstall. +[Scaling a BigBlueButton server down to a 1 GB node between uses](https://sachachua.com/blog/2025/01/scaling-a-bigbluebutton-server-down-to-a-1-gb-node-between-uses/) -Previous notes +When the server is downscaled, `/etc/nginx/sites-backup` configures the Nginx. +The web root is `/var/www/bigbluebutton-default/assets`, and the page is `/var/www/bigbluebutton-default/assets/backup/index.html`. + + +### Spinning BBB up again from scratch + +Previous notes: 1. Create a Nanode. 2. Update the DNS for bbb.emacsverse.org with its IP address. (dns.he.net) @@ -1131,13 +1428,29 @@ Resources: <a id="general-infrastructure-manual-irc-announcements"></a> -## Manual IRC announcements +## TODO Manual IRC announcements Success: - [X] You can /opall, /conftopic, and /broadcast +<a id="orgc7d6d2a"></a> + +## Check test crontab + + +<a id="general-infrastructure-confirm-automated-irc-announcements-from-res"></a> + +## TODO Confirm automated IRC announcements from res + +Success: + +- A test schedule can generate announcements. + + See C-u M-x emacsconf-stream-crontabs + + <a id="general-infrastructure-low-res-stream"></a> ## Low-res stream @@ -1190,6 +1503,53 @@ backup on live: ## Restream to YouTube +### TODO Set up YouTube restreaming events + +Need to have set up shifts first. + +1. Switch to the EmacsConf account. +2. Go to <https://studio.youtube.com/channel/UCwuyodzTl_KdEKNuJmeo99A> +3. Create > Go live > Schedule stream. + - [X] Gen - Sat AM + - [X] Gen - Sat PM + - [X] Dev - Sat AM + - [X] Dev - Sat PM + - [X] Gen - Sun AM + - [X] Gen - Sun PM + - [X] Test + + emacsconf-stream-copy-livestream-description + + Then set the emacsconf-rtmp-shifts variable. See the conf.org from 2025 for an example. + + This will be used in the shift hyperlists, which you can generate with emacsconf-pad-prepopulate-shift-hyperlists + + There’s also a note in the ansible README.org about restreaming. + Set up the `restreaming_platforms` variable in `prod-vars.yml` and + use `ansible-playbook -i inventory.yml prod-playbook.yml --tags restream` to set up + + +<a id="orgdc911e7"></a> + +## Restream to Toobnix + + +### DONE Set up Toobnix restreaming events + +1. <https://toobnix.org>, Publish, Permanent/recurring live +2. emacsconf-stream-toobnix-copy-livestream-description + + +<a id="org692ea98"></a> + +## i3 window manager + +termit is the terminal + +- M-d: launcher +- M-1, M-2: change workspace + + <a id="other"></a> # Other process notes @@ -1203,9 +1563,20 @@ backup on live: docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"] -<a id="other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted"></a> +<a id="org5f8f6b0"></a> + +## Schedule changes -## When a talk is added after the schedule has already been drafted +In general: + + (emacsconf-schedule-update-from-info) + (emacsconf-update-schedule) + (emacsconf-publish-watch-pages) + (emacsconf-publish-talks-json-to-files) + (emacsconf-publish-info-pages) + + +### When a talk is added after the schedule has already been drafted - [X] Add the talk to conf.org - [X] Add the talk ID to the draft schedule @@ -1223,21 +1594,23 @@ backup on live: - [ ] Record the intro -<a id="cancel-talk"></a> +### When a talk is cancelled -## When a talk is cancelled +<elisp:emacsconf-cancel-talk> 1. Update the status to CANCELLED. Update that specific talk with <elisp:emacsconf-publish-info-pages-for-talk>. 2. Rejig the schedule if needed, and update with <elisp:emacsconf-schedule-update-from-info> -3. Update the published schedule with <elisp:emacsconf-update-schedule> and push the wiki. -4. Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file. -5. Resync the cache directory. -6. Consider updating the screenshots for sat-open and sun-open. +3. Update the published schedule with <elisp:emacsconf-update-schedule>. + +Manual steps: +1. Push the wiki. +2. Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file. +3. Resync the cache directory. +4. Consider updating the screenshots for sat-open and sun-open. -<a id="other-process-notes-when-a-talk-changes-title"></a> -## When a talk changes title +### When a talk changes title 1. Update the title in conf.org. 2. Delete the FILE\_PREFIX property. @@ -1267,7 +1640,7 @@ backup on live: ## To play the other stream -- mpv –profile=full <https://live0.emacsconf.org/gen.webm> +- `mpv --profile=full https://live0.emacsconf.org/gen.webm` - Change the channel topic to note that discussion can be in the other channel |