From 3ddf955b120d6e138e0888c87c873a2021f3b076 Mon Sep 17 00:00:00 2001 From: Sacha Chua Date: Thu, 14 Nov 2024 10:23:07 -0500 Subject: Update org notebook --- 2024/organizers-notebook.md | 50 ++--- 2024/organizers-notebook/index.org | 2 +- organizers-notebook.md | 397 ++++++++++++++++++++++++++++++++++++- organizers-notebook/index.org | 72 +++---- 4 files changed, 455 insertions(+), 66 deletions(-) diff --git a/2024/organizers-notebook.md b/2024/organizers-notebook.md index 4fb8981f..e501f8e6 100644 --- a/2024/organizers-notebook.md +++ b/2024/organizers-notebook.md @@ -15,18 +15,18 @@ You might also like the [general organizers' notebook](/organizers-notebook) and - [Phases](#phases) - [Draft CFP](#cfp) - [Draft schedule](#draft-schedule) -- [Check EmacsConf infrastructure](#org8ba949b):project: - - [bbb.emacsverse.org replacement](#org0424b5e) - - [IRC web client](#org5a0fcc3) - - [IRC announcements](#orgfc5854f) - - [Media](#org34eea83) - - [Publishing resources to the wiki](#orgbbb5ff7) - - [Publishing videos to the media server](#orgd35db12) - - [Playing videos, switching to windows](#org921edcd) - - [Etherpad](#org31ad763) - - [Do a dry run](#org11fdfb4) -- [Processes and notes](#orgca4be3d) - - [ERC](#orga534b2b) +- [Check EmacsConf infrastructure](#org5b05724):project: + - [bbb.emacsverse.org replacement](#orgf90c350) + - [IRC web client](#org47f1067) + - [IRC announcements](#org8966827) + - [Media](#org75bdf93) + - [Publishing resources to the wiki](#orgb420d40) + - [Publishing videos to the media server](#org4e0acb2) + - [Playing videos, switching to windows](#org40b9e70) + - [Etherpad](#org1040525) + - [Do a dry run](#org215b633) +- [Processes and notes](#org73e9be9) + - [ERC](#orgedf8032) - [Decisions](#decisions) - [BigBlueButton replacement](#bbb) - [Support code](#support-code) @@ -272,7 +272,7 @@ Notes: - [learning](https://emacsconf.org/2024/talks/learning "Survival of the skillest: Thriving in the learning jungle"), [sharing](https://emacsconf.org/2024/talks/sharing "So you want to be an Emacs-fluencer?"), [literate](https://emacsconf.org/2024/talks/literate "Literate programming for the 21st Century") will probably be good for a general audience - + # TODO Check EmacsConf infrastructure :project: @@ -288,7 +288,7 @@ Notes: - [ ] YouTube - + ## WAITING bbb.emacsverse.org replacement @@ -303,7 +303,7 @@ Notes: ### TODO Create meeting rooms for each speaker - + ## IRC web client @@ -311,7 +311,7 @@ Notes: ### WAITING Ask libera.chat to increase connections allowed from chat.emacsconf.org on Dec 7 and 8 - + ## IRC announcements @@ -322,7 +322,7 @@ Notes: ### TODO Confirm automated IRC announcements from res - + ## Media @@ -336,19 +336,19 @@ Notes: You can generate the index with `emacsconf-publish-update-media`. - + ## TODO Publishing resources to the wiki ansible-playbook -i inventory.yml prod-playbook.yml –tags publish - + ## TODO Publishing videos to the media server - + ## Playing videos, switching to windows @@ -359,7 +359,7 @@ ansible-playbook -i inventory.yml prod-playbook.yml –tags publish ### TODO Document how to get that set up again - + ## Etherpad @@ -370,7 +370,7 @@ ansible-playbook -i inventory.yml prod-playbook.yml –tags publish ### TODO Generate the main index - + ## TODO Do a dry run @@ -381,12 +381,12 @@ ansible-playbook -i inventory.yml prod-playbook.yml –tags publish ### TODO Test connecting to VNC and streaming via OBS - + # Processes and notes - + ## ERC @@ -473,7 +473,7 @@ Some options: - Let’s plan for a week at high capacity. (\* 0.22 24 7) USD 36.96 - and a month of 10GB, possibly up to 30GB. USD 3 - That’s probably small enough that it doesn’t make as much sense to try to fully spin down for a week in between now and the conference (\* 0.04 24 7) - which would save USD 6.72 but require figuring out backing up, etc. - - **Option B: Run a slightly larger shared CPU instance from now until shortly after the conference** in case BBB does not work for installation/testing with less memory; block storage probably not needed + - **Option B: Run a slightly larger shared CPU instance from now until shortly before the conference, then scale up** in case BBB does not work for installation/testing with less memory; block storage probably not needed - (\* 0.07 24 7 3) USD 35.28 for 8GB, total (+ 35.28 36.96) USD 72.24 - (\* 0.14 24 7 3) USD 70.56 for 16GB, total (+ 70.56 36.96) USD 107.52 diff --git a/2024/organizers-notebook/index.org b/2024/organizers-notebook/index.org index 72b38de7..bbb1032a 100644 --- a/2024/organizers-notebook/index.org +++ b/2024/organizers-notebook/index.org @@ -1,4 +1,4 @@ -# [[elisp:(progn (memoize 'emacsconf-get-talk-info "5 seconds") (org-md-export-to-markdown) (memoize-restore 'emacsconf-get-talk-info))][Export this file to Markdown]] +# [[elisp:(progn (memoize 'emacsconf-get-talk-info "5 seconds") (org-md-export-to-markdown) (memoize-restore 'emacsconf-get-talk-info))][EPpxport this file to Markdown]] #+TAGS: emacsconf # [[elisp:(memoize 'emacsconf-get-talk-info "5 seconds")][Memoize emacsconf-get-talk-info]] - [[elisp:(memoize-restore 'emacsconf-get-talk-info)][Unmemoize]] #+todo: TODO(t) SOMEDAY STARTED INPROGRESS(i) WAITING(w) STANDBY(s) BLOCKED(b) | DONE(x) CANCELLED(c) diff --git a/organizers-notebook.md b/organizers-notebook.md index f5d6206b..7e338275 100644 --- a/organizers-notebook.md +++ b/organizers-notebook.md @@ -20,11 +20,32 @@ This document is the general organizers' notebook that can be used as a starting - [Create the public organizers’ notebook in the wiki](#starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki) - [Phases and lessons learned](#phases-and-lessons-learned) - [Draft CFP](#phases-and-lessons-learned-draft-cfp) - - [Process proposals](#orge2fe34a) - - [Accept proposals](#orgac059dd) + - [Announce](#org3608282) + - [Process proposals](#orgf94fb47) + - [Accept proposals](#org74dc473) - [Draft schedule](#draft-schedule) + - [Set up backstage and upload](#org4d60026) + - [Post the schedule](#org42db0c4) + - [Process uploaded files](#org60f6486) + - [Generate assets](#org6e66fc5) + - [Prepare the infrastructure](#org6983794) + - [Getting ready for captions](#orga1601de) +- [General infrastructure](#orga2c0410) + - [Icecast](#icecast) + - [Watching pages](#watch) + - [VNC](#vnc) + - [Streaming with OBS](#org0a63f1c) + - [IRC web client](#orgaffe686) + - [Etherpad](#org628c2fe) + - [Publishing media to the server and to the wiki](#media) + - [Automated IRC announcements](#org3ca9756) + - [Manual IRC announcements](#org4cbb72c) + - [Low-res stream](#org84b7e1e) + - [Restream to YouTube](#org1e5d62e) + - [Restream to Toobnix](#orga5282bc) - [Other process notes](#other) - [Adding another user to BBB](#bbb-user) + - [Ansible](#org8ae0b84) @@ -125,7 +146,15 @@ Put inside double square brackets: `!template id=pagedraft` - Put the speaker introduction right after the talk description for easier web page copying. - + + +## Announce + +- Post CFP to emacsconf-discuss, reddit.com/r/emacs, and in Emacs News +- Submit to + + + ## Process proposals @@ -154,7 +183,7 @@ When proposals come in: PUBLIC\_EMAIL - + ## Accept proposals @@ -187,6 +216,359 @@ For each talk: - Add missing items to plan + + +## Set up backstage and upload + +See README.org. + +Lessons learned: + +- Next year, let’s get upload.emacsconf.org in place as the domain name so that people don’t try to FTP files to it. + + + + +## Post the schedule + +- +- +- +- +- Update talks.md to refer to watch.md as well + + + + +## Process uploaded files + +Basic setup: + +1. Set up the media server and symlinks: README.org (Media). +2. Set up the prerec processing areas on res: README.org (Processing prerecs). +3. Set up the caption processor on res: README.org (Captioning). + +Process: + +1. Review uploaded files with +2. Open the JSON for the file you want to process and use . + If more than one file uses the same extension, provide a suffix to disambiguate them. +3. On res: + 1. Create a directory ~/current/files/$slug. + 2. Copy the uploaded file from ~/cache as `$video_slug--original.$extension`, or use `rename-original.sh $slug $file`. + 3. Call `process-prerec.sh $file`. +4. When the screens are finished, do locally: + 1. update-cache on my computer. + 2. or + 3. + + + + +## Generate assets + + +### In-between pages + +- Make the $year/assets/in-between directory and copy the template.svg from the previous year. +- + + +### Intros + +- In an empty VTT file, +- Record the audio and get captions +- Splice together into a file that will be processed with subed-record-compile-video. + + + + +## Prepare the infrastructure + +Checklist: + +- [ ] IRC + - [ ] Manual announcements + - [ ] Automated announcements +- [ ] Web conference +- [ ] Streaming assets + - [ ] Overlays + - [ ] Intros +- [ ] OBS +- [ ] VNC +- [ ] Publishing to the wiki +- [ ] Publishing to the media server +- [ ] Etherpad +- [ ] Mumble +- [ ] Icecast + + + + +## Getting ready for captions + +Onboarding: + +- Add captioner to conf.org with the `:caption:` tag, under a Volunteers heading with the `volunteer` tag. Make sure the `EMAIL` property is set. +- Add captioner to emacsconf-org mailing list. +- Send backstage information . (Can be sent to all captioning volunteers with ) + + + + +# General infrastructure + + + + +## Icecast + +Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv. + +- [X] Gen +- [X] Dev + +Also, recordings are available in /data. + +New year: + +1. Reprovision with + + ansible-playbook -i inventory.yml prod-playbook.yml --tags stream + + so that the year is updated in the configuration. +2. SSH to live0.emacsconf.org and run `/etc/init.d/emacsconf stop`. Confirm that icecast has stopped with `ps | grep emacsconf`; if not, kill the process. + Start it again with `/etc/init.d/emacsconf start` +3. To test, you can use FFmpeg like this for the gen stream: + + ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \ + -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ + -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/gen.webm" + + For dev: + + ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \ + -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ + -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/dev.webm" + + and then view it with + + mpv https://live0.emacsconf.org/gen.webm + + + + +## Watching pages + +Success: You can watch at . + +- [X] goes to the current year’s page. +- [X] gen stream works in the browser +- [X] dev stream works in the browser +- [X] Schedule is correct + +New year: + +1. ansible-playbook -i inventory.yml prod-playbook.yml –tags live +2. Republish with . +3. Update front:/var/www/live.emacsconf.org. + +Set up test streams (see ffmpeg instructions from [Icecast](#icecast)) and then: + + + + +## VNC + +We use VNC to connect to the X servers on res.emacsconf.org so that we can stream from it. + +Success: + +- [X] Confirm that you can connect to emacsconf-gen via VNC +- [X] Confirm that you can connect to emacsconf-dev via VNC + +Setting up + +- + + + + +#### Instructions + +NOTE: VNC+OBS doesn’t work well if you have a window manager that +automatically resizes windows, like i3. Please configure your window +manager so that the VNC window is not resized. + +1. Install a VNC viewer on your system (ex: tigervnc-viewer). + +2. Set up your local environment: + - gen: `export TRACK=gen; export TRACK_PORT=5905; export SSH_PORT=46668` + - dev: export TRACK=dev; export TRACK\_PORT=5906; export SSH\_PORT=46668 + +3. Copy the password: + + scp emacsconf-$TRACK@res.emacsconf.org:~/.vnc/passwd vnc-passwd-$TRACK -p $SSH\_PORT + +4. Forward your local ports and connect via VNC viewer to the + appropriate forwarded port from your laptop: + + ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT & + sleep 5 # Give it time to establish the tunnels + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & + +If you get the following error: + + channel 2: open failed: connect failed: Connection refused + CConn: End of stream + CConn: The connection was dropped by the server before the session could + be established. + +then the VNC server hasn’t started yet. You can start it with + + ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc + +and then connect with: + + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & + + + + +## Streaming with OBS + +Success: Confirm that you can stream + +- [X] gen +- [X] dev + +New year: reprovision with + + ansible-playbook -i inventory.yml prod-playbook.yml --tags obs + +so that the year is updated in the shell scripts. + + + + +#### Instructions + +1. [Connect to the VNC session for the track.](#vnc-instructions) + +2. Start **recording** (not streaming). If you don’t see OBS when you connect, it’s probably on workspace 2, so you can switch with Alt-2. If you still don’t see it there, you can open a terminal with Alt-Enter and then run `track-obs`. After you start recording, confirm that it is now broadcasting to the stream. + +3. Verify with MPV on your local system: + + mpv https://live0.emacsconf.org/$TRACK.webm & + + +### Icecast + +Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv. + +- [ ] gen +- [ ] dev + +New year: reprovision with + + ansible-playbook -i inventory.yml prod-playbook.yml --tags stream + +so that the year is updated in the configuration. + +This is on live.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`. + + + + +## IRC web client + +Success: + +- [X] You can join from +- [X] You can join from and the channels end up as the latest ones + +Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors. + +Message I’ve posted to support@libera.chat: + +Thank you for running libera.chat. We’re looking forward to using IRC +(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming +EmacsConf (Dec 7-8, ). In the past, we’ve needed to +coordinate with Libera staff to increase the number of connections +allowed from chat.emacsconf.org during conference days in order to avoid +connection exhaustion errors. Is that something we can plan now or would +you like me to bring it up a few days before the conference? + + + + +## Etherpad + +Success: + +- [X] pad.emacsconf.org redirects to this year’s entry +- [X] Each talk has its own Etherpad + + + + +## Publishing media to the server and to the wiki + +Start of year: + +1. Set `media_protect_root` to true in Ansible `group_vars/all.yml`. +2. `ansible-playbook -i inventory.yml prod-playbook.yml --tags media` +3. Generate the index with `emacsconf-publish-update-media` + +Confirm by setting a submitted talk to `PLAYING` and testing with + . The public media +directory should have the files and the entry should be in the index. +Switching it back to `TO_STREAM` and calling + should remove it. + + +### TODO Switch public media to unprotected root before the conference + +1. Clear public media directory. +2. Set `media_protect_root` to false in Ansible `group_vars/all.yml`. +3. `ansible-playbook -i inventory.yml prod-playbook.yml --tags media` + +You can generate the index with `emacsconf-publish-update-media`. + + + + +## Automated IRC announcements + +Success: + +- [ ] When a talk starts playing, it is announced in the relevant channel + + + + +## Manual IRC announcements + +Success: + +- [X] You can /opall, /conftopic, and /broadcast + + + + +## Low-res stream + + + + +## Restream to YouTube + + + + +## Restream to Toobnix + + # Other process notes @@ -199,3 +581,10 @@ For each talk: ssh bbb.emacsverse.org docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"] + + + +## Ansible + + git clone git@git.emacsconf.org:pub/emacsconf-ansible + diff --git a/organizers-notebook/index.org b/organizers-notebook/index.org index 59c73611..f8a67d44 100644 --- a/organizers-notebook/index.org +++ b/organizers-notebook/index.org @@ -287,7 +287,6 @@ New year: #+begin_src sh mpv https://live0.emacsconf.org/gen.webm #+end_src - ** Watching pages :PROPERTIES: :CUSTOM_ID: watch @@ -306,40 +305,6 @@ New year: Set up test streams (see ffmpeg instructions from [[#icecast][Icecast]]) and then: - -** IRC web client - -Success: -- [X] You can join from https://chat.emacsconf.org -- [X] You can join from https://live.emacsconf.org and the channels end up as the latest ones - -Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors. - -Message I've posted to support@libera.chat: - -Thank you for running libera.chat. We're looking forward to using IRC -(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming -EmacsConf (Dec 7-8, https://emacsconf.org). In the past, we've needed to -coordinate with Libera staff to increase the number of connections -allowed from chat.emacsconf.org during conference days in order to avoid -connection exhaustion errors. Is that something we can plan now or would -you like me to bring it up a few days before the conference? -** Etherpad - -Success: - -- [X] pad.emacsconf.org redirects to this year's entry -- [X] Each talk has its own Etherpad - -** Manual IRC announcements - -Success: - -- [ ] You can /opall, /conftopic, and /broadcast - - -** Automated IRC announcements - ** VNC :PROPERTIES: :CUSTOM_ID: vnc @@ -451,7 +416,30 @@ ansible-playbook -i inventory.yml prod-playbook.yml --tags stream so that the year is updated in the configuration. This is on live.emacsconf.org and can be restarted with =/etc/init.d/emacsconf restart=. -** Public media +** IRC web client + +Success: +- [X] You can join from https://chat.emacsconf.org +- [X] You can join from https://live.emacsconf.org and the channels end up as the latest ones + +Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors. + +Message I've posted to support@libera.chat: + +Thank you for running libera.chat. We're looking forward to using IRC +(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming +EmacsConf (Dec 7-8, https://emacsconf.org). In the past, we've needed to +coordinate with Libera staff to increase the number of connections +allowed from chat.emacsconf.org during conference days in order to avoid +connection exhaustion errors. Is that something we can plan now or would +you like me to bring it up a few days before the conference? +** Etherpad + +Success: + +- [X] pad.emacsconf.org redirects to this year's entry +- [X] Each talk has its own Etherpad +** Publishing media to the server and to the wiki :PROPERTIES: :CUSTOM_ID: media :END: @@ -475,6 +463,18 @@ elisp:emacsconf-publish-media-files-on-change should remove it. 3. =ansible-playbook -i inventory.yml prod-playbook.yml --tags media= You can generate the index with =emacsconf-publish-update-media=. +** Automated IRC announcements + +Success: + +- [ ] When a talk starts playing, it is announced in the relevant channel + +** Manual IRC announcements + +Success: + +- [X] You can /opall, /conftopic, and /broadcast + ** Low-res stream ** Restream to YouTube ** Restream to Toobnix -- cgit v1.2.3