diff options
Diffstat (limited to '2023/organizers-notebook')
| -rw-r--r-- | 2023/organizers-notebook/index.org | 567 |
1 files changed, 426 insertions, 141 deletions
diff --git a/2023/organizers-notebook/index.org b/2023/organizers-notebook/index.org index 586a239b..cc1b39fd 100644 --- a/2023/organizers-notebook/index.org +++ b/2023/organizers-notebook/index.org @@ -708,17 +708,56 @@ When do we want to do this? We don't have to worry too much, because we can offer them the option of doing it live, and we can have space in the schedule if they cancel last-minute. -** Prepare infrastructure +** Prepare for the conference :PROPERTIES: :CUSTOM_ID: check-infra :END: -*** DONE Get access to media.emacsconf.org so that we can set up the upload service and the backstage area + +#+BEGIN: columnview :id local :match "STATUS={.}" :format "%20ITEM %10TODO %20STATUS" +| ITEM | TODO | STATUS | +|-------------------------+------------+-----------------------------------------------------| +| Upload | DONE | Ready to go | +| Prerec | DONE | Ready to go | +| Captions | INPROGRESS | Ready to go; waiting for videos and captions | +| Backstage | DONE | Ready to go | +| Test assets | | Not yet started | +| BBB | | Rooms created, next step is to create redirects | +| VNC | | confirmed access to emacsconf-gen and emacsconf-dev | +| OBS | | confirmed that gen and dev can stream | +| Icecast | DONE | gen and dev confirmed with MPV | +| MPV | | confirmed that mpv can watch both streams | +| Watch page | | updated for 2023 | +| Status page | | Not yet started | +| Public media | | Not yet started | +| Mumble | | Not yet started | +| Etherpad | DONE | Ready to go, pads created | +| Pad proxy | DONE | Ready to go, pad.emacsconf.org works | +| Overlays | | Not yet started | +| Intros | | Not yet started | +| IRC channels | | Not yet started | +| IRC talk info | | Not yet started | +| Announcements | | Not yet started | +| Publishing updates live | | Not yet started | +| Autopilot | | Not yet started | +| YouTube | | Not yet started | +| Peertube | | Not yet started | +| 480p | | Delegated to corwin | +| Audio processing | | Help wanted | +#+END: + +*** DONE Upload +:PROPERTIES: +:STATUS: Ready to go +:CUSTOM_ID: upload +:END: + +**** DONE Get access to media.emacsconf.org so that we can set up the upload service and the backstage area CLOSED: [2023-10-13 Fri 10:01] SCHEDULED: <2023-10-03 Tue> :PROPERTIES: :CREATED: [023-09-26 Tue 10:3] :CUSTOM_ID: check-media-emacsconf :END: -**** DONE Decide what to do for backstage area and upload service :decision: +***** DONE Decide what to do for backstage area and upload service :decision: CLOSED: [2023-10-13 Fri 10:00] DEADLINE: <2023-10-13 Fri> :PROPERTIES: :CUSTOM_ID: res-or-media @@ -734,11 +773,138 @@ CLOSED: [2023-10-13 Fri 10:00] DEADLINE: <2023-10-13 Fri> - don't need to send people multiple e-mails, risk confusion/out-of-date info - access to media.emacsconf.org might be sorted out by [2023-10-13 Fri] - not in a big rush yet -*** TODO Double-check OBS setup and streaming on res +*** DONE Prerec +CLOSED: [2023-10-17 Tue 09:50] :PROPERTIES: -:CUSTOM_ID: check-streaming +:CUSTOM_ID: prerec +:STATUS: Ready to go +:END: +***** Set up for the new year +As orga@res: + +- mkdir /data/emacsconf/$year +- rm ~/current +- ln -s /data/emacsconf/$year current +- ln -s /data/emacsconf/$year $year + +When we receive files + +- change the TODO status to PROCESSING +- mkdir ~/current/$slug +- copy the files to there +- rename-original $slug $file +- process-prerec $video +- Copy the files to the res:~/cache, laptop:~/proj/emacsconf/2023/cache, and media:~/backstage +- emacsconf-cache-all-video-data +- emacsconf-publish-info-pages +- emacsconf-publish-backstage-index + + +(check that the reencode.sh process has kicked off; if not, call reencode.sh $video $prefix--reencoded.webm) + +*** INPROGRESS Captions +:PROPERTIES: +:CUSTOM_ID: captions +:STATUS: Ready to go; waiting for videos and captions +:END: +- OpenAI Whisper SaaS (https://platform.openai.com/docs/guides/speech-to-text/longer-inputs) limits audio files to 25MB, so it's probably easier to do it ourselves +**** TODO E-mail previous captioning volunteers to see if they're interested in helping out +SCHEDULED: <2023-11-04 Sat> +:PROPERTIES: +:CREATED: [023-10-16 Mon 12:5] +:END: +**** DONE E-mail captioning volunteers the backstage instructions +CLOSED: [2023-10-16 Mon 12:57] +**** DONE E-mail current caption volunteers backstage information, captioning process, etc. +CLOSED: [2023-10-17 Tue 09:37] SCHEDULED: <2023-10-17 Tue> +:PROPERTIES: +:CREATED: [023-10-13 Fri 10:1] +:END: + +I'll wait a few days for Yoni to get back to us about whether he wants +to caption his own talk and/or seeing if other speakers will get their +talks in. + +=emacsconf-mail-backstage-info= + + + +Hi ${name}! + +You're getting this e-mail because you have volunteered to help out +with captions for ${conf-name} ${year}. (Thank you so much!) + +I'm so excited! =) We're starting to get recorded talks, which means +it's time to get captions going. People really appreciate the +captions, especially when the captions have been lovingly edited by +volunteers who change things like "Emax" to "Emacs" and "metaX" to +"M-x". + +Just like last year, we're using OpenAI Whisper to give us a +reasonable starting point for transcripts. If working from scratch +works better for you, you're welcome to do that too. + +We've set up ${backstage} as the backstage area where you can view the +videos and resources uploaded so far. You can access it with the +username "${backstage-user}" and the password "${backstage-password}". +Please keep the backstage password and other speakers' talk resources +secret. ${backstage-use} + +To call dibs on a video to caption, just send a message to me at +sacha@sachachua.com and I can update the page so that it's assigned to +you. You can e-mail me the edited captions when you're done. Don't +worry too much about timestamps; we can re-align the text with the +audio afterwards. If life gets suddenly busy and you can't see it all +the way through, no worries. Just e-mail me what you've got and I'll +put it back in the pool. Every little bit helps! + +Thank you! + +Sacha Chua + +**** TODO E-mail the emacsconf-org mailing list the announcement and the invitation to volunteer +**** TODO [#C] Try out Deepgram, play around with it for last-minute submissions? +**** DONE Get the autocaptions for emms up in the backstage area +CLOSED: [2023-10-13 Fri 10:08] SCHEDULED: <2023-10-08 Sun> +**** Captions lessons learned :lessons: +- OpenAI had a breaking API change, need to call =whisper.utils.get_writer= + - https://github.com/dmarx/video-killed-the-radio-star/issues/101 + #+begin_src python :eval no + vtt_writer = whisper.utils.get_writer('vtt', os.path.dirname(new_file)) + txt_writer = whisper.utils.get_writer('txt', os.path.dirname(new_file)) + vtt_writer(result, work['audio'], {'max_line_width': 60, 'max_line_count': None, 'highlight_words': None}) + txt_writer(result, work['audio'], {'max_line_width': 60, 'max_line_count': None, 'highlight_words': None}) + #+end_src + +**** Reencoding +:PROPERTIES: +:CUSTOM_ID: check-reencoding :END: -*** TODO Generate BBB rooms +*** DONE Backstage +:PROPERTIES: +:CUSTOM_ID: backstage +:STATUS: Ready to go +:END: +*** Test assets +:PROPERTIES: +:CUSTOM_ID: test +:STATUS: Not yet started +:END: +**** TODO Generate test videos +*** BBB +:PROPERTIES: +:CUSTOM_ID: bbb +:STATUS: Rooms created, next step is to create redirects +:END: +**** TODO Generate redirects +:PROPERTIES: +:CUSTOM_ID: generate-redirects +:END: + +We use redirects for Q&A sessions with BBB web conferences so that people can easily join the web conference. This is blocked while waiting for media.emacsconf.org. + +**** DONE Generate BBB rooms +CLOSED: [2023-10-17 Tue 09:41] :PROPERTIES: :CUSTOM_ID: generate-bbb :END: @@ -808,176 +974,214 @@ document.querySelector('.create-room-button').click();" (dolist (group groups) (emacsconf-spookfox-create-bbb group))) #+end_src - -*** TODO Double-check icecast +**** TODO Send testing instructions +SCHEDULED: <2023-11-10 Fri> +*** VNC :PROPERTIES: -:CUSTOM_ID: check-icecast -:END: -*** TODO Figure out a better way to handle 480p stream :corwin: -:PROPERTIES: -:CUSTOM_ID: lowres +:CUSTOM_ID: vnc +:STATUS: confirmed access to emacsconf-gen and emacsconf-dev :END: -It kept dropping last year and sachac didn't have the mental bandwidth to figure it out. -Might need another node so that we don't risk it getting killed for memory reasons? +We use VNC to connect to the X servers on res.emacsconf.org so that we can stream from it. -Corwin has volunteered to take this on +Success: +- [X] Confirm that you can connect to emacsconf-gen via VNC +- [X] Confirm that you can connect to emacsconf-dev via VNC -*** BLOCKED Create pads for all the talks -SCHEDULED: <2023-11-05 Sun> +**** Instructions :PROPERTIES: -:CUSTOM_ID: create-pads -:CREATED: [023-10-13 Fri 10:1] +:CUSTOM_ID: vnc-instructions :END: -Because the pads refer to the next and previous talks and include the talk titles, this is best redone after the schedule has settled down. +NOTE: VNC+OBS doesn't work well if you have a window manager that +automatically resizes windows, like i3. Please configure your window +manager so that the VNC window is not resized. + 1. Install a VNC viewer on your system (ex: tigervnc-viewer). -*** TODO Doublecheck prerec processing workflow -:PROPERTIES: -:CUSTOM_ID: check-prerec -:END: -**** Process -***** Set up for the new year -As orga@res: + 2. Set up your local environment: -- mkdir /data/emacsconf/$year -- rm ~/current -- ln -s /data/emacsconf/$year current -- ln -s /data/emacsconf/$year $year + - gen: export TRACK=gen; export TRACK_PORT=5905; export SSH_PORT=46668 + - dev: export TRACK=dev; export TRACK_PORT=5906; export SSH_PORT=46668 -When we receive files + 3. Copy the password: -- change the TODO status to PROCESSING -- mkdir ~/current/$slug -- copy the files to there -- rename-original $slug $file -- process-prerec $video -- Copy the files to the res:~/cache, laptop:~/proj/emacsconf/2023/cache, and media:~/backstage -- emacsconf-cache-all-video-data -- emacsconf-publish-info-pages -- emacsconf-publish-backstage-index - - -(check that the reencode.sh process has kicked off; if not, call reencode.sh $video $prefix--reencoded.webm) + scp emacsconf-$TRACK@res.emacsconf.org:~/.vnc/passwd vnc-passwd-$TRACK -p $SSH_PORT -**** Captions -:PROPERTIES: -:CUSTOM_ID: check-captions -:END: -- OpenAI Whisper SaaS (https://platform.openai.com/docs/guides/speech-to-text/longer-inputs) limits audio files to 25MB, so it's probably easier to do it ourselves -***** TODO E-mail current caption volunteers backstage information, captioning process, etc. -SCHEDULED: <2023-10-17 Tue> -:PROPERTIES: -:CREATED: [023-10-13 Fri 10:1] -:END: + 4. Forward your local ports and connect via VNC viewer to the + appropriate forwarded port from your laptop: -I'll wait a few days for Yoni to get back to us about whether he wants -to caption his own talk and/or seeing if other speakers will get their -talks in. + #+begin_example + ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT & + sleep 5 # Give it time to establish the tunnels + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & + #+end_example -=emacsconf-mail-backstage-info= +If you get the following error: +#+begin_example +channel 2: open failed: connect failed: Connection refused +CConn: End of stream +CConn: The connection was dropped by the server before the session could + be established. +#+end_example +then the VNC server hasn't started yet. You can start it with -Hi ${name}! +#+begin_src sh :eval no +ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc +#+end_src -You're getting this e-mail because you have volunteered to help out -with captions for ${conf-name} ${year}. (Thank you so much!) +and then connect with: -I'm so excited! =) We're starting to get recorded talks, which means -it's time to get captions going. People really appreciate the -captions, especially when the captions have been lovingly edited by -volunteers who change things like "Emax" to "Emacs" and "metaX" to -"M-x". +#+begin_src sh :eval no +xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & +#+end_src -Just like last year, we're using OpenAI Whisper to give us a -reasonable starting point for transcripts. If working from scratch -works better for you, you're welcome to do that too. +**** TODO Ask bandali or zaeph to share their window manager configuration :bandali:zaeph: -We've set up ${backstage} as the backstage area where you can view the -videos and resources uploaded so far. You can access it with the -username "${backstage-user}" and the password "${backstage-password}". -Please keep the backstage password and other speakers' talk resources -secret. ${backstage-use} +*** OBS +:PROPERTIES: +:CUSTOM_ID: obs +:STATUS: confirmed that gen and dev can stream +:END: -To call dibs on a video to caption, just send a message to me at -sacha@sachachua.com and I can update the page so that it's assigned to -you. You can e-mail me the edited captions when you're done. Don't -worry too much about timestamps; we can re-align the text with the -audio afterwards. If life gets suddenly busy and you can't see it all -the way through, no worries. Just e-mail me what you've got and I'll -put it back in the pool. Every little bit helps! +We use OBS to stream to Icecast on live.emacsconf.org. -Thank you! +Success: Confirm that you can stream +- [X] gen +- [X] dev -Sacha Chua +New year: reprovision with -***** TODO E-mail the emacsconf-org mailing list the announcement and the invitation to volunteer -***** TODO [#C] Try out Deepgram, play around with it for last-minute submissions? -***** DONE Get the autocaptions for emms up in the backstage area -CLOSED: [2023-10-13 Fri 10:08] SCHEDULED: <2023-10-08 Sun> -***** Captions lessons learned :lessons: -- OpenAI had a breaking API change, need to call =whisper.utils.get_writer= - - https://github.com/dmarx/video-killed-the-radio-star/issues/101 - #+begin_src python :eval no - vtt_writer = whisper.utils.get_writer('vtt', os.path.dirname(new_file)) - txt_writer = whisper.utils.get_writer('txt', os.path.dirname(new_file)) - vtt_writer(result, work['audio'], {'max_line_width': 60, 'max_line_count': None, 'highlight_words': None}) - txt_writer(result, work['audio'], {'max_line_width': 60, 'max_line_count': None, 'highlight_words': None}) - #+end_src - -**** Reencoding +#+begin_src sh :eval no +ansible-playbook -i inventory.yml prod-playbook.yml --tags obs +#+end_src + +so that the year is updated in the shell scripts. + +**** Instructions :PROPERTIES: -:CUSTOM_ID: check-reencoding +:CUSTOM_ID: obs-instructions +:END: + +1. [[#vnc-instructions][Connect to the VNC session for the track.]] + +2. Start *recording* (not streaming). If you don't see OBS when you connect, it's probably on workspace 2, so you can switch with Alt-2. If you still don't see it there, you can open a terminal with Alt-Enter and then run ~track-obs~. After you start recording, confirm that it is now broadcasting to the stream. + +3. Verify with MPV on your local system: + + #+begin_example + mpv https://live0.emacsconf.org/$TRACK.webm & + #+end_example + +**** DONE Double-check OBS setup and streaming on res +CLOSED: [2023-10-17 Tue 10:22] +:PROPERTIES: +:CUSTOM_ID: check-streaming :END: -*** TODO Find volunteers for audio processing (normalization, noise reduction) and document the process :helpwanted: +*** DONE Icecast +CLOSED: [2023-10-17 Tue 10:30] :PROPERTIES: -:CUSTOM_ID: audio +:CUSTOM_ID: icecast +:STATUS: gen and dev confirmed with MPV :END: -- audio normalization -- noise reduction +Success: You can use [[#obs-instructions][OBS+VNC to record]], and the results can be viewed by mpv. +- [X] Gen +- [X] Dev +New year: reprovision with -*** TODO [#C] Write something for merging in information from previous years if not specified +#+begin_src sh :eval no +ansible-playbook -i inventory.yml prod-playbook.yml --tags stream +#+end_src + +so that the year is updated in the configuration. + +This is on live.emacsconf.org and can be restarted with =/etc/init.d/emacsconf restart=. + +**** DONE Double-check icecast +CLOSED: [2023-10-17 Tue 10:30] :PROPERTIES: -:CUSTOM_ID: previous-years +:CUSTOM_ID: check-icecast :END: -- availability -- timezone -- name -- short name -- pronouns -** Prepare assets +*** MPV :PROPERTIES: -:CUSTOM_ID: prepare-assets +:CUSTOM_ID: mpv +:STATUS: confirmed that mpv can watch both streams :END: -*** TODO Generate intro slides and overlays +*** DONE Watch page :PROPERTIES: -:CUSTOM_ID: generate-intro +:CUSTOM_ID: watch +:STATUS: updated for 2023 :END: -We generate intro slides to display in between talks so that people -can find out information about the previous talk and learn about the -next talk. It includes talk titles, speaker names, URLs, and Q&A -methods. The image will also be used for an introduction video if we -can record one before the conference. +live.emacsconf.org is on the front0.emacsconf.org server. -We also generate overlays that show talk information during the talk -itself. +To set up for the year: -SVGs don't support line-wrapping, so it helps to do a quick pass to -make sure all the talks are displayed properly. +1. Create directories and update the Nginx configuration + #+begin_src sh :eval no + ansible-playbook -i inventory.yml prod-playbook.yml --tags live + #+end_src + +2. Generate the pages + #+begin_src emacs-lisp :eval no + (emacsconf-publish-watch-pages) + #+end_src -*** BLOCKED Generate redirects +3. Add the $year/watch to the wiki. + +*** Status page :PROPERTIES: -:CUSTOM_ID: generate-redirects +:CUSTOM_ID: status +:STATUS: Not yet started :END: -We use redirects for Q&A sessions with BBB web conferences so that people can easily join the web conference. This is blocked while waiting for media.emacsconf.org. +*** Public media +:PROPERTIES: +:CUSTOM_ID: media +:STATUS: Not yet started +:END: +*** Mumble +:PROPERTIES: +:CUSTOM_ID: mumble +:STATUS: Not yet started +:END: +*** DONE Etherpad +:PROPERTIES: +:CUSTOM_ID: etherpad +:STATUS: Ready to go, pads created +:END: +**** DONE Create pads for all the talks +CLOSED: [2023-10-17 Tue 09:46] SCHEDULED: <2023-11-05 Sun> +:PROPERTIES: +:CUSTOM_ID: create-pads +:CREATED: [023-10-13 Fri 10:1] +:END: -*** TODO Write 1-2 sentence intros for all the talks +Because the pads refer to the next and previous talks and include the talk titles, this is best redone after the schedule has settled down. +*** DONE Pad proxy +:PROPERTIES: +:CUSTOM_ID: proxy +:STATUS: Ready to go, pad.emacsconf.org works +:END: +*** Overlays +:PROPERTIES: +:CUSTOM_ID: overlays +:STATUS: Not yet started +:END: +*** Intros +:PROPERTIES: +:CUSTOM_ID: intros +:STATUS: Not yet started +:END: +**** BLOCKED Record intro videos with 1-2 sentence intros for all the talks +:PROPERTIES: +:CUSTOM_ID: record-intros +:END: +**** TODO Write 1-2 sentence intros for all the talks :PROPERTIES: :CUSTOM_ID: write-intros :END: @@ -1004,15 +1208,15 @@ conf.org. Actually recording the introductions can wait until closer to the conference because talk titles and Q&A methods can change. We can verify speaker name pronunciations at that time. -**** Intros :levels: +***** Intros :levels: - Good: Use the same template as before: In this talk, SPEAKER shares TITLE OR SUMMARY. Afterwards, PRONOUN will handle questions over Q&A METHOD. - Better: Include some more biographical information to give listeners some context. - -**** DONE Do we want to use honorifics like Dr.? :decision: + +***** DONE Do we want to use honorifics like Dr.? :decision: CLOSED: [2023-10-03 Tue 15:38] DEADLINE: <2023-11-14 Tue> :PROPERTIES: :CUSTOM_ID: honorifics @@ -1043,22 +1247,103 @@ use it when recording the intros. - OPTION: Using honorifics based on e-mail signatures and intros: - Recognizes credentials -*** BLOCKED Record intro videos with 1-2 sentence intros for all the talks +**** TODO Generate intro slides :PROPERTIES: -:CUSTOM_ID: record-intros +:CUSTOM_ID: generate-intro :END: -** Coordinate volunteers + +We generate intro slides to display in between talks so that people +can find out information about the previous talk and learn about the +next talk. It includes talk titles, speaker names, URLs, and Q&A +methods. The image will also be used for an introduction video if we +can record one before the conference. + +We also generate overlays that show talk information during the talk +itself. + +SVGs don't support line-wrapping, so it helps to do a quick pass to +make sure all the talks are displayed properly. + +*** IRC channels :PROPERTIES: -:CUSTOM_ID: coordinate-volunteers +:CUSTOM_ID: irc +:STATUS: Not yet started :END: -*** DONE E-mail captioning volunteers the backstage instructions -CLOSED: [2023-10-16 Mon 12:57] -*** TODO E-mail previous captioning volunteers to see if they're interested in helping out -SCHEDULED: <2023-11-04 Sat> +*** IRC talk info :PROPERTIES: -:CREATED: [023-10-16 Mon 12:5] +:CUSTOM_ID: irc-talk-info +:STATUS: Not yet started +:END: +*** Announcements +:PROPERTIES: +:CUSTOM_ID: announcements +:STATUS: Not yet started +:END: +*** Publishing updates live +:PROPERTIES: +:CUSTOM_ID: publish +:STATUS: Not yet started +:END: +*** Autopilot +:PROPERTIES: +:CUSTOM_ID: autopilot +:STATUS: Not yet started +:END: +*** YouTube +:PROPERTIES: +:CUSTOM_ID: youtube +:STATUS: Not yet started +:END: +*** Peertube +:PROPERTIES: +:CUSTOM_ID: peertube +:STATUS: Not yet started +:END: +*** 480p +:PROPERTIES: +:STATUS: Delegated to corwin +:END: + +Consider increasing memory/cpu configuration on live? + +**** TODO Figure out a better way to handle 480p stream :corwin: +:PROPERTIES: +:CUSTOM_ID: lowres +:END: + +It kept dropping last year and sachac didn't have the mental bandwidth to figure it out. +Might need another node so that we don't risk it getting killed for memory reasons? + +Corwin has volunteered to take this on +*** Audio processing +:PROPERTIES: +:CUSTOM_ID: audio-proc +:STATUS: Help wanted +:END: + +**** TODO Find volunteers for audio processing (normalization, noise reduction) and document the process :helpwanted: +:PROPERTIES: +:CUSTOM_ID: audio +:END: + +- audio normalization +- noise reduction +*** TODO [#C] Write something for merging in information from previous years if not specified +:PROPERTIES: +:CUSTOM_ID: previous-years +:END: +- availability +- timezone +- name +- short name +- pronouns +** Volunteers +:PROPERTIES: +:CUSTOM_ID: coordinate-volunteers +:STATUS: Need to coordinate people :END: -*** TODO Prepare shift calendar +*** TODO Prepare shift calendar, ask people to sign up +SCHEDULED: <2023-11-01 Wed> :PROPERTIES: :CUSTOM_ID: shifts :END: |