path: root/playbook.org

#+PROPERTY: header-args:emacs-lisp :tangle playbook/emacsconf-playbook.el :eval no
#+OPTIONS: toc:4
#+begin_export md
<!-- This file was automatically generated from playbook.org. Instead of editing the .md, please edit the .org and republish. Thanks! -->
#+end_export

* Commands :noexport:

- [[elisp:(progn (org-md-export-to-markdown) (org-babel-tangle))][Export and tangle]]

* Roles/values
:PROPERTIES:
:CUSTOM_ID: roles
:END:

| B     |                                                                      | bandali |
| CHECK | check-ins                                                            | sachac  |
| HOST  | hosting, streaming, intros, reading, moving questions/answers around | zaeph   |

| ${protected}     | [[/ssh:front:/var/www/media.emacsconf.org/2021/protected/]] |
| ${stream-status} | [[/ssh:front:/var/www/live.emacsconf.org/index.html]] |
| ${upcoming}      | upcoming.org shared over CRDT                           |
| ${conf}          | conf.org shared over CRDT                               |
| ${playbook}      | playbook.org shared over CRDT                           |
| ${dump}          | [[/ssh:live:/data/]]                                        |
| ${media}         | [[/ssh:front:/var/www/media.emacsconf.org/2021/]]           |
| ${live-main} |  [[/ssh:front:/var/www/live.emacsconf.org/main.html]] | 
| ${live-480p} |  [[/ssh:front:/var/www/live.emacsconf.org/main-480p.html]] | 
| ${live-index} |  [[/ssh:front:/var/www/live.emacsconf.org/index.html]] | 
| ${live-alt}  |  [[/ssh:front:/var/www/live.emacsconf.org/alt.html]]  | 
| ${profile} |  [[/ssh:live:~/.profile]] | 

* Thursday or Friday before the conference
:PROPERTIES:
:CUSTOM_ID: days-before
:END:

- STREAM: Download prerecorded videos from ${protected}
  #+begin_src sh :eval no
  rsync -avzue ssh front:/var/www/media.emacsconf.org/2021/protected/*--main.webm .
  #+end_src

* On the day of the conference
:PROPERTIES:
:CUSTOM_ID: day-of
:END:
** Set up
:PROPERTIES:
:CUSTOM_ID: setup
:END:
*** Arrange screens
:PROPERTIES:
:CUSTOM_ID: screens
:END:

- CHECK:
  - Share ${upcoming}, ${playbook}, and ${conf} via CRDT: =conf-crdt-connect-and-share=
    - Current schedule, filenames/commands for playing, Q&A preference, IRC nick, pronunciation, intro notes, prerec duration, emergency contact information
    - =conf-upcoming-add-subtree=
  - Have #emacsconf-org, #emacsconf, #emacsconf-accessible, and #emacsconf-questions open
  - Use =/opall= to get op privileges in all the channels
  - Start backup process for pad
    #+begin_src sh :eval no :tangle playbook/backup-pad.sh
    while true; do
        curl https://etherpad.wikimedia.org/p/emacsconf-2021/export/html > emacsconf-$(date +"%Y%m%d-%H%M%S").html
      sleep 15m
    done
    #+end_src
  - Computer for alternate streaming:
    - Open browser for joining BBB
    - Open MPV for playing https://live0.emacsconf.org/main.webm
- HOST:
  - rsync the newest --main.webm from front: rsync -avze ssh front:/var/www/media.emacsconf.org/2021/protected/*--main.webm .
  - Check OBS scenes for sharing windows/tabs as a virtual camera:
    - chat.emacsconf.org with #emacsconf
    - Etherpad
    - Schedule
    - next talk page
    - Clock with current time on screen: =watch TZ=America/Toronto date=
  - Set up backchannel for easy viewing
    - ${upcoming}
    - #emacsconf-org and #emacsconf channels
  - (?) Join organizer room S
    
*** Start streaming                                                :stream:
:PROPERTIES:
:CUSTOM_ID: start-streaming
:END:

- HOST: Display getting-ready message and start streaming to main.webm
- HOST: Confirm that the stream is live at https://live.emacsconf.org/main.webm
- B: Update ${status} to say that the stream is live
- CHECK: Start low-resolution stream, confirm at https://live.emacsconf.org/main-480p.webm
  Call this on live0 with $CONF480PASS as the first parameter. The Icecast configuration is on =live0= at [[file:/ssh:live|sudo:/etc/icecast2/icecast.xml]]=.
  #+begin_src sh :eval no :tangle "playbook/restream-lowres.sh"
     PASS=$1
     while true; do ffmpeg -loglevel 24 -f webm -reconnect_at_eof 1 -reconnect_streamed 1  -re -i http://localhost:8000/main.webm -vf scale=854:480 -f webm -c:a copy -b:v 500k -maxrate 1M -bufsize 1M  -content_type video/webm -c:v libvpx icecast://ec2020main480pmu:$PASS@localhost:8000/main-480p.webm; done
  #+end_src
- CHECK: Start Youtube and Toobnix streams. Call this with $YOUTUBE1PASS, $YOUTUBE2PASS, or $TOOBNIX as the parameter
  #+begin_src sh :eval no :tangle "playbook/restream-flv.sh"
     MOUNT=$1
     while true; do ffmpeg -loglevel 24 -i http://localhost:8000/main.webm -cluster_size_limit 2M -cluster_time_limit 5100 -b:v 1M -crf 30 -g 125 -deadline good -threads 4 -vcodec libx264 -acodec libmp3lame -f flv $MOUNT; done
  #+end_src
- CHECK: Verify YouTube and Toobnix streams and the CPU load on live0.
- CHECK: Set the YouTube and Toobnix streams to public.
- B: Verify with #emacsconf that the stream is active.
- CHECK: Play main stream on alternate laptop. Start alternate stream and verify. Update ${status}.
- CHECK: Announce on Twitter (@emacs, @emacsconf, @sachac) and in #emacs
  EmacsConf 2021 starting now: https://emacsconf.org/2021/
   
** Talk process
:PROPERTIES:
:CUSTOM_ID: talk
:END:
*** Check in a speaker
:PROPERTIES:
:CUSTOM_ID: check-in
:END:

Exception: [[*CHECK is unavailable][CHECK is unavailable]]

- Speaker checks in on #emacsconf-org via IRC or via e-mail ~30m before
- CHECK notes IRC nick for speaker.
- CHECK confirms Q&A preference: live/IRC/Etherpad, preferred way of getting questions
- [? unknown] Thanks for checking in! How would you like to handle Q&A
          today - live video, the collaborative Etherpad at
          https://etherpad.wikimedia.org/p/emacsconf-2021 , or IRC (like
          this)?
- [? IRC] Thanks for checking in! Feel free to keep an eye on
          #emacsconf for questions and discussion, and we'll copy things from
          the pad to there. If the volume gets overwhelming, let us know and
          we can forward questions to #emacsconf-questions for you. If you'd
          like to try Q&A over live video or the collaborative pad instead, or
          if you need help, please let us know.
- [? Etherpad] Thanks for checking in! The collaborative pad we'll be
          using for questions is at
          https://etherpad.wikimedia.org/p/emacsconf-2021 . We'll collect
          questions from #emacsconf and put them there. If you'd like to jump
          to your part of the document, you might be able to keep an eye on
          questions. Please let us know if you need help, or if you want to
          switch to live Q&A.
- [? live] Thanks for checking in! I'll send you some private messages
          with instructions, so please check there. Let me know if you don't
          get them.
  - Private messages:
    - (erc-message "PRIVMSG" (format "%s You can use this BBB room: %s . I'll join you there shortly to set up the room and do the last-minute tech check." nick room-url))
    - (erc-message "PRIVMSG" (format "%s The collaborative pad we'll be using for questions is at %s . We'll collect questions from #emacsconf and put them there. If you'd like to jump to your part of the document, you might be able to keep an eye on questions. Alternatively, we can read questions to you." nick conf-collaborative-pad))
    - (erc-message "PRIVMSG" (format "%s Leo Vivier will join when it's time, and he will give you the go-ahead when it's time to present. See you in the BBB room!" nick))
  - CHECK directs speaker to available room with =/checkin <room> <nick>=
  - Speaker joins talk room
  - CHECK makes speaker presenter and moderator, does last-minute tech check
    - Hello, thanks
    - Speaker tries screen sharing and webcam (optional)
    - check screen readability
    - CHECK briefs speaker on process, including:
      - live Q&A: reading questions themselves (can do in any order,
        can skip; coach possible responses for awkward things) or asking HOST to read questions to them
        - HOST can share the pad or IRC; speaker shares screen only if doing demo
      - encouragement of webcam, although it's optional
      - how HOST will join shortly before the prerec ends and then
        give them the go-ahead
      - closing any tabs watching the stream as their talk starts
        (otherwise the audio is confusing)
    - If the speaker will be giving a live presentation, CHECK
      collects emergency contact information (in case of technical
      issues) and shares it with HOST in the CRDT buffer
    - Okay to do other things until the prerec ends
  - CHECK updates ${upcoming} with link to the talk room and
    preferences for Q&A-. CHECK will also /msg the relevant
    information.

*** Present talk
:PROPERTIES:
:CUSTOM_ID: present
:END:

- CHECK announces the next talk on IRC and marks the previous talk as done. (=conf-announce=)
- PAD clears pad colours.
- [? prerec]
  - HOST switches to MPV scene in OBS and plays the video (with captions if available).
    - Exception: [[*Last-minute prerecording submission][Last-minute prerecording submission]]
    - Exception: [[*Last-minute caption update][Last-minute caption update]]
  - [[*Publish information][CHECK publishes information]]
  - [[*Handle Q&A][HOST gets a head start on handling Q&A]]
  - When prerec finishes, HOST switches the OBS scene to show BBB.
- [? live]
  - Exception: [[*Speaker does not have a prerec and has not checked in][Speaker does not have a prerec and has not checked in]]
  - HOST joins the BBB room and double-checks that recording is on.
  - CHECK-alternate joins the BBB room and pauses main MPV.
  - HOST switches to OBS scene for BBB.
  - Speaker presents.
    - Exception: [[*Technical issues during a live presentation][Technical issues during a live presentation]]
  - [? talk needs to be wrapped up]
    - HOST nudges speaker verbally.
  
*** Publish information
:PROPERTIES:
:CUSTOM_ID: publish
:END:

  - CHECK updates the schedule in:
    - ${conf}
    - ${upcoming}
    - wiki
  - CHECK publishes the video to media.emacsconf.org using =conf-publish-files=
  - CHECK commits the wiki page and the captions for the talk.
  - CHECK publishes the video on YouTube and ToobNix using =conf-video-share=.
    - Update description:
      #+begin_example
This video is available under the terms of the
Creative Commons Attribution-ShareAlike 4.0 International (CC
BY-SA 4.0) license.

You can view it using free and open source software at
${url}

${description}
      #+end_example
    - Mark it as public.
    - Add it to EmacsConf 2021 playlist.
    - Update title and description.
    - Mark it as public.
    - Doublecheck subtitles
    - Add it to the EmacsConf 2021 playlist.
  - [? live sections]
    - CHECK does a rough-cut of the recording from ${dump} to get the last X minutes or by time range. There's about a 1-2 minute delay.
      Ex: =(kill-new (conf-dump-get "alt" "10:24" "10:30" "qa_"))=
    - When there's an opportunity to do so:
      - CHECK finetunes the rough-cut recording (trim start and end) and posts it to:
        - media.emacsconf.org/2021
        - wiki page for talk
    
*** Handle Q&A
:PROPERTIES:
:CUSTOM_ID: questions
:END:

Exceptions:
- [[*Speaker has not checked in][Speaker has not checked in]]

- [? live]
  - CHECK-alternate joins the BBB room and pauses MPV.
  - HOST joins the BBB room
  - HOST starts recording in BBB or confirms that it's already recording
  - HOST switches to the BBB scene in OBS.
  - HOST describes how to ask questions.
  - [? No questions yet]
    - HOST thanks speaker, says nice things about talk, and asks a couple of prepared questions
  - [? Awkward question]
    - HOST can try rephrasing the question.
    - HOST adds note to IRC/Etherpad that speakers can answer in any order, skip questions, answer afterwards, etc.
  - [? Q&A needs to be wrapped up]
    - HOST writes in Etherpad/IRC or nudges speaker verbally.
  - CHECK notes the time that the live Q&A finished and switches back to the main stream on CHECK-alternate.
- [? IRC/pad]
  - HOST switches to pad/chat OBS scene.
  - HOST describes Q&A method and shows it on the screen.
  - While there's buffer time before the next talk, HOST can read out
    questions and answers, or transition to the next talk early
  - HOST: It's time for the next talk, but if you want to keep
    discussing the previous talk, please feel free to continue doing
    so on IRC or the pad.
- [? speaker will answer after the conference]
  - HOST switches to pad/chat OBS scene.
  - HOST says the speaker is not available right now, but we'll
    forward the questions to the speaker and we'll post the speaker's
    answers on the wiki page. Leave your contact information if you
    want to be notified, or subscribe to the emacsconf-discuss mailing
    list to get the announcement. Please feel free to continue
    discussing the talk on IRC or the pad.
- [[*Present talk][Present next talk]]
  
** Break time
:PROPERTIES:
:CUSTOM_ID: break
:END:

- CHECK marks the last talk as done. =conf-end-current-talk=
- CHECK stops and restarts the Toobnix restreaming process, and re-checks the stream
- CHECK uses =conf-upcoming-add-subtree= to add the afternoon talks to upcoming.org
- HOST doublechecks network transfer limit and server health
  
** End of stream
:PROPERTIES:
:CUSTOM_ID: end
:END:

- CHECK stops ffmpeg process for Youtube
- CHECK stops ffmpeg process for Toobnix
- CHECK stops ffmpeg process for main-480p
- STREAM stops streaming
  
** In case of...
:PROPERTIES:
:CUSTOM_ID: exceptions
:END:
*** Last-minute prerecording submission
:PROPERTIES:
:CUSTOM_ID: last-minute-prerec
:END:

- CHECK will copy it from the FTP upload server to ${protected} and name it appropriately.
- CHECK will notify STREAM with the scp command and the mpv command so that STREAM can choose.
  
*** Last-minute caption update
:PROPERTIES:
:CUSTOM_ID: last-minute-captions
:END:

- CHECK uploads the --main.vtt file to ${protected}
- CHECK notifies STREAM via ${upcoming}
- STREAM uses the provided commands to download the VTT file and load it into MPV with =--sub-file=
  
*** Speaker has not checked in
:PROPERTIES:
:CUSTOM_ID: missing
:END:
- Let the previous talk run a little longer for Q&A; end at least in time for the prerec
- After the previous Q&A wraps up, play the prerec
- [? still not around after prerec finishes]
  - HOST: Speaker might be having some difficulty connecting, but we'll collect your questions on the pad and send them afterwards.
  - Can play next prerec a few minutes early
*** Speaker does not have a prerec and has not checked in
:PROPERTIES:
:CUSTOM_ID: really-missing
:END:
- Let the previous talk do live Q&A/demo if ready
- Close to the time of the missing talk:
  - See if any of the previous speakers want to be set up for an impromptu talk/extension in a BBB room, just in case
  - HOST: The next speaker might be having some difficulty connecting. In the meantime, let's...
    - OR:
      - highlight ongoing discussions
      - invite another speaker for an impromptu extension; mplsCorwin will keep a list of possible speakers who are still active
      - replay a short prerec
      - let mplsCorwin or zaeph fill in
*** Speaker whose talk was reallocated shows up and has a prerec
:PROPERTIES:
:CUSTOM_ID: reallocated-prerec
:END:
- CHECK copies it to ${protected}
- Plan to play prerec at the end of the day, or in any gaps if a live talk falls through
*** Speaker whose talk was reallocated shows up and wants to do it live
:PROPERTIES:
:CUSTOM_ID: reallocated-live
:END:
- See if there's enough time if buffers are shuffled back; if so, set up for a live presentation
- Check for alternate stream volunteers
- [? not enough time] Offer to set up a BBB room for recording or to
  accept a prerecording afterwards, then include it on the site and in
  post-conference communication
*** Alternate stream volunteer wants to stream  (nice to have, at risk)
:PROPERTIES:
:CUSTOM_ID: alternate
:END:
- CHECK gives ALTERNATE the BBB room URL for the talk they are interested in
- ALTERNATE starts streaming to assigned end point
- CHECK confirms stream
- CHECK updates ${stream-status}
- CHECK notifies STREAM and HOST
  - After prerec plays:
    - HOST: This talk has an extended demo/Q&A. You can go to ${alternate-url} to watch it, and we'll post a recording afterwards.
    - HOST sends ${alternate-url} to IRC: Alternate stream for ${title}: ${alternate-url}
- ALTERNATE notifies #emacsconf-org when the stream is done.
- CHECK updates ${stream-status} to note that the alternate stream is finished.

  FFMPEG process for sending the desktop and audio to the $CONFALT mountpoint on Linux with X11 and Alsa:

  1. Set the CONFALT environment variable to icecast://user:password@live0.emacsconf.org:8000/alt.webm
  2. Install pavucontrol if you don't have it already.
  3. Start the following command ([[file:playbook/stream-desktop-and-audio.sh]]:
     #+begin_src sh :eval no :tangle playbook/stream-desktop-and-audio.sh
     while true; do ffmpeg -loglevel 0 -ar 48000 -f alsa -channels 2 -sample_rate 48000 -i default -re -video_size 1280x720 -framerate 25 -f x11grab -i :0.0  -cluster_size_limit 2M -cluster_time_limit 5100 -content_type video/webm -c:v libvpx -b:v 1M -crf 30 -g 125 -deadline good -threads 4  -f webm $CONFALT; done
     #+end_src
  4. Use pavucontrol to set the recording source for the ffmpeg
     command to be the audio monitor, so you get system output as
     well.
     - OR:
       - [? splitting audio] [[#sound][Set up sinks for sound]]
       - [? same audio]
     - Set up audio monitor as the input for FFMPEG
     - MPV goes to MPV sink, browser goes to recording sink, FFMPEG takes in recording monitor

*** Pad malfunction or mess-up
:PROPERTIES:
:CUSTOM_ID: pad-broken
:END:

- PAD resets the pad using https://etherpad.wikimedia.org/p/emacsconf-2021/timeslider
- [? still not recovered]
  - PAD reimports the pad from backup
    
*** CRDT malfunctions
:PROPERTIES:
:CUSTOM_ID: crdt-broken
:END:

- HOST notifies CHECK and tries reconnecting
- [? still doesn't work]
  - Switch to backup Etherpad
    
*** Conduct guidelines issue
:PROPERTIES:
:CUSTOM_ID: conduct
:END:

- HOST addresses it (on-camera if needed) with a reminder and/or a kick or a ban
  
*** CHECK is unavailable
:PROPERTIES:
:CUSTOM_ID: check-gone
:END:

- HOST does check-ins
- HOST refers to conf.org for Q&A preference etc.
- STREAM checks conf.org for prerec filenames etc.
- Dropped goals:
  - Publishing recordings ASAP
  - Updating schedule/wiki on the fly
    
*** HOST is unavailable
:PROPERTIES:
:CUSTOM_ID: host-gone
:END:

- STREAM joins the BBB room and streams directly from there (or streams the prerecs).
- STREAM will do the hosting.
      
*** Technical issues during a live presentation
:PROPERTIES:
:CUSTOM_ID: live-issues
:END:

- HOST tries to contact the speaker
- [? back on track]
  - [? can be squeezed into remaining time]: Continue
  - [? need extra time]: CHECK fiddles with buffer of following talks in conf.org and updates schedule
  - [? need too much extra time (ex: 10min)]: HOST acknowledges
    technical issues and says we may be able to follow up after the
    conference
- [? can't resume]: HOST acknowledges technical issues and says we may
  be able to follow up after the conference
  
*** Big technical issues with streaming
:PROPERTIES:
:CUSTOM_ID: stream-issues
:END:

- HOST notifies #emacsconf and #emacsconf-org and adds a note at the top of the ${pad}.
- HOST updates the 2021.md wiki page
- CHECK publishes prerecordings
  - media.emacsconf.org
  - wiki
  - Toobnix
  - Peertube
- STREAM e-mails the mailing list
  
*** live0 can't handle the load or is close to network transfer limit
:PROPERTIES:
:CUSTOM_ID: network
:END:

- OR:
  - Redirect some viewers via asking in #emacsconf:
    - watch via main-480p
    - watch via Toobnix (if we can get that working)
  - Consider dropping the restream to Toobnix (lower audience?) or to Youtube
  - Add additional node to Linode account for shared transfer pool (TODO: doublecheck)
    
* Before the conference
:PROPERTIES:
:CUSTOM_ID: before
:END:
** Help speakers with tech checks
:PROPERTIES:
:CUSTOM_ID: tech-checks
:END:
- Explain process
- Test audio, webcam, screensharing, collaborative pad
  - Music demos and other things that use system audio will need to be prerecorded (or done through virtual loopback device, maybe? Technical risk.)
  - Multi-monitor setups might not be handled well by BBB; share window instead of desktop
- Check if comfortable checking into IRC: chat.emacsconf.org/?join=emacsconf,emacsconf-org
- Get IRC nick, phone number for emergency contact, store in private wiki
- Try to record name pronunciation
- Encourage webcam for Q&A, although make it clear that it's totally optional
- Possible picture-in-picture approach to maximize screen real estate
** Experiment with setup to allow MPV / BBB sound isolation
:PROPERTIES:
:CUSTOM_ID: sound
:END:

Set MIC and OUTPUT appropriately.

#+begin_src sh :eval no
MIC=alsa_input.usb-046d_0819_A68D6BE0-02.mono-fallback
OUTPUT=alsa_output.pci-0000_00_1b.0.analog-stereo
if [ "$1" == "-u" ]; then
  pactl unload-module module-loopback
  pactl unload-module module-null-sink
else
  pacmd load-module module-null-sink sink_name=recording sink_properties=device.description=recording channels=2
  pacmd load-module module-null-sink sink_name=mpv sink_properties=device.description=mpv
  pacmd load-module module-loopback source=$MIC sink=recording
  pacmd load-module module-loopback source=mpv.monitor sink=recording
  pacmd load-module module-loopback source=recording.monitor sink=$OUTPUT
fi
#+end_src

You can probably then use =pavucontrol= (choose All Streams instead of
Applications on the Playback tab) or something like the following to
redirect the mpv output to the mpv sink.

#+begin_src sh :eval no
pacmd list-sink-inputs  # notice the ID for the process you want to redirect
pacmd move-sink-input 1 mpv
#+end_src