diff options
Diffstat (limited to 'organizers-notebook')
| -rw-r--r-- | organizers-notebook/index.org | 991 |
1 files changed, 991 insertions, 0 deletions
diff --git a/organizers-notebook/index.org b/organizers-notebook/index.org index 297c0889..17a60f76 100644 --- a/organizers-notebook/index.org +++ b/organizers-notebook/index.org @@ -76,9 +76,15 @@ Create linked pages - [ ] submit page *** How to mark pages as drafts +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-draft-cfp-how-to-mark-pages-as-drafts +:END: Put inside double square brackets: =!template id=pagedraft= *** Previous years +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-draft-cfp-previous-years +:END: [[file:~/proj/emacsconf/wiki/2023/cfp.org][2023]] @@ -120,7 +126,38 @@ Put inside double square brackets: =!template id=pagedraft= (especially for Pacific time). Extending beyond 5pm puts strain on European organizers and volunteers, though. - Put the speaker introduction right after the talk description for easier web page copying. +** Announce +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-announce +:END: + +- Post CFP to emacsconf-discuss, reddit.com/r/emacs, and in Emacs News +- Submit to https://foss.events/ + +*** Promote the conference +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-announce-promote-the-conference +:END: + +Submit to https://foss.events/ + +Maybe it would be good to get in touch with podcasts like + +- [[https://systemcrafters.net/][System Crafters]] +- [[https://www.youtube.com/playlist?list=PLbFVcOQ-YH_LRP687N0YeN78YZmBp5wqF][This Week in Linux]] +- [[https://linuxunplugged.com/][Linux Unplugged]] +- [[http://asknoahshow.com/][Ask Noah]] +- [[https://linuxafterdark.net/][Linux After Dark]] +- [[https://anonradio.net/][Lispy Gopher Show]] + +etc to give a heads up on EmacsConf before it +happens and also mention to them when videos are +available to spark more interest. + ** Process proposals +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-process-proposals +:END: 1. In the private repository, create a conf.org. 2. Set ~emacsconf-org-file~ to its location. @@ -148,6 +185,9 @@ When proposals come in: TRACK, TIMEZONE, CATEGORY, DATE_SUBMITTED, PUBLIC_EMAIL ** Accept proposals +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-accept-proposals +:END: Setup: @@ -174,3 +214,954 @@ For each talk: - Q_AND_A: pad - Create draft-schedule section in the public organizers notebook for the year - Add missing items to plan +** Set up backstage and upload +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-set-up-backstage-and-upload +:END: + +See [[emacsconf-ansible:README.org]]. + +Lessons learned: + +- Next year, let's get upload.emacsconf.org in place as the domain name so that people don't try to FTP files to it. + +** Post the schedule +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-post-the-schedule +:END: + +- [[elisp:(setq emacsconf-publishing-phase 'schedule)]] +- elisp:emacsconf-publish-info-pages +- elisp:emacsconf-publish-schedule-org-files +- elisp:emacsconf-ical-generate-all +- Update talks.md to refer to watch.md as well +** Process uploaded files +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-process-uploaded-files +:END: + +Basic setup: + +1. Set up the media server and symlinks: [[emacsconf-ansible:README.org]] (Media). +2. Set up the prerec processing areas on res: [[emacsconf-ansible:README.org]] (Processing prerecs). +2. Set up the caption processor on res: [[emacsconf-ansible:README.org]] (Captioning). + +Process: + +1. Review uploaded files with elisp:emacsconf-upload-dired +2. Open the JSON for the file you want to process and use elisp:emacsconf-upload-copy-from-json . + If more than one file uses the same extension, provide a suffix to disambiguate them. +3. On res: + 1. Create a directory ~/current/files/$slug. + 2. Copy the uploaded file from ~/cache as =$video_slug--original.$extension=, or use =rename-original.sh $slug $file=. + 3. Call =process-prerec.sh $file=. +4. When the screens are finished, do locally: + 1. update-cache on my computer. + 2. elisp:emacsconf-publish-cache-video-data or elisp:emacsconf-cache-all-video-data + 3. elisp:emacsconf-publish-backstage-index +** Generate assets +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-generate-assets +:END: +*** In-between pages +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-generate-assets-in-between-pages +:END: +- Make the $year/assets/in-between directory and copy the template.svg from the previous year. +- elisp:emacsconf-stream-generate-in-between-pages +*** Intros +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-generate-assets-intros +:END: +- In an empty VTT file, elisp:emacsconf-subed-intro-subtitles +- Record the audio and get captions +- Splice together into a file that will be processed with subed-record-compile-video. +** Prepare the infrastructure +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-prepare-the-infrastructure +:END: + +Checklist: + +- [ ] IRC + - [ ] Manual announcements + - [ ] Automated announcements +- [ ] Web conference +- [ ] Streaming assets + - [ ] Overlays + - [ ] Intros +- [ ] OBS +- [ ] VNC +- [ ] Publishing to the wiki +- [ ] Publishing to the media server +- [ ] Etherpad +- [ ] Mumble +- [ ] Icecast + +** Getting ready for captions +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-captions +:END: + +Onboarding: + +- Add captioner to conf.org with the ~:caption:~ tag, under a Volunteers heading with the ~volunteer~ tag. Make sure the ~EMAIL~ property is set. +- Add captioner to emacsconf-org mailing list. +- Send backstage information elisp:emacsconf-mail-backstage-info-to-volunteer . (Can be sent to all captioning volunteers with elisp:emacsconf-mail-backstage-info-to-captioning-volunteers) +** Getting ready for the conference +:PROPERTIES: +:CUSTOM_ID: pre-conf +:END: + +elisp:emacsconf-pad-prepopulate-shift-hyperlists +*** TODO Send check-in details, schedule update, etc. +SCHEDULED: <2024-11-30 Sat> +:PROPERTIES: +:CREATED: [2024-11-27 Wed 17:34] +:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-send-check-in-details-schedule-update-etc +:END: + +*** Two days before the conference +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-two-days-before-the-conference +:END: + +Go through the :tminustwo: tags. + +*** One day before the conference +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-getting-ready-for-the-conference-one-day-before-the-conference +:END: + +Go through the :tminusone: tags. + +- Set ~emacsconf-publishing-phase~ to ~conference~. +- Set emacsconf-publish-include-pads to t and republish the info pages. + +** During the conference +:PROPERTIES: +:CUSTOM_ID: conf +:END: +elisp:emacsconf-pad-open-shift-hyperlist +** After the conference +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference +:END: +*** Capture stats +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-capture-stats +:END: +2024 stats + +Saturday: +gen: 177 peak + 14 peak lowres +dev: 226 peak + 79 peak lowres + +Sunday: +gen: 89 peak + 10 peak lowres + +meet peak 409% CPU (100% is 1 CPU), average 69.4% +front peak 70.66% CPU (100% is 1 CPU) +live peak 552% CPU (100% is 1 CPU) average 144% +res peak 81.54% total CPU (each OBS ~250%), mem 7GB used + +so far we have used 1.2TB of transfer this month +*** Back up the data +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data +:END: +**** Icecast +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-icecast +:END: + +rsync -avze ssh live:'/data/emacsconf-2024-*' media:~/2024 + +rsync -avze ssh live:'/data/emacsconf-2024-*' . + +**** BigBlueButton +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-bigbluebutton +:END: + +rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/ + +ssh root@66.175.208.243 'dd if=/dev/sda bs=5M ' | dd of=bbb-img-2024-12-08.img status=progress + +The img can be loaded with +sudo losetup --find --show bbb-img-2024-12-08.img +sudo mount /dev/loop10 test +cd test +***** Reviewing all the Q&A durations and chat messages +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-back-up-the-data-bigbluebutton-reviewing-all-the-q-a-durations-and-chat-messages +:END: +#+begin_src emacs-lisp :eval no +(let ((default-directory "/home/sacha/proj/emacsconf/2024/backups/bigbluebutton/published/video")) + (mapconcat + (lambda (dir) + (let ((chat (xml-parse-file + (expand-file-name "video.xml" dir))) + (metadata (xml-parse-file + (expand-file-name "metadata.xml" dir)))) + (concat + "- " + (dom-text (car (dom-by-tag metadata 'meetingName))) " (" + (format-seconds "%.2h:%z%.2m:%.2s" + (/ (string-to-number (dom-text (dom-by-tag metadata 'duration))) + 1000)) + ")" + "\n" + (if (dom-by-tag chat 'chattimeline) + (mapconcat (lambda (node) + (concat " - " (dom-attr node 'name) ": " + (dom-attr node 'message) "\n")) + (dom-by-tag chat 'chattimeline) + "") + "") + "") + ) + ) + (directory-files "." nil "-")) + ) +#+end_src + +- Copy IRC logs to cache and therefore to backstage: (ex: 2024-12-07-emacsconf-gen.txt) + +*** Combine the IRC logs and the pads, and add them to the wiki +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-combine-the-irc-logs-and-the-pads-and-add-them-to-the-wiki +:END: +(did we e-mail the speakers before or after?) +*** Process the Q&A recordings and live presentations +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-process-the-q-a-recordings-and-live-presentations +:END: + +- Save answers videos as --answers.webm. +- From the pad file: emacsconf-extract-copy-pad-to-wiki +- update cache +- Set ~QA_PUBLIC~ property to t as each Q&A gets processed. +- change status to ~TO_INDEX_QA~ +- emacsconf-publish-media-files-on-change +- emacsconf-publish-info-pages-for-talk + +Indexing + +- emacsconf-extract-insert-note-with-question-heading +- emacsconf-subed-make-chapter-file-based-on-comments +- change status to ~TO_CAPTION_QA~ + +Also copy the YouTube comments and IRC comments +emacsconf-extract-irc-anonymize-log + +*** Send thank-you emails to the speakers +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-send-thank-you-emails-to-the-speakers +:END: +*** Notify emacs-discuss that the Q&A is up +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-notify-emacs-discuss-that-the-q-a-is-up +:END: + +https://lists.gnu.org/archive/html/emacsconf-discuss/2024-12/msg00000.html + +#+begin_example +Hello, everyone! + +https://emacsconf.org/2024/talks and https://media.emacsconf.org/2024 +should have the talks and Q&A sessions now, hooray, including chapter +indices for the Q&A. I've also copied over notes from Etherpad and IRC +on the talk pages, and I've e-mailed them to the speakers so they can +update the wiki directly or send in additional answers. + +BigBlueButton audio mixing is kinda iffy, so I'm holding off on +uploading the Q&A videos (and the live talk for secrets) to +YouTube/Toobnix until Leo and other volunteers have a listen to see +whether the audio needs to be tweaked, and maybe even to do some kind of +normalization. Unfortunately, recordings with separate audio tracks per +participant aren't available--it would be cool to figure that out next +year--so some talks might need some manual processing. (Please feel free +to volunteer if that's up your alley!) In the meantime, you can check +out the Q&A videos from the talk pages on the wiki. Please let us know +if there's anything that needs tweaking. + +Enjoy! + +Sacha +#+end_example + +*** Write a report +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-write-a-report +:END: +Sample: https://emacsconf.org/2023/report/ +*** Send thank-you emails to the volunteers +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-send-thank-you-emails-to-the-volunteers +:END: + +emacsconf-mail-template-ask-volunteer-for-mailing-address + +*** Process the lessons learned +:PROPERTIES: +:CUSTOM_ID: phases-and-lessons-learned-after-the-conference-process-the-lessons-learned +:END: +* General infrastructure +:PROPERTIES: +:CUSTOM_ID: general-infrastructure +:END: +** Linode instance sizes +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-linode-instance-sizes +:END: +*** Production +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-linode-instance-sizes-production +:END: +- front0 shared Linode 32GB +- live0 shared Linode 64GB +- meet Dedicated CPU 16GB +*** Dormant +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-linode-instance-sizes-dormant +:END: +Resize disk to 25GB, copying large files to media if needed + +- nanode front0 +- nanode live0 +- delete meet after downloading all the recordings, or resize down to nanode +** File uploads +:PROPERTIES: +:CUSTOM_ID: upload +:END: + +[[https://github.com/psi-4ward/psitransfer][psi-4ward/psitransfer: Simple open source self-hosted file sharing solution]] + +Before the conference: + +1. Set ~upload_enabled~ to ~true~ in ~prod-vars.yml~ and run ~ansible-playbook -i inventory.yml prod-playbook.yml --tags upload~. +2. Confirm that https://upload.emacsconf.org has the upload interface and works with the password. +3. Use ~emacsconf-mail-template-to-all-groups~ and ~emacsconf-mail-upload-and-backstage-info~ to e-mail the upload information to all speakers. + +After the conference: + +1. Set ~upload_enabled~ to ~false~ in ~prod-vars.yml~ and run ~ansible-playbook -i inventory.yml prod-playbook.yml --tags upload~. +2. Confirm that https://upload.emacsconf.org shows the in-between page. + +** Icecast +:PROPERTIES: +:CUSTOM_ID: icecast +:END: + +Success: You can use [[#obs-instructions][OBS+VNC to record]], and the results can be viewed by mpv. +- [X] Gen +- [X] Dev +Also, recordings are available in /data. + +New year: + +1. Reprovision with + #+begin_src sh :eval no + ansible-playbook -i inventory.yml prod-playbook.yml --tags stream + #+end_src + so that the year is updated in the configuration. +2. SSH to live0.emacsconf.org and run ~/etc/init.d/emacsconf stop~. Confirm that icecast has stopped with ~ps | grep emacsconf~; if not, kill the process. + Start it again with ~/etc/init.d/emacsconf start~ +3. To test, you can use FFmpeg like this for the gen stream: + #+begin_src sh + ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \ + -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ + -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/gen.webm" + #+end_src + For dev: + #+begin_src sh + ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \ + -c:v libvpx -b:v 500K -crf 25 -b:a 128k \ + -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/dev.webm" + #+end_src + and then view it with + #+begin_src sh + mpv https://live0.emacsconf.org/gen.webm + #+end_src + +** Watching pages +:PROPERTIES: +:CUSTOM_ID: watch +:END: + +Success: You can watch at https://live.emacsconf.org. +- [X] https://live.emacsconf.org goes to the current year's page. +- [X] gen stream works in the browser +- [X] dev stream works in the browser +- [X] Schedule is correct + +New year: +1. ansible-playbook -i inventory.yml prod-playbook.yml --tags live +2. Republish with elisp:emacsconf-publish-watch-pages. +3. Update front:/var/www/live.emacsconf.org. + +Set up test streams (see ffmpeg instructions from [[#icecast][Icecast]]) and then: + +** VNC +:PROPERTIES: +:CUSTOM_ID: vnc +:END: + +We use VNC to connect to the X servers on res.emacsconf.org so that we can stream from it. + +Success: +- [X] Confirm that you can connect to emacsconf-gen via VNC +- [X] Confirm that you can connect to emacsconf-dev via VNC + +Setting up +- elisp:emacsconf-publish-res-index + +*** Instructions +:PROPERTIES: +:CUSTOM_ID: vnc-instructions +:END: + +NOTE: VNC+OBS doesn't work well if you have a window manager that +automatically resizes windows, like i3. Please configure your window +manager so that the VNC window is not resized. + + 1. Install a VNC viewer on your system (ex: tigervnc-viewer). + + 2. Set up your local environment: + + - gen: ~export TRACK=gen; export TRACK_PORT=5905; export SSH_PORT=46668~ + - dev: ~export TRACK=dev; export TRACK_PORT=5906; export SSH_PORT=46668~ + + 3. Copy the password: + + scp emacsconf-$TRACK@res.emacsconf.org:~/.vnc/passwd vnc-passwd-$TRACK -p $SSH_PORT + + 4. Forward your local ports and connect via VNC viewer to the + appropriate forwarded port from your laptop: + + #+begin_example + ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT & + sleep 5 # Give it time to establish the tunnels + xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & + #+end_example + +If you get the following error: + +#+begin_example +channel 2: open failed: connect failed: Connection refused +CConn: End of stream +CConn: The connection was dropped by the server before the session could + be established. +#+end_example + +then the VNC server hasn't started yet. You can start it with + +#+begin_src sh :eval no +ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc +#+end_src + +and then connect with: + +#+begin_src sh :eval no +xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK & +#+end_src + +** Streaming with OBS +:PROPERTIES: +:CUSTOM_ID: obs +:END: + +Success: Confirm that you can stream +- [X] gen +- [X] dev + +New year: reprovision with + +#+begin_src sh :eval no +ansible-playbook -i inventory.yml prod-playbook.yml --tags obs +#+end_src + +so that the year is updated in the shell scripts. + +*** Instructions +:PROPERTIES: +:CUSTOM_ID: obs-instructions +:END: + +1. [[#vnc-instructions][Connect to the VNC session for the track.]] + +2. Start *recording* (not streaming). If you don't see OBS when you connect, it's probably on workspace 2, so you can switch with Alt-2. If you still don't see it there, you can open a terminal with Alt-Enter and then run ~track-obs~. After you start recording, confirm that it is now broadcasting to the stream. + +3. Verify with MPV on your local system: + + #+begin_example + mpv https://live0.emacsconf.org/$TRACK.webm & + #+end_example + +*** Icecast +:PROPERTIES: +:CUSTOM_ID: icecast +:END: + +Success: You can use [[#obs-instructions][OBS+VNC to record]], and the results can be viewed by mpv. +- [ ] gen +- [ ] dev + +New year: reprovision with + +#+begin_src sh :eval no +ansible-playbook -i inventory.yml prod-playbook.yml --tags stream +#+end_src + +so that the year is updated in the configuration. + +This is on live.emacsconf.org and can be restarted with =/etc/init.d/emacsconf restart=. +** Shell scripts +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-shell-scripts +:END: + +Success: From the commandline, hosts can: +- [X] play just the intro: intro $SLUG +- [X] play a file without an intro: play $SLUG +- [X] play a file with an intro: play-with-intro $SLUG (hmm, this didn't show the subtitles) + ah, it's because the intros need to be in the cache dir? +- [X] open the Q&A for a talk: handle-qa $SLUG + - [X] open the pad: pad $SLUG + - [X] open BBB: bbb $SLUG +- [ ] do the whole thing: handle-session $SLUG +- [ ] rebroadcast the other stream: + - rebroadcast gen + - rebroadcast dev +- [ ] control background music: music, stop-music + +Setup: +- OBS must be streaming from the stream you're testing. [[#obs][Streaming with OBS]] +- You may want to generate test assets and copy them over to the assets/stream directory. +- Generate all the overlays and copy them to assets/overlays +- Set up the assets/music directory +** Backstage +:PROPERTIES: +:CUSTOM_ID: backstage +:END: + +Update this with elisp:emacsconf-publish-backstage-index + +Explanation of files: + +| --original.mp4/mov/webm | original file as uploaded by the speaker | +| --silence.mp4/mov/webm | silence sample if uploaded by speaker | +| --reencoded.webm | converted to .webm format and compressed | +| --reencoded.opus | audio only | +| --normalized.opus | normalized audio | +| --reencoded.vtt | WhisperX captions, WebVTT format | +| --reencoded.json | WhisperX word data | +| --reencoded.srt | WhisperX captions, SRT format | +| --reencoded.txt | WhisperX captions, plain text | +| --main.vtt | edited captions | +| --main--chapters.vtt | chapter markers | +| --main.txt | captions as text (optional) | +| --main.webm | version for streaming based on the most recent resources | + +Other files might also have been uploaded by the speaker, such as slides or notes. + +** IRC web client +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-irc-web-client +:END: + +Success: +- [X] You can join from https://chat.emacsconf.org +- [X] You can join from https://live.emacsconf.org and the channels end up as the latest ones + +Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors. + +Message I've posted to support@libera.chat: + +Thank you for running libera.chat. We're looking forward to using IRC +(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming +EmacsConf (Dec 7-8, https://emacsconf.org). In the past, we've needed to +coordinate with Libera staff to increase the number of connections +allowed from chat.emacsconf.org during conference days in order to avoid +connection exhaustion errors. Is that something we can plan now or would +you like me to bring it up a few days before the conference? +** Etherpad +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-etherpad +:END: + +Success: + +- [X] pad.emacsconf.org redirects to this year's entry +- [X] Each talk has its own Etherpad + +Lessons learned: + +- After 2024: Added pronouns and pronunciation to the pad template since that's what the hosts will be looking at +** Publishing media to the server and to the wiki +:PROPERTIES: +:CUSTOM_ID: media +:END: + +Start of year: + +1. Set =media_protect_root= to true in Ansible =group_vars/all.yml=. +2. =ansible-playbook -i inventory.yml prod-playbook.yml --tags media= +3. Generate the index with =emacsconf-publish-update-media= + +Confirm by setting a submitted talk to =PLAYING= and testing with +elisp:emacsconf-publish-media-files-on-change . The public media +directory should have the files and the entry should be in the index. +Switching it back to =TO_STREAM= and calling +elisp:emacsconf-publish-media-files-on-change should remove it. + +*** TODO Switch public media to unprotected root before the conference :tminustwo: +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-publishing-media-to-the-server-and-to-the-wiki-switch-public-media-to-unprotected-root-before-the-conference +:END: + +1. Clear public media directory. +2. Set =media_protect_root= to false in Ansible =group_vars/all.yml=. +3. =ansible-playbook -i inventory.yml prod-playbook.yml --tags media= + +You can generate the index with =emacsconf-publish-update-media=. +** Automated IRC announcements +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-automated-irc-announcements +:END: + +Success: + +- [ ] When a talk starts playing, it is announced in the relevant channel +** BigBlueButton +:PROPERTIES: +:CUSTOM_ID: bbb +:END: + +ssh orga@media.emacsconf.org "~/bin/bbb-before sat-open" +https://media.emacsconf.org/2024/current/bbb-sat-open.html + +ssh orga@media.emacsconf.org "~/bin/bbb-open sat-open" + +After restarting, and in case of "Firefox can’t establish a connection to the server at wss://bbb.emacsverse.org/graphql." + +ssh root@bbb +cd greenlight-v3 +docker compose restart + +Sizes: + +- dormant: 1 GB nanode +- testing: 4 core 8 GB shared CPU +- production: 8 core 16 GB dedicated CPU (roughly half CPU load for 107 simultaneous users, 2024) + +*** Installing BigBlueButton + +1. Create the instance. + - Linode 4core 8GB shared CPU + - Image: Ubuntu 22.04 +2. Update the DNS for bbb.emacsverse.org with its IP address. +3. Download the BBB installer and make it executable. +4. ufw allow 16384:32768/udp +5. Run the BBB installer: ~./bbb-install.sh -v jammy-300 -s bbb.emacsverse.org -e emacsconf@sachachua.com -g~ +6. ~sudo apt install bbb-playback-video~ and then follow the instructions at [[https://docs.bigbluebutton.org/3.0/administration/customize/#install-additional-recording-processing-formats][Server Customization | BigBlueButton]] . + - /usr/local/bigbluebutton/core/scripts/bigbluebutton.yml: + #+begin_example + steps: + archive: 'sanity' + sanity: 'captions' + captions: + - 'process:presentation' + - 'process:video' + 'process:presentation': 'publish:presentation' + 'process:video': 'publish:video' + #+end_example + - systemctl restart bbb-rap-resque-worker.service +7. docker exec -it greenlight-v3 bundle exec rake admin:create['name','email','password'] + + + +*** System audio limitations +:PROPERTIES: +:CUSTOM_ID: bbb-audio-limitations +:END: + +Sharing system audio by changing the audio device to "Built-in Audio Analog Stereo" is very choppy, only gets captured in the recording if I'm using Firefox (not Chrome) AND there's at least one listener, and doesn't allow simultaneous narration from the speaker. + +Using pulseaudio to change the app's recording device to the audio monitor results in smoother audio (because it's pretending to be the microphone), but still doesn't allow simultaneous narration from the speaker. + +Combining microphone + system audio into a new virtual microphone source is probably more complicated than we want to walk speakers through. + +BBB says sharing system audio works only if you use Chrome on Mac OS X or Microsoft Windows (https://support.bigbluebutton.org/hc/en-us/articles/1500005315642-Can-I-play-a-video-using-screen-sharing). I tested it on Linux and sharing system audio is not an option. If you happen to know of any more elegant way to do this, I'd love to hear it. + +Related: https://github.com/bigbluebutton/bigbluebutton/issues/8632 + + +*** Creating talk BBB rooms +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-bigbluebutton-creating-talk-bbb-rooms +:END: + +- =docker exec -it greenlight-v3 /bin/bash= +- =bundle exec rails console= +- =user_id = User.find_by_email("sacha@sachachua.com").id= + +#+begin_src emacs-lisp :results replace +(mapconcat (lambda (group) + (format + "Room.create(user_id: user_id, name: \"%s - %s\")\n" + (plist-get (cadr group) :speakers) + (string-join (mapcar (lambda (talk) (plist-get talk :slug)) + (cdr group))))) + (emacsconf-mail-groups (emacsconf-active-talks (emacsconf-get-talk-info))) + "") +#+end_src + +- elisp:emacsconf-publish-bbb-static-redirects + +Print out the room IDs with + +#+begin_src ruby +Room.all.each { |x| puts x.friendly_id + " " + x.name }; nil +#+end_src + +It's possible to change the friendly_id and then use ~x.save!~. + +*** Setting up moderator access codes +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-bigbluebutton-setting-up-moderator-access-codes +:END: +*** Backing up BBB :backup: +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-bigbluebutton-backing-up-bbb +:END: + +rsync -avze ssh root@bbb:/var/bigbluebutton/ bigbluebutton/ + +ssh root@bbb 'tar zcvf - /var/bigbluebutton /etc/bigbluebutton /root/greenlight-v3 /usr/local/bigbluebutton /usr/share/bbb-web' > bbb-backup-$(date "+%Y-%m-%d").tar.gz + +ssh root@bbb 'dd if=/dev/sda bs=5M ' | dd of=bbb-img-$(date "+%Y-%m-%d").img status=progress + +*** Spinning BBB up again + +[2025-01-22 Wed] ... actually, this ran into some issues with the certificate, so I'm just going to do a reinstall. + +Previous notes + +1. Create a Nanode. +2. Update the DNS for bbb.emacsverse.org with its IP address. (dns.he.net) +3. Reboot into Rescue mode. In Lish: + #+begin_src sh :eval no + passwd # set interactively + sed -i -e 's/#PasswordAuthentication yes/PasswordAuthentication yes/g' /etc/ssh/sshd_config + echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config + service ssh start + #+end_src +4. From the backup directory: + gunzip -c bbb-img-2024-12-08.img.gz | ssh root@bbb.emacsconf.org "dd of=/dev/sda bs=4M status=progress oflag=direct iflag=fullblock" +5. After I resized to 4core 8GB RAM, I couldn't log in with SSH, so I used the Linode shell. ~sshd -t~ said no host keys available. Fix: + #+begin_src sh :eval no + ssh-keygen -A + service ssh --full-restart + #+end_src +6. ./bbb-install.sh -v jammy-300 -s bbb.emacsverse.org -e emacsconf@sachachua.com + +Resources: + +- https://overto.eu/posts/gunzip-into-dd/ +- https://www.linode.com/community/questions/20386/how-do-i-ssh-to-linode-from-the-finnix-boot-in-order-to-copy-my-local-vm-over-to +- https://github.com/bigbluebutton/bigbluebutton/issues/9485 + +** Manual IRC announcements +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-manual-irc-announcements +:END: + +Success: + +- [X] You can /opall, /conftopic, and /broadcast + +** Low-res stream +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-low-res-stream +:END: +Should be automatic, handled by Icecast +backup on live: +- /usr/local/bin/emacsconf-lowres-gen-on-connect +- /usr/local/bin/emacsconf-lowres-dev-on-connect +** Upload talks to YouTube +SCHEDULED: <2024-11-30 Sat> +:PROPERTIES: +:CREATED: [2024-11-27 Wed 17:33] +:CUSTOM_ID: general-infrastructure-upload-talks-to-youtube +:END: + +[[https://studio.youtube.com/channel/UCwuyodzTl_KdEKNuJmeo99A/videos/upload?filter=%5B%5D&sort=%7B%22columnType%22%3A%22date%22%2C%22sortOrder%22%3A%22DESCENDING%22%7D][Channel videos]] + +#+begin_src emacs-lisp :results table replace +(seq-keep + (lambda (o) + (when (and (string= (plist-get o :status) "TO_STREAM") + (not (plist-get o :youtube))) + (list + (plist-get o :slug) + (org-link-make-string + (concat "copy:" + ) + "video file") + (org-link-make-string + (format "elisp:(emacsconf-publish-video-description \"%s\" t)" + (plist-get o :slug)) + "desc") + (if (emacsconf-talk-file o "--main.vtt") + (org-link-make-string + (concat "copy:" + (emacsconf-talk-file o "--main.vtt")) + "sub file") + "") + (org-link-make-string + (concat "elisp:(emacsconf-set-property-from-slug \"" + (plist-get o :slug) + "\" \"YOUTUBE\" (read-string \"YouTube URL: \"))" + ) + "save") + (plist-get o :scheduled)))) + (emacsconf-publish-prepare-for-display (emacsconf-get-talk-info))) +#+end_src + +#+RESULTS: +:results: +| shell | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-shell--emacs-as-a-shell--christopher-howard--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "shell" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-shell--emacs-as-a-shell--christopher-howard--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "shell" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-07 Sat 13:55-14:35> | +| writing | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-writing--emacs-writing-studio--peter-prevos--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "writing" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-writing--emacs-writing-studio--peter-prevos--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "writing" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-07 Sat 16:05-16:15> | +| sat-close | [[copy:][video file]] | [[elisp:(emacsconf-publish-video-description "sat-close" t)][desc]] | | [[elisp:(emacsconf-set-property-from-slug "sat-close" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-07 Sat 17:00-17:10> | +| links | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-links--unlocking-linked-data-replacing-specialized-apps-with-an-orgbased-semantic-wiki--abhinav-tushar--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "links" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-links--unlocking-linked-data-replacing-specialized-apps-with-an-orgbased-semantic-wiki--abhinav-tushar--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "links" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 09:10-09:20> | +| blee | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-blee--about-blee-towards-an-integrated-emacs-environment-for-enveloping-our-own-autonomy-directed-digital-ecosystem--mohsen-banan--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "blee" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-blee--about-blee-towards-an-integrated-emacs-environment-for-enveloping-our-own-autonomy-directed-digital-ecosystem--mohsen-banan--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "blee" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 10:30-11:15> | +| hyperbole | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-hyperbole--fun-things-with-gnu-hyperbole--mats-lidell--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "hyperbole" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-hyperbole--fun-things-with-gnu-hyperbole--mats-lidell--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "hyperbole" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 11:30-11:45> | +| pgmacs | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-pgmacs--pgmacs-browsing-and-editing-postgresql-databases-from-emacs--eric-marsden--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "pgmacs" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-pgmacs--pgmacs-browsing-and-editing-postgresql-databases-from-emacs--eric-marsden--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "pgmacs" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 13:40-13:55> | +| literate | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-literate--literate-programming-for-the-21st-century--howard-abrams--main.webm][video file]] | [[elisp:(emacsconf-publish-video-description "literate" t)][desc]] | [[copy:/home/sacha/proj/emacsconf/2024/cache/emacsconf-2024-literate--literate-programming-for-the-21st-century--howard-abrams--main.vtt][sub file]] | [[elisp:(emacsconf-set-property-from-slug "literate" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 14:15-14:35> | +| sun-close | [[copy:][video file]] | [[elisp:(emacsconf-publish-video-description "sun-close" t)][desc]] | | [[elisp:(emacsconf-set-property-from-slug "sun-close" "YOUTUBE" (read-string "YouTube URL: "))][save]] | <2024-12-08 Sun 16:50-17:00> | +:end: + +** Restream to YouTube +:PROPERTIES: +:CUSTOM_ID: general-infrastructure-restream-to-youtube +:END: + + +* Other process notes +:PROPERTIES: +:CUSTOM_ID: other +:END: +** Adding another user to BBB +:PROPERTIES: +:CUSTOM_ID: bbb-user +:END: + +#+begin_src ssh :eval no +ssh bbb.emacsverse.org +docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"] +#+end_src +** When a talk is added after the schedule has already been drafted +:PROPERTIES: +:CUSTOM_ID: other-process-notes-when-a-talk-is-added-after-the-schedule-has-already-been-drafted +:END: +- [X] Add the talk to conf.org +- [X] Add the talk ID to the draft schedule +- [X] Run the draft schedule block +- [X] elisp:emacsconf-schedule-update-from-info +- [X] elisp:emacsconf-publish-talks-json-to-files +- [X] Generate pads with ~ansible-playbook -i inventory.yml prod-playbook.yml --tags pad~ +- [X] elisp:emacsconf-set-file-prefixes +- [X] elisp:emacsconf-update-schedule +- [X] elisp:emacsconf-stream-generate-overlays +- [X] elisp:emacsconf-stream-generate-test-videos +- [X] ~rsync -avze ssh ~/proj/emacsconf/2024/assets/ res:/data/emacsconf/shared/2024/assets/~ +- [X] Send the upload instructions +- [X] Generate the intro text +- [ ] Record the intro + +** When a talk is cancelled +:PROPERTIES: +:CUSTOM_ID: cancel-talk +:END: + +1. Update the status to CANCELLED. Update that specific talk with elisp:emacsconf-publish-info-pages-for-talk. +2. Rejig the schedule if needed, and update with elisp:emacsconf-schedule-update-from-info +3. Update the published schedule with [[elisp:emacsconf-update-schedule]] and push the wiki. +4. Regenerate the intros (before, after). elisp:emacsconf-stream-generate-in-between-pages, then use elisp:subed-record-compile-video in the intros file. +5. Resync the cache directory. +6. Consider updating the screenshots for sat-open and sun-open. + +** When a talk changes title +:PROPERTIES: +:CUSTOM_ID: other-process-notes-when-a-talk-changes-title +:END: + +1. Update the title in conf.org. +2. Delete the FILE_PREFIX property. +3. Call elisp:emacsconf-set-file-prefix-if-needed. +4. Rename existing files with elisp:emacsconf-update-file-prefixes. +5. Update the intro VTT file with the new title. +6. Update the wiki page for the talk. +7. Regenerate the in-between images with elisp:emacsconf-stream-generate-in-between-pages +8. Rerecord and recompile the intro, and recompile the intro for the talk after it. +9. elisp:emacsconf-update-schedule +10. elisp:emacsconf-pad-prepopulate-hyperlists +** When a talk Q&A changes method +:PROPERTIES: +:CUSTOM_ID: change-talk-qa +:END: + +1. Update the talk property. +2. Update the schedule with [[elisp:emacsconf-update-schedule]] +3. Regenerate the intros (before, after). elisp:emacsconf-stream-generate-in-between-pages, then use elisp:subed-record-compile-video in the intros file. +4. Sync the new intros to the /data/emacsconf/shared/$YEAR/assets/intros directory. +5. Regenerate the pads: elisp:emacsconf-pad-prepopulate-all-talks +6. Regenerate talks.json: elisp:emacsconf-publish-talks-json-to-files +** To play the other stream +:PROPERTIES: +:CUSTOM_ID: other-process-notes-to-play-the-other-stream +:END: + +- mpv --profile=full https://live0.emacsconf.org/gen.webm +- Change the channel topic to note that discussion can be in the other channel + +** When a system is down +:PROPERTIES: +:CUSTOM_ID: other-process-notes-when-a-system-is-down +:END: + + +Let people know in #emacsconf and #emacsconf-org + +Update the status page: + +[[file:/ssh:orga@front0.emacsconf.org:/var/www/status.emacsconf.org/index.html]] + +/broadcast <message> may also be helpful +** When we need to bring on a last-minute volunteer +:PROPERTIES: +:CUSTOM_ID: other-process-notes-when-we-need-to-bring-on-a-last-minute-volunteer +:END: + +/backstage nick + +will give them the backstage credentials and links to the index + +** Checking people in +:PROPERTIES: +:CUSTOM_ID: other-process-notes-checking-people-in +:END: + +/checkin nick +/room nick (should be automatic if live, but there just in case) + +** Ansible +:PROPERTIES: +:CUSTOM_ID: other-process-notes-ansible +:END: + +#+begin_src sh :eval no +git clone git@git.emacsconf.org:pub/emacsconf-ansible +#+end_src |