diff options
| -rw-r--r-- | 2022/organizers-notebook.md | 401 | ||||
| -rw-r--r-- | 2022/organizers-notebook/index.org | 23 |
2 files changed, 381 insertions, 43 deletions
diff --git a/2022/organizers-notebook.md b/2022/organizers-notebook.md index 49e6da3d..5419f1ef 100644 --- a/2022/organizers-notebook.md +++ b/2022/organizers-notebook.md @@ -551,7 +551,7 @@ in prod or docker container ### STARTED icecast on live0 -(find-file “/ssh:live|sudo::/etc/icecast2/icecast.xml”) +(find-file &lsquo ;/ssh:live|sudo::/etc/icecast2/icecast.xml’) #### TODO Set up watch/gen-480p @@ -707,19 +707,56 @@ Large and medium might do better on a system with a GPU. I’ll default to t ### DONE Commit subed-tsv so that people can try a cleaner output -### TODO Investigate more granular timestamps for the output from OpenAPI Whisper +### DONE Investigate more granular timestamps for the output from OpenAPI Whisper -<https://stackoverflow.com/questions/73822353/how-can-i-get-word-level-timestamps-in-openais-whisper-asr> +<https://git.emacsconf.org/emacsconf-ansible/tree/roles/caption/templates> ### DONE Upload srv2 from YouTube for word-level +### TODO Move publishing the backstage index to res so that we can trigger it after the files are uploaded + + ### TODO Compare with Google Cloud Speech API ~/code/speech +### TODO E-mail for bringing new captioning volunteers onboard + + +#### Template + +Hi ${name}! + +Thank you for volunteering to help with the captions for ${conf-name} +${year}! Last year, we were able to get almost all the talks captioned +in time for streaming. Participants found them very useful for +understanding different technical terms, names, accents, and so on. +We’d love to be able to pull that off again this year! + +We’ve set up ${backstage} as the backstage area where you can view the +videos and resources uploaded so far. You can access it with the +username "${backstage-user}&lsquo ; and the password ’${backstage-password}&lsquo ;. +Please keep the backstage password and other speakers” talk resources +secret. ${backstage-use}${wrap} + +You can find captioning tips at <https://emacsconf.org/captioning/> . +You can use your favourite subtitle editor, and you can convert it to +whatever format you like. If you prefer to work with plain text, we +can probably even figure out the timestamps afterwards. + +Thank you! + +Sacha Chua + + +### TODO jiwer · PyPI - measure error rate + +<https://pypi.org/project/jiwer/> + + <a id="write-viewing"></a> ## DONE Update viewing instructions and watch pages :sachac: @@ -790,6 +827,12 @@ Other things that aren’t tracked by todo status: - **CHECK\_IN: t:** once the speaker has been checked into the BBB room or IRC +##### TODO Prerec publishing + +Move to public directory and create a link from the backstage directory +Update the wiki page + + <a id="irc-announce"></a> ### DONE Prepare to announce talks on IRC @@ -1044,13 +1087,76 @@ Resources: - <https://xpra.org/> -### TODO corwin: plan an approach to use a centralized OBS/nginix “bouncer” :corwin: +### TODO corwin: plan an approach to use a centralized OBS/nginix &lsquo ;bouncer’ :corwin: ### TODO corwin: document such that someone else could use/fix it :corwin: -### TODO corwin: recruit at least one more person to help operate the “video bouncer” :corwin: +### TODO corwin: recruit at least one more person to help operate the &lsquo ;video bouncer’ :corwin: + + +### SOMEDAY <https://docs.vdo.ninja/> + + +### SOMEDAY Live Streaming using low configuration vps | cheapest streaming server | google cloud ubuntu server + +<https://www.youtube.com/watch?v=3iBYYgkG1eM&t=953> + + +### SOMEDAY ansible\_linode/instance.md at main · linode/ansible\_linode + +<https://github.com/linode/ansible_linode/blob/main/docs/modules/instance.md> + + +### SOMEDAY Use the Linode Ansible Collection to Deploy a Linode | Linode + +<https://www.linode.com/docs/guides/deploy-linodes-using-linode-ansible-collection/> + + +### SOMEDAY Snowmix - The Swiss Army Knife of Open Source Live Video Mixing. + +<https://snowmix.sourceforge.io/Examples/input.html> + + +### SOMEDAY YouTube <https://m.youtube.com> › watch YouTube Live Stream from Free VPS without OBS | No GPU | FFmpeg + +<https://www.google.com/url?sa=t&source=web&rct=j&url=https://m.youtube.com/watch%3Fv%3D9Jj0pVKRaOE&ved=2ahUKEwjPru_TqOv6AhVMkokEHXL9Dm4QtwJ6BAgrEAI&usg=AOvVaw3gyboiaXqCQBflqeWwSf9w> + + +### SOMEDAY YouTube <https://m.youtube.com> › watch VPS Streaming Setup for Free 24x7 YouTube Live Stream Without OBS + +<https://www.google.com/url?sa=t&source=web&rct=j&url=https://m.youtube.com/watch%3Fv%3D1y-DUYiECWQ&ved=2ahUKEwjPru_TqOv6AhVMkokEHXL9Dm4QtwJ6BAgqEAI&usg=AOvVaw17mbCEiFL6dGVY4YEBufcy> + + +### SOMEDAY OBS Studio 26.0 | Hacker News + +<https://news.ycombinator.com/item?id=24633139> + + +### SOMEDAY mviereck/x11docker: Run GUI applications and desktops in docker and podman containers. Focus on security. + +<https://github.com/mviereck/x11docker#sound> + + +### SOMEDAY vcs.fsf.org Git - streamdesktop.git/tree + +<https://vcs.fsf.org/?p=streamdesktop.git;a=tree> + + +### SOMEDAY Is it possible to run a conference using only free software? | Opensource.com + +<https://opensource.com/article/20/5/conference-free-software> + + +### SOMEDAY soonum/hubangl: A free/libre and easy to use streaming software. + +<https://github.com/soonum/hubangl> + + +### SOMEDAY Question / Help - Run OBS on VM in the cloud | OBS Forums + +<https://obsproject.com/forum/threads/run-obs-on-vm-in-the-cloud.122543/> <a id="maybe-projects"></a> @@ -1684,6 +1790,93 @@ Where is the new stuff? … hah, maybe I forgot to push +## Backstage + + +### File suffixes, and what they correspond to + +Per categories, earlier suffixes come earlier in the process. + +<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides"> + + +<colgroup> +<col class="org-left" /> + +<col class="org-left" /> +</colgroup> +<thead> +<tr> +<th scope="col" class="org-left">Suffix</th> +<th scope="col" class="org-left">Description</th> +</tr> +</thead> + +<tbody> +<tr> +<td class="org-left">VIDEO</td> +<td class="org-left"> </td> +</tr> + + +<tr> +<td class="org-left">–original.EXT</td> +<td class="org-left">File as submitted by speaker</td> +</tr> + + +<tr> +<td class="org-left">–reencoded.webm</td> +<td class="org-left">Reencode via ffmpeg incantation</td> +</tr> + + +<tr> +<td class="org-left">–final.webm</td> +<td class="org-left">Broadcast-ready reencode with normalized audio</td> +</tr> +</tbody> + +<tbody> +<tr> +<td class="org-left">AUDIO</td> +<td class="org-left"> </td> +</tr> + + +<tr> +<td class="org-left">–original.EXT</td> +<td class="org-left">Extracted audio track from speaker upload; used for speech-recognition</td> +</tr> + + +<tr> +<td class="org-left">–normalized.opus</td> +<td class="org-left">Normalized audio track</td> +</tr> +</tbody> + +<tbody> +<tr> +<td class="org-left">SUBTITLES</td> +<td class="org-left"> </td> +</tr> + + +<tr> +<td class="org-left">–?(incomplete-)transcript</td> +<td class="org-left">Transcript provided by speaker</td> +</tr> + + +<tr> +<td class="org-left">–main.EXT</td> +<td class="org-left">Broadcast-ready reencode; different formats for different uses</td> +</tr> +</tbody> +</table> + + <a id="other"></a> # Other tasks and processes @@ -1821,7 +2014,7 @@ We tried using q56 before, but it was a little too aggressive. Q=32 is the defau Hi, ${speakers-short}! Just a quick note to let you know that I’ve downloaded your -submission for “${title}”. We’ve added your submission to the backstage +submission for &lsquo ;${title}’. We’ve added your submission to the backstage area at ${backstage} (username: ${backstage-user}, password: ${backstage-password}), and we’ll post the files on your talk page when the talk is public. A quick check shows that your video is about @@ -2017,9 +2210,9 @@ Exception: [CHECK is unavailable](#check-gone) with instructions, so please check there. Let me know if you don’t get them. - Private messages: - - (erc-message “PRIVMSG” (format “%s You can use this BBB room: %s . I’ll join you there shortly to set up the room and do the last-minute tech check.” nick room-url)) - - (erc-message “PRIVMSG” (format “%s The collaborative pad we’ll be using for questions is at %s . We’ll collect questions from #emacsconf and put them there. If you’d like to jump to your part of the document, you might be able to keep an eye on questions. Alternatively, we can read questions to you.” nick conf-collaborative-pad)) - - (erc-message “PRIVMSG” (format “%s Leo Vivier will join when it’s time, and he will give you the go-ahead when it’s time to present. See you in the BBB room!” nick)) + - (erc-message &lsquo ;PRIVMSG’ (format &lsquo ;%s You can use this BBB room: %s . I’ll join you there shortly to set up the room and do the last-minute tech check.’ nick room-url)) + - (erc-message &lsquo ;PRIVMSG’ (format &lsquo ;%s The collaborative pad we’ll be using for questions is at %s . We’ll collect questions from #emacsconf and put them there. If you’d like to jump to your part of the document, you might be able to keep an eye on questions. Alternatively, we can read questions to you.’ nick conf-collaborative-pad)) + - (erc-message &lsquo ;PRIVMSG’ (format &lsquo ;%s Leo Vivier will join when it’s time, and he will give you the go-ahead when it’s time to present. See you in the BBB room!’ nick)) - CHECK directs speaker to available room with `/checkin <room> <nick>` - Speaker joins talk room - CHECK makes speaker presenter and moderator, does last-minute tech check @@ -2333,7 +2526,7 @@ Probably focus on grabbing the audio first and seeing what’s worth keeping Make a table of the form -<table id="org10608d5" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides"> +<table id="org5c473fc" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides"> <colgroup> @@ -2600,7 +2793,7 @@ mpv –vf=negate $url Extra stuff to consider adding: - DONE Suggestions for themes (especially wrt colourscape) -- DONE “Please don’t squeeze your talk by fast-forwarding your speech. Trimming silences and filler words can help, though. Sometimes it’s easier to write, record, and edit your voiceover, and then record the video to go along with it.“ +- DONE “Please don’t squeeze your talk by fast-forwarding your speech. Trimming silences and filler words can help, though. Sometimes it’s easier to write, record, and edit your voiceover, and then record the video to go along with it.&lsquo ; - DONE Allowing speakers to plant questions, esp. to cover tangential stuff that couldn’t fit in the prerec @@ -2624,7 +2817,7 @@ See the bottom of conf.org for some automated validation Extra stuff to consider adding: - DONE Suggestions for themes (especially wrt colourscape) -- DONE “Please don’t squeeze your talk by fast-forwarding your speech. Trimming silences and filler words can help, though. Sometimes it’s easier to write, record, and edit your voiceover, and then record the video to go along with it.“ +- DONE “Please don’t squeeze your talk by fast-forwarding your speech. Trimming silences and filler words can help, though. Sometimes it’s easier to write, record, and edit your voiceover, and then record the video to go along with it.&lsquo ; - DONE Allowing speakers to plant questions, esp. to cover tangential stuff that couldn’t fit in the prerec @@ -2732,7 +2925,7 @@ Slightly annoying to do with JS/CSS because I want it to fall back to an interle ## DONE Process confirmations as we receive them :sachac:zaeph: -- Reply to the speaker and Cc -submit to confirm the confirmation. Something like “Confirming your confirmation, no reply needed to this one. Thank you!” +- Reply to the speaker and Cc -submit to confirm the confirmation. Something like &lsquo ;Confirming your confirmation, no reply needed to this one. Thank you!’ - Update talk to WAITING\_FOR\_PREREC in conf.org - Add a note in the logbook (C-c C-z - org-add-note) for the talk entry - Add :PUBLIC\_EMAIL: t if given permission to use the e-mail on the @@ -3100,15 +3293,15 @@ It would be nice to have track-specific icals as well. zaeph: I can work on it, but I’m not experienced in this domain, so I’d prefer to be a back-up. task is currently with bandali -Lesson learned from last year: “Since people kept running into ftp +Lesson learned from last year: &lsquo ;Since people kept running into ftp problems, we might want to set up a web-frontend next year to minimise -problems.” +problems.’ Maybe we could ask some of the volunteers who wanted to help us with the infra? It shouldn’t be complicated to deploy a ready-made solution. -“file drop” is a common keyword for looking for information. +&lsquo ;file drop’ is a common keyword for looking for information. Considerations: - Probably run it on media.emacsconf.org @@ -3267,7 +3460,7 @@ Where: Nice if there’s an Ansible playbook sachac’s notes: - <file:///home/sacha/code/docker/emacsconf-publish/> + <file:///home/zaeph/code/docker/emacsconf-publish/> - probably good to set it up on front It’s now on front. @@ -3413,8 +3606,8 @@ generated captions that we can use as a starting point. We’ve set up ${backstage} as the backstage area where you can view the videos and resources uploaded so far. You can access it with the -username "${backstage-user}“ and the password ”${backstage-password}“. -Please keep the backstage password and other speakers’ talk resources +username "${backstage-user}&lsquo ; and the password ’${backstage-password}&lsquo ;. +Please keep the backstage password and other speakers” talk resources secret. ${backstage-use}${wrap} Thank you! @@ -3681,9 +3874,33 @@ Sacha Chua ## Future +### Captions for approval + + (defun emacsconf-mail-captions-for-approval (talk) + (interactive (list (emacsconf-complete-talk-info))) + (let ((captions (expand-file-name (concat (plist-get talk :video-slug) "--main.vtt") emacsconf-cache-dir)) + (captioner-info + (with-current-buffer (find-file-noselect emacsconf-org-file) + (org-entry-properties (org-find-property "CUSTOM_ID" (plist-get talk :captioner)))))) + (emacsconf-mail-prepare + (emacsconf-mail-merge-get-template "captions") + (plist-get talk :email) + (list + :speakers-short (plist-get talk :speakers-short) + :year emacsconf-year + :email (plist-get talk :email) + :title (plist-get talk :title) + :captioner (assoc-default "NAME_SHORT" captioner-info) + :url (format "https://media.emacsconf.org/%s/backstage" emacsconf-year) + :password emacsconf-backstage-password + :captioner-email (assoc-default "EMAIL" captioner-info) + :captions (with-temp-buffer (insert-file-contents captions) (buffer-string)))) + (mml-attach-file captions "text/vtt" "Subtitles" "attachment"))) + + <a id="approve-captions"></a> -### Captions for approval +#### Template Hi ${speakers-short}! @@ -3693,17 +3910,23 @@ volunteered to caption your talk video, and here it is. I’ve attached the caption text file in case you want to review it, suggest any corrections, or use the text in a blog post or elsewhere. You can look at the attached file or watch your video with closed captions at -${url} . Thanks again for your contribution! +${url} (username: emacsconf, password: ${password}). I've also +included the captions at the end of this e-mail for your convenience. +Thanks again for your contribution!${wrap} + +${captioner}: Thanks! Sacha +${captions} + ### Speakers we haven’t confirmed e-mail communications with Hi, ${name}! I think we haven’t heard from you since we accepted your EmacsConf -${year} proposal for "${title}“. EmacsConf coming up soon, so I wanted +${year} proposal for "${title}&lsquo ;. EmacsConf coming up soon, so I wanted to check in with you to see how you’re doing. Could you please e-mail us to let us know if you’re still working on @@ -4144,47 +4367,139 @@ chatter in the other lists Cc’d on this message. Thank you. set -eu - sleep_duration=1800 + sleep_duration=600 - data=prerec-wait-new.data - touch "$data" + data="$(basename "$0" ".sh").data" - plural() { + plural () { if [ "$2" = 1 ] || [ "$2" = -1 ]; then echo "${1}" else case $1 in "is" ) - "are" + echo "are" ;; * ) - "${1}s" + echo "${1}s" esac fi } + current_time() { + date +"[%T]" + } + + log() { + printf "%s $1\n" "$(current_time)" + } + + color_green="\e[32m" + color_white="\e[0m" + + log2() { + log "${color_green}$1${color_white}" + } + notify() { - echo "$1" - notify-send -t 0 "EmacsConf" "$1" + log2 "$1" + notify-send -t 0 "EmacsConf" "$(log "$1")" } - echo "Currently waiting for prerec" + _sleep() { + log "Checking again in ${sleep_duration}s" + sleep "$sleep_duration" + } - prerecs_number_past=$(cat "$data") - diff=0 + fetch() { + TERM=xterm ssh orga@media.emacsconf.org -- ls -1 /srv/upload | wc -l + } - while true; do - prerecs_number_current="$(TERM=xterm ssh orga@media.emacsconf.org -- ls -1 /srv/upload | wc -l)" - diff=$((prerecs_number_current - prerecs_number_past)) - if [ $diff -gt 0 ]; then - break + clean() { + rm -f "$data" + log "Data file has been cleaned" + } + + if [ "$#" -gt 0 ]; then + command="$1" + if [ "$command" = "clean" ]; then + clean + exit 0 fi - sleep "$sleep_duration" + + if [ "$command" = "clean-start" ]; then + clean + fi + fi + + touch "$data" + + log2 "Currently waiting for prerec" + + while true; do + prerecs_number_past=$(cat "$data") + diff=0 + + while true; do + log "Checking..." + prerecs_number_current="$(fetch)" + diff=$((prerecs_number_current - prerecs_number_past)) + if [ $diff -gt 0 ]; then + break + fi + log "No new prerec" + _sleep + done + + notify "$diff new $(plural "prerec" $diff) $(plural "is" $diff) available!" + + echo "$prerecs_number_current" > "$data" + + _sleep done + + +### Fetch upload data + + #!/usr/bin/env sh + + set -eu + + current_time() { + date +"[%T]" + } + + log() { + printf "%s $1\n" "$(current_time)" + } + + if [ -t 0 ]; then + log "Fetching data..." + fi + + TERM=xterm ssh orga@media.emacsconf.org -- cat /srv/upload/*/*json + + +### Fetch new talk data formatted + + #!/usr/bin/env sh + + set -eu + + echo "Fetching data..." + data_raw="$(./fetch-upload-data.sh)" - notify "Yay~, $diff new $(plural "prerec" $diff) $(plural "is" $diff) available!" + echo - echo "$prerecs_number_current" > "$data" + echo "$data_raw" | awk "$(cat << EOF + func dewrap(a) {gsub(/^ *".*": "|",$/, "", a); return a}; + BEGIN {i=0}; + /^ "sid"/ { a[0] = dewrap(\$0); }; + /^ "name"/ { a[2] = sprintf("%s\t", dewrap(\$0)) }; + /^ "comment"/ { a[3] = sprintf("%s\t", dewrap(\$0)); if (a[3] == "\t"){a[3] = "No comment"} }; + /^ "key"/ { a[1] = dewrap(\$0); }; + /^\}/ { printf "[%d]\t%s\t%s\t%s\n\t%s\n\n", i, a[0], a[1], a[2], a[3]; i=i+1}; + EOF + )" ## Publish this page @@ -4320,10 +4635,10 @@ see emacsconf-import-comments-from-etherpad-text format makes it harder not only for the organizers, but also for the speakers themselves (since they will have to rethink their presentation). Maybe we can have an e-mail template for a quick - reply that says something like “Just in case we need to squeeze + reply that says something like &lsquo ;Just in case we need to squeeze talks into shorter times, could you please also propose an outline for a possible 10-minute talk that could get people interested in - your topic and point them to where they can find out more?” + your topic and point them to where they can find out more?’ - Two people is the sweet number of reviewers to have for the proposals before sending the notifications, and there’d be diminishing returns with more. Two is enough to release the pressure diff --git a/2022/organizers-notebook/index.org b/2022/organizers-notebook/index.org index 30b7c273..da7172b1 100644 --- a/2022/organizers-notebook/index.org +++ b/2022/organizers-notebook/index.org @@ -1303,6 +1303,29 @@ but git log does not have the new stuff Where is the new stuff? ... hah, maybe I forgot to push +** Backstage +:PROPERTIES: +:CUSTOM_ID: infra-backstage +:END: +*** File suffixes, and what they correspond to +Per categories, earlier suffixes come earlier in the process. +|----------------------------+------------------------------------------------------------------------| +| Suffix | Description | +|----------------------------+------------------------------------------------------------------------| +| VIDEO | | +| --original.EXT | File as submitted by speaker | +| --reencoded.webm | Reencode via ffmpeg incantation | +| --final.webm | Broadcast-ready reencode with normalized audio | +|----------------------------+------------------------------------------------------------------------| +| AUDIO | | +| --original.EXT | Extracted audio track from speaker upload; used for speech-recognition | +| --normalized.opus | Normalized audio track | +|----------------------------+------------------------------------------------------------------------| +| SUBTITLES | | +| --?(incomplete-)transcript | Transcript provided by speaker | +| --main.EXT | Broadcast-ready reencode; different formats for different uses | +|----------------------------+------------------------------------------------------------------------| + * Other tasks and processes :PROPERTIES: :CUSTOM_ID: other |