update task status

2 files changed, 225 insertions, 26 deletions

diff --git a/2022/organizers-notebook.md b/2022/organizers-notebook.md
index 5923c1a5..54ea1f4e 100644
--- a/2022/organizers-notebook.md
+++ b/2022/organizers-notebook.md
@@ -298,7 +298,9 @@ Previous priorities;
 
 # Next comms update
 
--   next step is to review all the playbacks and publish them
+-   next step is to make chapter markers
+-   <span class="timestamp-wrapper"><span class="timestamp">[2022-12-15 Thu] </span></span> all Q&A videos posted, large model VTTs available
+-   <span class="timestamp-wrapper"><span class="timestamp">[2022-12-12 Mon] </span></span> Updated grail, updated backstage view
 -   <span class="timestamp-wrapper"><span class="timestamp">[2022-12-11 Sun] </span></span> Thank-you notes sent to all the speakers with Toobnix and YouTube URLs, BBB playback info
 -   <span class="timestamp-wrapper"><span class="timestamp">[2022-12-10 Sat] </span></span> small-model VTTs uploaded for easier review and chapter markers, still working on large-model transcripts
 
@@ -340,18 +342,24 @@ Previous priorities;
 ### Publish more Q&A recordings
 
 
-#### DONE Write code to make it easy to do so
-
-emacsconf-extract-publish-qa
+#### TODO Add audio-only options to the publishing process
 
 
-#### SOMEDAY Handwritten needs to be trimmed
+#### DONE Check status
 
+    (emacsconf-collect-prop :slug
+     (seq-remove
+    	(lambda (o)
+    		(or (null (string-match "live" (or (plist-get o :q-and-a) "")))
+    				(null (emacsconf-talk-file o "--bbb-webcams.webm"))
+    				(and (emacsconf-talk-file o "--answers.webm")
+    						 (plist-get o :qa-public))))
+    	(emacsconf-prepare-for-display (emacsconf-get-talk-info))))
 
-#### SOMEDAY Sqlite needs trimming
 
+#### DONE Write code to make it easy to do so
 
-#### TODO Add audio-only options to the publishing process
+emacsconf-extract-publish-qa
 
 
 #### DONE Fetch the reencoded maint answers and post it to backstage
@@ -436,6 +444,9 @@ To avoid duplicating work, maybe we can use this Etherpad to call dibs on talks
 If you&rsquo;re already set up to edit the wiki or you can figure out the slightly complicated setup for editing, feel free to edit the page directly. If not, you can email a Markdown or Org Mode snippet to emacsconf-submit@gnu.org and let me know what page to put it on.
 
 
+### TODO Split up relevant sections from IRC logs so that volunteers can go through them
+
+
 ### DONE Check duration of Q&A BBB recordings
 
 About 15 hours total
@@ -533,13 +544,10 @@ in bbb-whisper
 See links at the top of backstage
 
 
-### TODO Split up relevant sections from IRC logs so that volunteers can go through them
-
-
 ### SOMEDAY Copy logs for analysis
 
 
-### SOMEDAY Transcribe recordings
+### DONE Transcribe recordings
 
 
 ### DONE Collect recordings from bbb
@@ -715,6 +723,36 @@ Is it DeskShareStartRTMP ?
 ### SOMEDAY Copy irc, haven&rsquo;t actually done that yet
 
 
+### SOMEDAY Create harvesting etherpad if needed to track the status?
+
+
+### DONE Update task status now that I have TO\_REVIEW\_QA, TO\_INDEX\_QA, TO\_CAPTION\_QA
+
+    (with-current-buffer (find-file-noselect emacsconf-org-file)
+    	(org-map-entries
+    	 (lambda ()
+    		 (if (file-exists-p (expand-file-name (concat (org-entry-get (point) "VIDEO_SLUG") "--answers.webm")
+    																						emacsconf-cache-dir))
+    				 (org-todo "TO_INDEX_QA")
+    			 (org-todo "TO_REVIEW_QA"))
+    		 (when (string-match "large" (or (org-entry-get (point) "QA_NOTE") ""))
+    			 (org-entry-put (point) "QA_MODEL" "large")))
+    	 "TODO=\"TO_ARCHIVE\"+SLUG={.}")
+    	)
+    
+    (with-current-buffer (find-file-noselect emacsconf-org-file)
+    	(org-map-entries
+    	 (lambda ()
+    		 (unless (file-exists-p (expand-file-name (concat (org-entry-get (point) "VIDEO_SLUG") "--bbb-webcams.webm")
+    																							emacsconf-cache-dir))
+    			 (org-todo "TO_ARCHIVE")))
+    	 "TODO=\"TO_REVIEW_QA\"+SLUG={.}")
+    	)
+
+
+### TODO consider number-anonymizing the IRC chat
+
+
 <a id="thanks"></a>
 
 ## TODO Send thanks
@@ -1412,7 +1450,7 @@ tags: emacs,emacsconf
     							 (plist-get talk :speakers-with-pronouns)))
     		(shell-command "xdotool sleep 1 mousemove 331 440 click :1 key Ctrl+a Delete sleep 1 key Ctrl+Shift+v sleep 2")
     		
-    		(kill-new (emacsconf-publish-video-description talk t t))
+    		(kill-new (emacsconf-publish-video-description talk t))
     		(shell-command "xdotool mousemove 474 632 click 1 sleep 1 key Ctrl+a sleep 1 key Delete sleep 1 key Ctrl+Shift+v"))
     	(read-string "Press a key once you've pasted in the description")
     
@@ -2017,6 +2055,9 @@ in prod or docker container
 ### DONE Set up Etherpad with MySQL
 
 
+### TODO Make the upload ansible configuration controllable via a variable
+
+
 <a id="breakouts"></a>
 
 ## SOMEDAY Consider breakout rooms for lunch break
@@ -2210,6 +2251,15 @@ if we can get the anchor plugin
 <https://emacs.ch/@sachac/109486006078029919>
 
 
+### SOMEDAY <https://graz.social/@publicvoit/109496340869869181>
+
+publicvoit@graz.social - Organizers of online #video #conferences: take a look at the amazing work the organizers of #EmacsConf are doing with the pre- and post-processing of a 9 minute demo #video: <https://emacsconf.org/2022/talks/orgsuperlinks/>
+
+Subtitles, transcript, (Q&A from the live discussion are following), chapter marks, discussion thread from the #Etherpad used, &#x2026;
+
+I only wrote the &rsquo;Description&rsquo; section and contributed the raw video file. It&rsquo;s really impressive what @sachac and the other organizers + volunteers are doing here.  
+
+
 <a id="maybe-projects"></a>
 
 # Ideas for next year     :nextyear:
@@ -2465,9 +2515,17 @@ or a VM
 ### SOMEDAY Investigate bacalhau compute over data
 
 
+### SOMEDAY Using video transcoding on Amazon ECS - Amazon Elastic Container Service
+
+<https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-vt1.html>
+
+
 ## SOMEDAY Add timer for 5 minute and 2 minute warnings to go to emacsconf-org
 
 
+## TODO make the Org agenda versions more visible
+
+
 <a id="decisions"></a>
 
 # Things to figure out / decisions to make
@@ -2942,7 +3000,7 @@ AM: 9-12 PM EST, PM: 1-5 PM EST (plus a little extra for setup/transition)
 
 Saturday Dec 3
 
-<table id="org191b5f0" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
+<table id="org29a30f1" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
 
 
 <colgroup>
@@ -3022,7 +3080,7 @@ publicvoit - pad until 4pm on Sat, until 2pm on Sun
 
 Sunday Dec 4
 
-<table id="org1aee553" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
+<table id="org9ecf02a" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
 
 
 <colgroup>
@@ -4133,7 +4191,7 @@ Probably focus on grabbing the audio first and seeing what&rsquo;s worth keeping
 
 Make a table of the form
 
-<table id="org8d5c592" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
+<table id="org91cdd59" border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
 
 
 <colgroup>
@@ -4231,7 +4289,7 @@ See <https://pad.emacsconf.org/premortem> for more.
 
 -   [Common broadcast (gen and dev)](#common):stream:
 -   [Stream OBS needs to be restarted](#obs-restart):stream:
--   [Stuck wiki](#ikiwiki-stuck):publish:
+-   [Wiki](#ikiwiki-stuck):publish:
 -   [Cancelled talk](#cancelled):schedule:
 -   [Last-minute prerecording submission](#last-minute-prerec):upload:
 -   [Technical issues during a live presentation](#live-issues):stream:
@@ -4250,10 +4308,10 @@ See <https://pad.emacsconf.org/premortem> for more.
 -   [In case we need to do things manually because the task status hooks don&rsquo;t work](#manual)
 -   [Alternate stream volunteer wants to stream](#alternate)
 -   [Video playing error, need to play with mpv manually](#play-mpv-manually):stream:
--   [Shift changes](#org873d736)
+-   [Shift changes](#org83fcc24)
 -   [Need to restart 480p stream](#restart-480p):stream:
 -   [Renamed talk](#renamed):sched:
--   [Need to restart the Toobnix streams so we don&rsquo;t exceed 5 hours](#orgc2028ca)
+-   [Need to restart the Toobnix streams so we don&rsquo;t exceed 5 hours](#orgee62746)
 
 
 <a id="common"></a>
@@ -4290,10 +4348,18 @@ emacsconf-stream-rebroadcast - specify the source track
 
 <a id="ikiwiki-stuck"></a>
 
-## Stuck wiki     :publish:
+## Wiki     :publish:
+
+Stuck:
 
 sudo -iu ikiwiki &#x2013; ikiwiki &#x2013;setup ~ikiwiki/emacsconf.setup
 
+emacsconf.setup changed:
+
+ssh orga@front0.emacsconf.org
+sudo su - ikiwiki
+ikiwiki &#x2013;setup emacsconf.setup &#x2013;rebuild &#x2013;wrappers
+
 
 <a id="cancelled"></a>
 
@@ -4475,12 +4541,12 @@ mpv &#x2013;vf=negate $url
 
 export SLUG=the ID of the talk
 
--   [TO\_STREAM -> PLAYING](#org84bcf82):stream:
+-   [TO\_STREAM -> PLAYING](#org63f4951):stream:
 -   [CLOSED\_Q -> OPEN\_Q](#open)
 -   [OPEN\_Q -> TO\_ARCHIVE](#archive)
 
 
-<a id="org84bcf82"></a>
+<a id="org63f4951"></a>
 
 ### TO\_STREAM -> PLAYING     :stream:
 
@@ -4549,7 +4615,7 @@ You can skip the intro and play a video by specifying the filename, like this:
 play ~/stream/emacsconf-2022-journalism-\*
 
 
-<a id="org873d736"></a>
+<a id="org83fcc24"></a>
 
 ## Shift changes
 
@@ -4600,7 +4666,7 @@ lspbridge science
 health eev
 
 
-<a id="orgc2028ca"></a>
+<a id="orgee62746"></a>
 
 ## TODO Need to restart the Toobnix streams so we don&rsquo;t exceed 5 hours
 
@@ -6889,6 +6955,16 @@ Also look into finetuneas
 <https://github.com/openai/whisper/discussions/395>
 
 
+### SOMEDAY Pointers for running this on a GPU via a cloud service? · Discussion #398 · openai/whisper
+
+<https://github.com/openai/whisper/discussions/398>
+
+
+### SOMEDAY How to chunk text into paragraphs using python | by N Polovinkin | Medium
+
+<https://medium.com/@npolovinkin/how-to-chunk-text-into-paragraphs-using-python-8ae66be38ea6>
+
+
 ## DONE Find a way to accommodate a specific return-speaker
 
 We’re not sure if we’re going to get a presentation or a prerec for them
diff --git a/2022/organizers-notebook/index.org b/2022/organizers-notebook/index.org
index 539f0647..b73e8d43 100644
--- a/2022/organizers-notebook/index.org
+++ b/2022/organizers-notebook/index.org
@@ -99,7 +99,8 @@ Previous priorities;
 :CREATED:  [022-12-11 Sun 10:0]
 :END:
 
-- next step is to review all the playbacks and publish them
+- next step is to make chapter markers
+- [2022-12-15 Thu] all Q&A videos posted, large model VTTs available
 - [2022-12-12 Mon] Updated grail, updated backstage view
 - [2022-12-11 Sun] Thank-you notes sent to all the speakers with Toobnix and YouTube URLs, BBB playback info
 - [2022-12-10 Sat] small-model VTTs uploaded for easier review and chapter markers, still working on large-model transcripts
@@ -113,8 +114,26 @@ Previous priorities;
 :PROPERTIES:
 :CUSTOM_ID: harvest
 :END:
-*** Publish more Q&A recordings
-**** TODO Add audio-only options to the publishing process
+*** STARTED Add audio-only options to the publishing process
+:PROPERTIES:
+:Effort:   0:30
+:END:
+:LOGBOOK:
+CLOCK: [2022-12-15 Thu 10:29]
+:END:
+*** Learn from how other conferences harvest their talks and Q&A
+- DebConf posts the talk video (prerecorded + in person Q&A), no transcripts or index
+  Ex: https://debconf22.debconf.org/talks/71-sequoia-pgp-v5-openpgp-authentication-and-debian/ . They link to the video and the Etherpad.
+- FOSSDEM has directory listings by track: https://video.fosdem.org/2022/D.conference/
+  also embedded video and links to slides, video recording, chat room (but not logs) https://archive.fosdem.org/2022/schedule/event/community_contributions/
+- LibrePlanet has thumbnails:
+  https://media.libreplanet.org/u/libreplanet/tag/libreplanet-2022-video/ . Talk page has video and links to slides, download.
+  https://media.libreplanet.org/u/libreplanet/m/taking-back-the-web-with-haketilo/
+- NeovimConf: YouTube playlist, lots of comments on YouTube, key moments
+- VS Code Day: link to YouTube video, edited CC captions in YouTube, links and a hashtag in the video description, key moments
+  
+*** DONE Publish more Q&A recordings
+CLOSED: [2022-12-15 Thu 10:27]
 **** DONE Check status
 CLOSED: [2022-12-15 Thu 00:39]
 #+begin_src emacs-lisp :results replace :eval no
@@ -207,6 +226,54 @@ Here's a good/better/best scale for stuff we can do during this phase:
 - Totally awesome:
   - Edited captions/transcripts for the Q&A
   - Answers copied into the Q&A section, possibly with linked timestamps
+**** Process for reviewing and trimming the videos
+
+Helper function:
+- emacsconf-extract-publish-qa (call with C-u in order to specify a time for truncating the video)
+
+
+Skim the transcript and the videos to see if anything needs to be removed, and which video to use
+
+BigBlueButton gives us the webcams and audio as one video
+(`--bbb-webcams.webm`) and the screenshare (if any) as another video
+(`--bbb-deskshare.webm`). If the speaker shared their screen, we can
+focus on that instead of their webcam. The following ffmpeg command
+combines the audio from the webcams (which has been previously
+extracted into a separate file, `--bbb-webcams.opus`) with the video
+from the screenshare.
+
+    ffmpeg -i example--bbb-webcams.opus -i example--bbb-deskshare.webm -c copy example--answers.webm
+
+We also want to check if people accidentally shared sensitive
+information on their screen, or if anyone said something that they
+might not have said if they remembered that thethe Q&A videos will be
+shared after the talk. Sometimes there's some time before we get
+around to closing the meeting at the end of the Q&A. Usually, a quick
+read of the transcript will show anything that needs to be trimmed.
+Here's how to stop the recording at a specified time:
+
+    ffmpeg -i input.webm -to hh:mm:ss -c copy output.webm
+
+Cutting out stuff from the middle of a recording is slightly more
+complicated. It might be easier to use a nonlinear video editor such
+as kdenlive to edit the video. If you want to use ffmpeg, using
+filters to select the frames and reencode the video will probably work
+out better than splitting the file into multiple parts and then
+concatenating them without reencoding, as the latter tends to need to
+be split on keyframes. Here's a sample command based on this
+[StackOverflow](https://stackoverflow.com/questions/64866231/remove-a-section-from-the-middle-of-a-video-without-concat) answer that removes the section between 15 seconds and
+45 seconds:
+
+    ffmpeg -i input.webm \
+      -vf  "select='not(between(t,15,45))',  setpts=N/FRAME_RATE/TB" \
+      -af "aselect='not(between(t,15,45))', asetpts=N/SR/TB" \
+      output.webm
+
+Alternatively, you can let us know what parts needs to be trimmed, and
+we can figure that part out.
+
+
+
 
 *** TODO Prepare groundwork for volunteers to help
 **** CANCELLED Document IRC/pad/BBB chat extraction tasks
@@ -610,6 +677,57 @@ CLOSED: [2022-12-13 Tue 09:55]
 :CREATED:  [2022-12-12 Mon 09:34]
 :END:
 
+*** TODO switch to resources view
+:PROPERTIES:
+:CREATED: [2022-12-15 Thu 01:16]
+:END:
+*** TODO Harvest the closing remarks
+:PROPERTIES:
+:CREATED: [2022-12-15 Thu 00:59]
+:END:
+*** TODO Check normalization of answers
+:PROPERTIES:
+:CREATED: [2022-12-15 Thu 00:11]
+:END:
+https://stackoverflow.com/questions/42492246/how-to-normalize-the-volume-of-an-audio-file-in-python
+**** SOMEDAY Scripting - Audacity Manual
+:PROPERTIES:
+:CREATED:  [2022-12-15 Thu 00:02]
+:END:
+
+https://manual.audacityteam.org/man/scripting.html
+
+*** SOMEDAY Post transcripts, start working on chapter workflow
+:PROPERTIES:
+:CREATED:  [2022-12-14 Wed 22:24]
+:END:
+
+*** SOMEDAY Add audio-only handler for media card
+:PROPERTIES:
+:CREATED:  [2022-12-15 Thu 01:18]
+:END:
+
+*** SOMEDAY How to chunk text into paragraphs using python | by N Polovinkin | Medium
+:PROPERTIES:
+:CREATED:  [2022-12-14 Wed 17:17]
+:END:
+
+https://medium.com/@npolovinkin/how-to-chunk-text-into-paragraphs-using-python-8ae66be38ea6
+
+*** SOMEDAY Figure out ways to make sense of IRC logs
+:PROPERTIES:
+:CREATED:  [2022-12-13 Tue 16:37]
+:END:
+
+The challenge: make it easier to make sense of an IRC log considering the speaker is responding to lots of people.
+Possible: thread speaker messages?
+Manual:
+- search backward to find the lines, put something under
+- avy to indent it under that line
+- use properties to suggest lines
+- copy everything, use Emacs News code to refile entries
+- anonymize after
+
 ** TODO Send thanks
 :PROPERTIES:
 :CUSTOM_ID: thanks
@@ -2622,6 +2740,11 @@ https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-vt1.html
 :PROPERTIES:
 :CREATED: [2022-12-13 Tue 10:54]
 :END:
+** SOMEDAY Explore idea of OBS virtual webcam showing the question vs screenshare
+:PROPERTIES:
+:CREATED:  [2022-12-13 Tue 16:38]
+:END:
+
 * Things to figure out / decisions to make
 :PROPERTIES:
 :CUSTOM_ID: decisions