diff options
Diffstat (limited to 'captioning.md')
| -rw-r--r-- | captioning.md | 215 |
1 files changed, 142 insertions, 73 deletions
diff --git a/captioning.md b/captioning.md index ed9e6890..e0078652 100644 --- a/captioning.md +++ b/captioning.md @@ -7,77 +7,119 @@ easier to understand and search. If you see a talk that you'd like to caption, feel free to download it and start working on it with your favourite subtitle editor. Let me know what you pick by e-mailing me at <sacha@sachachua.com> so that I -can update the index and try to avoid duplication of work. [Find talks that need captions here](https://emacsconf.org/help_with_main_captions). +can update the backstage index and try to avoid duplication of work. [Find talks that need captions here](https://emacsconf.org/help_with_main_captions). You can also help by [adding chapter markers to Q&A sessions](https://emacsconf.org/help_with_chapter_markers). -We've been using <https://github.com/sachac/subed> to caption things -as VTT or SRT in Emacs, often starting with autogenerated captions -from YouTube (the .vtt or .srt file), but you're welcome to make -captions using your favourite tool. +You're welcome to work with captions using your favourite tool. We've +been using <https://github.com/sachac/subed> to caption things as VTT +or SRT in Emacs, often starting with autogenerated captions from +OpenAI Whisper or WhisperX (the .vtt file backstage). -We'll be posting VTT files so that they can be included by the HTML5 video -player (demo: <https://emacsconf.org/2021/talks/news/>), so if you use a -different tool that produces another format, any format that can be -converted into that one (like SRT or ASS) is fine. You can e-mail me the -subtitles when you're done, and then I can merge it into the video. +We'll be posting VTT files so that they can be included by the HTML5 +video player (demo: <https://emacsconf.org/2021/talks/news/>), so if +you use a different tool that produces another format, any format that +can be converted into that one (like SRT or ASS) is fine. `subed` has +a `subed-convert` command that might be useful for turning WebVTT +files into tab-separated values (TSV) and back again, if you prefer a +more concise format. -# Formatting tips +You can e-mail me the subtitles when you're done, and then I can merge +it into the video. You might find it easier to start with the autogenerated captions -and then refer to any resources provided by the speaker in order to +and then refer to the video or any resources provided by the speaker in order to figure out spelling. Sometimes speakers provide pretty complete scripts, which is great, but they also tend to add extra words. -Emacs being Emacs, you can use some code ( -[example subed configuration](https://sachachua.com/dotemacs/#subed), see -`my-subed-fix-common-error` and `my-subed-common-edits`) to help with -capitalization and commonly misrecognized words. - -Please keep captions to one line each so that they can be displayed -without wrapping, as we plan to broadcast by resizing the video and -displaying open captions below. Maybe 50 characters max? Since the -captions are also displayed as text on the talk pages, you can omit -filler words. Split the captions at natural pausing points (ex: -phrases) so that they're displayed nicely. - -For example, instead of: - -- so i'm going to talk today about a -- fun rewrite i did of uh of the bindat -- package - -you can edit it to be more like: - -- So I'm going to talk today -- about a fun rewrite I did -- of the bindat package. - -If you don't understand a word or phrase, add two question marks (??) -and move on. We'll ask the speakers to review the subtitles and can -sort that out then. - -If there are multiple speakers, indicate switches between speakers -with a `[speaker-name]:` tag. - -During questions and answers, please introduce the question with a -`[question]:` tag. When the speaker answers, use a `[speaker-name]:` -tag to make clear who is talking. - -# Editing autogenerated captions - -If you want to take advantage of the autogenerated captions and the -word-level timing data from YouTube, you can start with the VTT file -for the video you want, then use `my-caption-load-word-data` from -<https://sachachua.com/dotemacs/#word-level> to load the srv2 file -(also attached), and then use `my-caption-split` to split using the -word timing data if possible. You can bind this to a keystroke with -something like `M-x local-set-key M-' my-caption-split`. +# Edit the VTT to fix misrecognized words + +The first step is to edit misrecognized words. VTT files are plain text, so +you can edit them with regular `text-mode` if you want to. If you're +editing subtitles within Emacs, +[subed](https://github.com/sachac/subed) can conveniently synchronize +video playback with subtitle editing, which makes it easier to figure +out technical words. subed tries to load the video based on the +filename, but if it can't find it, you can use `C-c C-v` +(`subed-mpv-find-media`) to play a file or `C-c C-u` to play a URL. + +Look for misrecognized words and edit them. We also like to change +things to follow Emacs keybinding conventions (C-c instead of Control C). We sometimes spell out +acronyms on first use or add extra information in brackets. The +captions will be used in a transcript as well, so you can add +punctuation, remove filler words, and try to make it read better. + +Sometimes you may want to tweak how the captions are split. You can +use `M-j` (`subed-jump-to-current-subtitle`) to jump to the caption if +I'm not already on it, listen for the right spot, and maybe use +`M-SPC` to toggle playback. Use `M-.` (`subed-split-subtitle`) to +split a caption at the current MPV playing position and `M-m` +(`subed-merge-with-next`) to merge a subtitle with the next one. + +If you don't understand a word or phrase, add two +question marks (`[??]`) and move on. We'll ask the +speakers to review the subtitles and can sort that +out then. + +If there are multiple speakers, you can indicate switches between speakers +with a `[speaker-name]:` tag, or just leave it plain. + + +<video src="https://media.emacsconf.org/editing.webm" controls=""></video> + +Once you've gotten the hang of things, it might take between 1x to 4x +the video time to edit captions. + +# Subtitle timing + +Times don't need to be very precise. If you notice +that the times are way out of whack and it's +getting in the way of your subtitling, we can +adjust the times using the [aeneas forced +alignment tool](https://www.readbeyond.it/aeneas/) +and `subed-align`. + +## Splitting and merging subtitles + +If you want to split and merge subtitles, you can +use `M-.` (`subed-split-subtitle`) and `M-m` +(`subed-merge-dwim`). If the playback position is +in the current subtitle, splitting will use the +playback position. If it isn't, it will guess an +appropriate time based on characters per second +for the current subtitle. + +## Splitting with word-level timing data + +If there is a `.json` or `.srv2` file with +word-level timing data, you can load it with +`subed-word-data-load-from-file` from +`subed-word-data.el` in the subed package. You can +then split with the usual `M-.` +(`subed-split-subtitle`), and it should use +word-level timestamps when available. + +# Playing your subtitles together with the video + +MPV should automatically load subtitle files if +they're in the same directory as the video. To +load a specific subtitle file in MPV, you can use +the `--sub-file=` or `--sub-files=` command-line +argument. + +If you're using subed, the video should autoplay if it's named the +same as your subtitle file. If not, you can use `C-c C-v` +(`subed-mpv-play-from-file`) to load the video file. You can toggle +looping over the current subtitle with `C-c C-l` +(`subed-toggle-loop-over-current-subtitle`), synchronizing player to +point with `C-c ,` (`subed-toggle-sync-player-to-point`), and +synchronizing point to player with `C-c .` +(`subed-toggle-sync-point-to-player`). # Starting from a script -Some talks don't have autogenerated captions because YouTube didn't -produce any. Whenever the speaker has provided a script, you can use -that as a starting point. One way is to start by making a VTT file with -one subtitle spanning the whole video, like this: +Some talks don't have autogenerated captions, or you may prefer to +start from scratch. Whenever the speaker has provided a script, you +can use that as a starting point. One way is to start by making a VTT +file with one subtitle spanning the whole video, like this: ```text WEBVTT @@ -87,32 +129,59 @@ If the speaker provided a script, I usually put the script under this heading. ``` If you're using subed, you can move to the point to a good stopping -point for a phrase, toggle playing with `M-SPC`, and then `M-.` +point for a phrase, use `M-SPC` to toggle pausing `M-.` (`subed-split-subtitle`) when the player reaches that point. If it's too fast, use `M-j` to repeat the current subtitle. # Starting from scratch -Sometimes there are no autogenerated captions and there's no script, -so we have to start from scratch. - -You can send us a text file with just the text transcript in it and -not worry about the timestamps. We can figure out the timing using +One option is to send us a text file with just the text transcript in it +and not worry about the timestamps. We can figure out the timing using [aeneas for forced alignment](https://www.readbeyond.it/aeneas/). -If you want to try timing as you go, you might find it easier to start -by making a VTT file with one subtitle spanning the whole video, like -this: +If you want to try timing as you go, you might +find it easier to start by making a VTT file with +one subtitle spanning the whole video (either +using the video duration or a very large +duration), like this: ```text WEBVTT -00:00:00.000 -> 00:39:07.000 +00:00:00.000 -> 24:00:00.000 +``` + +Use `C-c C-p` (`subed-toggle-pause-while-typing`) +to automatically pause when typing. Then start +playback with `M-SPC` and type, using `M-.` +(`subed-split-subtitle`) to split after a +reasonable length for a subtitle. If it's too +fast, use `M-j` to repeat the current subtitle or +adjust `subed-mpv-plackback-speed`. + +# Chapter markers + +In addition to the captions, you may also want to add chapter markers. +An easy way to do that is to add a =NOTE Chapter heading= before the +subtitle that starts the chapter. For example: + +```text +... +00:05:13.880 --> 00:05:20.119 +So yeah, like that's currently the problem. + +NOTE Embeddings + +00:05:20.120 --> 00:05:23.399 +So I want to talk about embeddings. +... ``` -Then start playback and type, using `M-.` (`subed-split-subtitle`) to -split after a reasonable length for a subtitle. If it's too fast, use -`M-j` to repeat the current subtitle. +We can then extract those with +`emacsconf-subed-make-chapter-file-based-on-comments`. + +For an example of how chapter markers allow people to quickly navigate +videos, see <https://emacsconf.org/2021/talks/bindat/> . Please let us know if you need any help! |