path: root/organizers-notebook.md

<!-- organizers-notebook.md is exported from organizers-notebook/index.org, please modify that instead. -->
[[!sidebar content=""]]

This file is automatically exported from [/organizers-notebook/index.org](/organizers-notebook/index.org). You might prefer to navigate this as an Org file instead. To do so, [clone the wiki repository](https://emacsconf.org/edit/).

This document is the general organizers' notebook that can be used as a starting point each year. For specific years, see:

- [2024](/2024/organizers-notebook)
- [2023](/2023/organizers-notebook)
- [2022](/2022/organizers-notebook)
- [2021](/2021/organizers-notebook)
- [2020](/2020/organizers-notebook)
- [2019](/2019/organizers-notebook)


# Table of Contents

-   [Starting up the conference planning process](#starting-up-the-conference-planning-process)
    -   [Create the private repository](#starting-up-the-conference-planning-process-create-the-private-repository)
    -   [Create the public organizers&rsquo; notebook in the wiki](#starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki)
-   [Phases and lessons learned](#phases-and-lessons-learned)
    -   [Draft CFP](#phases-and-lessons-learned-draft-cfp)
    -   [Announce](#org0e21b8c)
    -   [Process proposals](#org0b948ff)
    -   [Accept proposals](#org709c243)
    -   [Draft schedule](#draft-schedule)
    -   [Set up backstage and upload](#orgdc12c78)
    -   [Post the schedule](#orgce4ff13)
    -   [Process uploaded files](#org6cdaa28)
    -   [Generate assets](#org3c7eeb9)
    -   [Prepare the infrastructure](#orgf502d28)
    -   [Getting ready for captions](#org00f1270)
-   [General infrastructure](#orga2f2794)
    -   [Icecast](#icecast)
    -   [Watching pages](#watch)
    -   [VNC](#vnc)
    -   [Streaming with OBS](#obs)
    -   [Shell scripts](#org93c38b4)
    -   [Backstage](#backstage)
    -   [IRC web client](#org75fb0fd)
    -   [Etherpad](#orgadd3307)
    -   [Publishing media to the server and to the wiki](#media)
    -   [Automated IRC announcements](#org0b45c02)
    -   [Manual IRC announcements](#orgf31a707)
    -   [Low-res stream](#org2b9c281)
    -   [Restream to YouTube](#org00999f1)
    -   [Restream to Toobnix](#org1d92e71)
-   [Other process notes](#other)
    -   [Adding another user to BBB](#bbb-user)
    -   [When a talk is cancelled](#cancel-talk)
    -   [When a talk Q&A changes method](#change-talk-qa)
    -   [Ansible](#orge2b5368)


<a id="starting-up-the-conference-planning-process"></a>

# Starting up the conference planning process

Check in with core organizers from the previous year to see who&rsquo;s in.


<a id="starting-up-the-conference-planning-process-create-the-private-repository"></a>

## Create the private repository

This repository will store submissions, contact information, etc.

1.  Create the repository on `front0`. Ex:
    
        sudo su - git
        cd repositories
        mkdir emacsconf-2024-private.git
        cd emacsconf-2024-private.git
        git init --bare
2.  Edit `gitolite-admin/config/gitolite-conf`. Add lines for the group and the repo. Commit and push. Ex:
    
        @emacsconf-2024-org = bandali sachac ...
        repo emacsconf-2024-private
             RW+ = @admins @emacsconf-2024-org orga
3.  Clone the private repository into the appropriate location. `git clone git@git.emacsconf.org:emacsconf-2024-private private`


<a id="starting-up-the-conference-planning-process-create-the-public-organizers-notebook-in-the-wiki"></a>

## Create the public organizers&rsquo; notebook in the wiki

Ex: 2024/organizers-notebook/index.org


<a id="phases-and-lessons-learned"></a>

# Phases and lessons learned


<a id="phases-and-lessons-learned-draft-cfp"></a>

## Draft CFP

Create linked pages

-   [ ] year index
-   [ ] volunteer page
-   [ ] submit page


### How to mark pages as drafts

Put inside double square brackets: `!template id=pagedraft`


### Previous years

[2023](file:///home/sacha/proj/emacsconf/wiki/2023/cfp.md)


### Lessons learned

-   Maybe incentivize proper timezone specification by saying we can translate times to their local time?
-   Make sure to include cfp.org as an attachment instead of inline
-   bandali likes having the commitment to freedom section in the CFP as a form of activism.
-   Ask for public e-mail or contact information, IRC handle in CFP.
-   Nudge everyone towards 5/10 or 20 minutes, and consider < or > on a case-by-case basis.
    -   In previous years, we offered 40 minutes as a
        regular choice, but that was difficult because
        we had to figure out how to squeeze lots of
        talks into the schedule and we didn&rsquo;t know
        what could be shortened. In 2022, we asked
        people to provide quick outlines for 10-, 20-,
        and 40-minute formats if they wanted a long
        talk slot, which was helpful in prioritizing.
        In 2023, we just suggested 5/10 and 20-minute
        slots knowing that there would probably be
        plenty of time for Q&A, with some longer talks
        approved on a case-by-case basis. Because
        talks are (ideally) prerecorded, 20 minutes
        seems to be a good length to get into
        something interesting while leaving time for
        Q&A so that people could dive deeper into
        particular things that interested them.
-   Rolling acceptances were very nice. We gave
    people a week to respond, which felt a little
    quick sometimes, but gave people enough time to
    speak up to see if they had any red flags or
    suggestions.
-   We didn&rsquo;t extend the CFP period in 2023, great!
-   Two streams worked out well. Sunday morning had network congestion issues, so keep an eye out for those.
-   It’s easier for us to extend beyond 5pm than to go before 9am
    (especially for Pacific time).  Extending beyond 5pm puts strain on
    European organizers and volunteers, though.
-   Put the speaker introduction right after the talk description for easier web page copying.


<a id="org0e21b8c"></a>

## Announce

-   Post CFP to emacsconf-discuss, reddit.com/r/emacs, and in Emacs News
-   Submit to  <https://foss.events/>


<a id="org0b948ff"></a>

## Process proposals

1.  In the private repository, create a conf.org.
2.  Set `emacsconf-org-file` to its location.
3.  Create a heading for Submissions and set `CUSTOM_ID` to `submissions`.
4.  Prepare the `prepare` and `submit` pages on the wiki.

When proposals come in:

1.  Acknowledge with an e-mail along the lines of:
    
        Hello, ....!
        
        <feedback> We'll wait another week in case anyone else wants to
        chime in before I add it to the wiki and send further notes. :) Thanks for
        submitting this!

2.  Use `emacsconf-mail-add-submission` if it works, or create the entry and paste it in.
    `org-toggle-item` can help convert Org-style headings into a nested list.
3.  Set the following fields (`my/org-set-property`
    can be helpful): EMERGENCY, Q\_AND\_A,
    AVAILABILITY, NAME, PRONOUNS, TIME,
    SLUG, EMAIL, NAME\_SHORT, CUSTOM\_ID,
    TRACK, TIMEZONE, CATEGORY, DATE\_SUBMITTED,
    PUBLIC\_EMAIL


<a id="org709c243"></a>

## Accept proposals

Setup:

1.  Create YYYY/talks.md.

For each talk:

1.  Set up &ldquo;Talk abstract&rdquo; and &ldquo;Talk details&rdquo; sections in the conf.org talk subtree.
2.  Change status to `TO_CONFIRM`.
3.  Create wiki page with `emacsconf-publish-add-talk`.
4.  Use `emacsconf-publish-info-pages` and `emacsconf-publish-schedule` as needed.
5.  Send acceptance e-mail with `emacsconf-mail-accept-talk`


<a id="draft-schedule"></a>

## Draft schedule

-   Create entries for opening and closing remarks
    -   FIXED: t
    -   SCHEDULED
    -   TIME: 10
    -   BUFFER: 0
    -   CUSTOM\_ID:
    -   SLUG:
    -   Q\_AND\_A: pad
-   Create draft-schedule section in the public organizers notebook for the year
-   Add missing items to plan


<a id="orgdc12c78"></a>

## Set up backstage and upload

See <a href="https://git.emacsconf.org/emacsconf-ansible/tree/README.org">README.org</a>.

Lessons learned:

-   Next year, let&rsquo;s get upload.emacsconf.org in place as the domain name so that people don&rsquo;t try to FTP files to it.


<a id="orgce4ff13"></a>

## Post the schedule

-   <elisp:(setq emacsconf-publishing-phase 'schedule)>
-   <elisp:emacsconf-publish-info-pages>
-   <elisp:emacsconf-publish-schedule-org-files>
-   <elisp:emacsconf-ical-generate-all>
-   Update talks.md to refer to watch.md as well


<a id="org6cdaa28"></a>

## Process uploaded files

Basic setup:

1.  Set up the media server and symlinks: <a href="https://git.emacsconf.org/emacsconf-ansible/tree/README.org">README.org</a> (Media).
2.  Set up the prerec processing areas on res: <a href="https://git.emacsconf.org/emacsconf-ansible/tree/README.org">README.org</a> (Processing prerecs).
3.  Set up the caption processor on res: <a href="https://git.emacsconf.org/emacsconf-ansible/tree/README.org">README.org</a> (Captioning).

Process:

1.  Review uploaded files with <elisp:emacsconf-upload-dired>
2.  Open the JSON for the file you want to process and use <elisp:emacsconf-upload-copy-from-json> .
    If more than one file uses the same extension, provide a suffix to disambiguate them.
3.  On res:
    1.  Create a directory ~/current/files/$slug.
    2.  Copy the uploaded file from ~/cache as `$video_slug--original.$extension`, or use `rename-original.sh $slug $file`.
    3.  Call `process-prerec.sh $file`.
4.  When the screens are finished, do locally:
    1.  update-cache on my computer.
    2.  <elisp:emacsconf-publish-cache-video-data> or <elisp:emacsconf-cache-all-video-data>
    3.  <elisp:emacsconf-publish-backstage-index>


<a id="org3c7eeb9"></a>

## Generate assets


### In-between pages

-   Make the $year/assets/in-between directory and copy the template.svg from the previous year.
-   <elisp:emacsconf-stream-generate-in-between-pages>


### Intros

-   In an empty VTT file, <elisp:emacsconf-subed-intro-subtitles>
-   Record the audio and get captions
-   Splice together into a file that will be processed with subed-record-compile-video.


<a id="orgf502d28"></a>

## Prepare the infrastructure

Checklist:

-   [ ] IRC
    -   [ ] Manual announcements
    -   [ ] Automated announcements
-   [ ] Web conference
-   [ ] Streaming assets
    -   [ ] Overlays
    -   [ ] Intros
-   [ ] OBS
-   [ ] VNC
-   [ ] Publishing to the wiki
-   [ ] Publishing to the media server
-   [ ] Etherpad
-   [ ] Mumble
-   [ ] Icecast


<a id="org00f1270"></a>

## Getting ready for captions

Onboarding:

-   Add captioner to conf.org with the `:caption:` tag, under a Volunteers heading with the `volunteer` tag. Make sure the `EMAIL` property is set.
-   Add captioner to emacsconf-org mailing list.
-   Send backstage information <elisp:emacsconf-mail-backstage-info-to-volunteer> . (Can be sent to all captioning volunteers with <elisp:emacsconf-mail-backstage-info-to-captioning-volunteers>)


<a id="orga2f2794"></a>

# General infrastructure


<a id="icecast"></a>

## Icecast

Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv.

-   [X] Gen
-   [X] Dev

Also, recordings are available in /data.

New year:

1.  Reprovision with
    
        ansible-playbook -i inventory.yml prod-playbook.yml --tags stream
    
    so that the year is updated in the configuration.
2.  SSH to live0.emacsconf.org and run `/etc/init.d/emacsconf stop`. Confirm that icecast has stopped with `ps | grep emacsconf`; if not, kill the process.
    Start it again with `/etc/init.d/emacsconf start`
3.  To test, you can use FFmpeg like this for the gen stream:
    
        ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \
                                 -c:v libvpx -b:v 500K -crf 25  -b:a 128k \
                                 -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/gen.webm"
    
    For dev:
    
        ffmpeg -y -f lavfi -i testsrc=size=1280x720:rate=10 -i background-music.opus \
                                 -c:v libvpx -b:v 500K -crf 25  -b:a 128k \
                                 -content_type video/webm "icecast://emacsconf:$EMACSCONF_ICECAST_SOURCE_PASSWORD@live0.emacsconf.org:8001/dev.webm"
    
    and then view it with
    
        mpv https://live0.emacsconf.org/gen.webm


<a id="watch"></a>

## Watching pages

Success: You can watch at <https://live.emacsconf.org>.

-   [X] <https://live.emacsconf.org> goes to the current year&rsquo;s page.
-   [X] gen stream works in the browser
-   [X] dev stream works in the browser
-   [X] Schedule is correct

New year:

1.  ansible-playbook -i inventory.yml prod-playbook.yml &#x2013;tags live
2.  Republish with <elisp:emacsconf-publish-watch-pages>.
3.  Update front:/var/www/live.emacsconf.org.

Set up test streams (see ffmpeg instructions from [Icecast](#icecast)) and then:


<a id="vnc"></a>

## VNC

We use VNC to connect to the X servers on res.emacsconf.org so that we can stream from it.

Success:

-   [X] Confirm that you can connect to emacsconf-gen via VNC
-   [X] Confirm that you can connect to emacsconf-dev via VNC

Setting up

-   <elisp:emacsconf-publish-res-index>


<a id="vnc-instructions"></a>

### Instructions

NOTE: VNC+OBS doesn&rsquo;t work well if you have a window manager that
automatically resizes windows, like i3. Please configure your window
manager so that the VNC window is not resized.

1.  Install a VNC viewer on your system (ex: tigervnc-viewer).

2.  Set up your local environment:
    -   gen: `export TRACK=gen; export TRACK_PORT=5905; export SSH_PORT=46668`
    -   dev: `export TRACK=dev; export TRACK_PORT=5906; export SSH_PORT=46668`

3.  Copy the password:
    
    scp emacsconf-$TRACK@res.emacsconf.org:~/.vnc/passwd vnc-passwd-$TRACK -p $SSH\_PORT

4.  Forward your local ports and connect via VNC viewer to the
    appropriate forwarded port from your laptop:
    
        ssh emacsconf-$TRACK@res.emacsconf.org -N -L $TRACK_PORT:127.0.0.1:$TRACK_PORT -p $SSH_PORT &
        sleep 5   # Give it time to establish the tunnels
        xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK &

If you get the following error:

    channel 2: open failed: connect failed: Connection refused
    CConn:       End of stream
    CConn:       The connection was dropped by the server before the session could
                 be established.

then the VNC server hasn&rsquo;t started yet. You can start it with

    ssh emacsconf-$TRACK@res.emacsconf.org -p $SSH_PORT /home/emacsconf-$TRACK/bin/track-vnc

and then connect with:

    xvncviewer 127.0.0.1:$TRACK_PORT -shared -geometry 1280x720 -passwd vnc-passwd-$TRACK &


<a id="obs"></a>

## Streaming with OBS

Success: Confirm that you can stream

-   [X] gen
-   [X] dev

New year: reprovision with

    ansible-playbook -i inventory.yml prod-playbook.yml --tags obs

so that the year is updated in the shell scripts.


<a id="obs-instructions"></a>

### Instructions

1.  [Connect to the VNC session for the track.](#vnc-instructions)

2.  Start **recording** (not streaming). If you don&rsquo;t see OBS when you connect, it&rsquo;s probably on workspace 2, so you can switch with Alt-2. If you still don&rsquo;t see it there, you can open a terminal with Alt-Enter and then run `track-obs`. After you start recording, confirm that it is now broadcasting to the stream.

3.  Verify with MPV on your local system:
    
        mpv https://live0.emacsconf.org/$TRACK.webm &


### Icecast

Success: You can use [OBS+VNC to record](#obs-instructions), and the results can be viewed by mpv.

-   [ ] gen
-   [ ] dev

New year: reprovision with

    ansible-playbook -i inventory.yml prod-playbook.yml --tags stream

so that the year is updated in the configuration.

This is on live.emacsconf.org and can be restarted with `/etc/init.d/emacsconf restart`.


<a id="org93c38b4"></a>

## Shell scripts

Success: From the commandline, hosts can:

-   [X] play just the intro: intro $SLUG
-   [X] play a file without an intro: play $SLUG
-   [X] play a file with an intro: play-with-intro $SLUG (hmm, this didn&rsquo;t show the subtitles)
    ah, it&rsquo;s because the intros need to be in the cache dir?
-   [X] open the Q&A for a talk: handle-qa $SLUG
    -   [X] open the pad: pad $SLUG
    -   [X] open BBB: bbb $SLUG
-   [ ] do the whole thing: handle-session $SLUG
-   [ ] rebroadcast the other stream:
    -   rebroadcast gen
    -   rebroadcast dev
-   [ ] control background music: music, stop-music

Setup:

-   OBS must be streaming from the stream you&rsquo;re testing. [Streaming with OBS](#obs)
-   You may want to generate test assets and copy them over to the assets/stream directory.
-   Generate all the overlays and copy them to assets/overlays
-   Set up the assets/music directory


<a id="backstage"></a>

## Backstage

Update this with <elisp:emacsconf-publish-backstage-index>

Explanation of files:

<table>


<colgroup>
<col  class="org-left">

<col  class="org-left">
</colgroup>
<tbody>
<tr>
<td class="org-left">&#x2013;original.mp4/mov/webm</td>
<td class="org-left">original file as uploaded by the speaker</td>
</tr>

<tr>
<td class="org-left">&#x2013;silence.mp4/mov/webm</td>
<td class="org-left">silence sample if uploaded by speaker</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.webm</td>
<td class="org-left">converted to .webm format and compressed</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.opus</td>
<td class="org-left">audio only</td>
</tr>

<tr>
<td class="org-left">&#x2013;normalized.opus</td>
<td class="org-left">normalized audio</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.vtt</td>
<td class="org-left">WhisperX captions, WebVTT format</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.json</td>
<td class="org-left">WhisperX word data</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.srt</td>
<td class="org-left">WhisperX captions, SRT format</td>
</tr>

<tr>
<td class="org-left">&#x2013;reencoded.txt</td>
<td class="org-left">WhisperX captions, plain text</td>
</tr>

<tr>
<td class="org-left">&#x2013;main.vtt</td>
<td class="org-left">edited captions</td>
</tr>

<tr>
<td class="org-left">&#x2013;main&#x2013;chapters.vtt</td>
<td class="org-left">chapter markers</td>
</tr>

<tr>
<td class="org-left">&#x2013;main.txt</td>
<td class="org-left">captions as text (optional)</td>
</tr>

<tr>
<td class="org-left">&#x2013;main.webm</td>
<td class="org-left">version for streaming based on the most recent resources</td>
</tr>
</tbody>
</table>

Other files might also have been uploaded by the speaker, such as slides or notes.


<a id="org75fb0fd"></a>

## IRC web client

Success:

-   [X] You can join from <https://chat.emacsconf.org>
-   [X] You can join from <https://live.emacsconf.org> and the channels end up as the latest ones

Remember to schedule a task to ask libera.chat to increase the number of users it will accept from chat.emacsconf.org to avoid connection exhaustion errors.

Message I&rsquo;ve posted to support@libera.chat:

Thank you for running libera.chat. We&rsquo;re looking forward to using IRC
(#emacsconf-gen, #emacsconf-dev) for the discussions for the upcoming
EmacsConf (Dec 7-8, <https://emacsconf.org>). In the past, we&rsquo;ve needed to
coordinate with Libera staff to increase the number of connections
allowed from chat.emacsconf.org during conference days in order to avoid
connection exhaustion errors. Is that something we can plan now or would
you like me to bring it up a few days before the conference?


<a id="orgadd3307"></a>

## Etherpad

Success:

-   [X] pad.emacsconf.org redirects to this year&rsquo;s entry
-   [X] Each talk has its own Etherpad


<a id="media"></a>

## Publishing media to the server and to the wiki

Start of year:

1.  Set `media_protect_root` to true in Ansible `group_vars/all.yml`.
2.  `ansible-playbook -i inventory.yml prod-playbook.yml --tags media`
3.  Generate the index with `emacsconf-publish-update-media`

Confirm by setting a submitted talk to `PLAYING` and testing with
<elisp:emacsconf-publish-media-files-on-change> . The public media
directory should have the files and the entry should be in the index.
Switching it back to `TO_STREAM` and calling
<elisp:emacsconf-publish-media-files-on-change> should remove it.


### TODO Switch public media to unprotected root before the conference

1.  Clear public media directory.
2.  Set `media_protect_root` to false in Ansible `group_vars/all.yml`.
3.  `ansible-playbook -i inventory.yml prod-playbook.yml --tags media`

You can generate the index with `emacsconf-publish-update-media`.


<a id="org0b45c02"></a>

## Automated IRC announcements

Success:

-   [ ] When a talk starts playing, it is announced in the relevant channel


<a id="orgf31a707"></a>

## Manual IRC announcements

Success:

-   [X] You can /opall, /conftopic, and /broadcast


<a id="org2b9c281"></a>

## Low-res stream


<a id="org00999f1"></a>

## Restream to YouTube


<a id="org1d92e71"></a>

## Restream to Toobnix


<a id="other"></a>

# Other process notes


<a id="bbb-user"></a>

## Adding another user to BBB

    ssh bbb.emacsverse.org
    docker exec -it greenlight-v2 bundle exec rake user:create["USERNAME","EMAIL","PASSWORD","user"]


<a id="cancel-talk"></a>

## When a talk is cancelled

1.  Update the status to CANCELLED.
2.  Rejig the schedule if needed, and update with <elisp:emacsconf-schedule-update-from-info>
3.  Update the published schedule with <elisp:emacsconf-update-schedule>
4.  Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file.
5.  Sync the new intros to the /data/emacsconf/shared/$YEAR/assets/intros directory.
6.  Regenerate the pads: <elisp:emacsconf-pad-prepopulate-all-talks>


<a id="change-talk-qa"></a>

## When a talk Q&A changes method

1.  Update the talk property.
2.  Update the schedule with <elisp:emacsconf-update-schedule>
3.  Regenerate the intros (before, after). <elisp:emacsconf-stream-generate-in-between-pages>, then use <elisp:subed-record-compile-video> in the intros file.
4.  Sync the new intros to the /data/emacsconf/shared/$YEAR/assets/intros directory.
5.  Regenerate the pads: <elisp:emacsconf-pad-prepopulate-all-talks>
6.  Regenerate talks.json: <elisp:emacsconf-publish-talks-json-to-files>


<a id="orge2b5368"></a>

## Ansible

    git clone git@git.emacsconf.org:pub/emacsconf-ansible