path: root/README.org

This repository contains infrastructure-as-code ansible configurations
for various pieces of the EmacsConf infrastructure.

ansible-galaxy collection install community.general

Production: needs prod-vars.yml, see prod-vars.yml.sample
Docker: needs docker-vars.yml, see docker-vars.yml.sample


* Ansible setup

We have Python 3.7 on front/pad and Python 3.13 on res, so we need Ansible 2.14 and community.general 7.x.

#+begin_src sh
rm -rf .venv
uv venv --python 3.11
source .venv/bin/activate
uv pip install "ansible-core>=2.14.0,<2.15.0"
ansible-galaxy collection install "community.general:<8.0.0" --force
ansible-galaxy collection install community.mysql --force
ansible front,res -i inventory.yml -m ping
#+end_src

* How to use this playbook
:PROPERTIES:
:CUSTOM_ID: how-to-use-this-playbook
:END:

1. Install ansible on your local machine and check out this repo.
2. Copy the ansible_vars block from conf.org to prod-vars.yml in this repo (alongside inventory.yml), or set emacsconf-ansible-directory in Emacs and then use emacsconf-ansible-tangle-vars to tangle the file.
3. scp orga@res.emacsconf.org:~/authorized_keys .    (if you're setting up any user accounts)
4. Find the ansible-playbook command you want to run and try it out.

Debugging: add -v or -vv to the =ansible-playbook= command.

Due to the older version of SSH installed on front0, you will need to add =LogLevel QUIET= to your =~/.ssh/config=.

* Setting up a vault
:PROPERTIES:
:CUSTOM_ID: vault
:END:

You can store passwords in vault files if you like. [[https://stackoverflow.com/questions/37297249/how-to-store-ansible-become-pass-in-a-vault-and-how-to-use-it][More info]]

Put this text into =host_vars/media/plain= and =host_vars/upload/plain=:

#+begin_example
ansible_become_pass: "{{ vaulted_become_pass }}"
#+end_example

Use =ansible-vault create host_vars/media/crypted= and =ansible-vault create host_vars/upload/crypted= to create files with the contents:

#+begin_example
vaulted_become_pass: "yourpasswordhere"
#+end_example

To set the password for this console session:

#+begin_src sh :eval no
  export VAULT_PASSWORD=...
#+end_src

To change the password for a file:

#+begin_src sh :eval no
ansible-vault rekey $FILE --ask-vault-pass
#+end_src

* Processes
:PROPERTIES:
:CUSTOM_ID: processes
:END:

At the start of the conference preparation period, change
=emacsconf-year= in [[file:group_vars/all.yml]]

To start a local copy of the wiki for testing, see [[#wiki-docker][Ikiwiki - Docker]].

* Wiki
:PROPERTIES:
:CUSTOM_ID: wiki
:END:
** Ikiwiki
:PROPERTIES:
:CUSTOM_ID: wiki-ikiwiki
:END:
*** Prod
:PROPERTIES:
:CUSTOM_ID: wiki-ikiwiki-prod
:END:

When you update htmlscrubber.pm in wiki/templates or emacsconf.setup:

ansible-playbook -i inventory.yml prod-playbook.yml --tags wiki-plugins
ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags wiki-plugins

ansible-playbook -i inventory.yml prod-playbook.yml --tags wiki
*** Docker
:PROPERTIES:
:CUSTOM_ID: wiki-docker
:END:

Goal:
- [X] Load the wiki at http://localhost:28080
- [X] Add SSH key
- [X] Add as remote
- [X] Push to the wiki
- [X] Have the changes show up automatically
- [X] Have ansible copy the SSH key
  
file:/docker:emacsconf-front:/home/ikiwiki/emacsconf.setup

Creating:
ansible-playbook -i docker-inventory.yml docker-playbook.yml --tags wiki

Reusing:
ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags wiki

Restarting after a reboot:
docker restart emacsconf-front

Copying your SSH key:
set the docker_ssh_key Ansible variable to the path of your public key
or
docker cp ~/.ssh/id_rsa.pub emacsconf-front:/home/ikiwiki/.ssh/authorized_keys2
docker exec emacsconf-front chown ikiwiki:ikiwiki /home/ikiwiki/.ssh/authorized_keys2
docker exec emacsconf-front chmod 600 /home/ikiwiki/.ssh/authorized_keys2

http://localhost:28080/
ssh localhost -p 2222

docker exec -it emacsconf-front /bin/bash

git remote add docker ssh://ikiwiki@127.0.0.1:2222/var/www/wiki.git

Debugging
ssh wiki 'cd /var/www/wiki.git; git update-ref refs/heads/master HEAD^' && git push docker 2022-pages

Stuck wiki:
ssh ikiwiki@localhost -p 2222 ikiwiki --setup /home/ikiwiki/emacsconf.setup -v

* Processing prerecs
:PROPERTIES:
:CUSTOM_ID: processing-prerecs
:END:

1. Update =group_vars/all.yml=: set =emacsconf_year=.
2. In the conf.org file, call =M-x emacsconf-set-file-prefixes= to set the file prefixes. Tweak as needed.
3. Export the talks.json with =M-x emacsconf-ansible-export-talks=.
4. ansible-playbook -i inventory.yml prod-playbook.yml --tags prerec
5. ansible-playbook -i inventory.yml prod-playbook.yml --tags caption
6. ansible-playbook -i inventory.yml local-playbook.yml

When you receive a file, create a directory for it named =~/current/files/$slug=. Copy the uploaded file as =$video_slug--original.$extension=, or use =rename-original.sh $slug $file=.

Then call =process-prerec.sh $file=. It will launch some screen sessions for reencoding the file and creating the VTT.


* Setting up the backstage area
:PROPERTIES:
:CUSTOM_ID: setting-up-the-backstage-area
:END:

1. Doublecheck the media host in [[file:inventory.yml]] and the variables in [[file:roles/media/defaults/main.yml]].
2. ansible-playbook -i inventory.yml prod-playbook.yml --tags media --ask-become-pass
   (or =ansible-playbook -i inventory.yml prod-playbook.yml --tags media --ask-vault-pass= if you've [[#vault][stored it in a vault]])
3. Update the following variables in your Emacs configuration:
   - emacsconf-backstage-dir
   - emacsconf-backstage-phase
4. [[elisp:(let ((dir (format "~/proj/emacsconf/%s/cache" emacsconf-year))) (unless (file-directory-p dir) (make-directory dir)))][(Run)]] Create ~/proj/emacsconf/{year}/cache on your local system.
5. Copy over the include-in-index-footer.html and include-in-public-index.html templates from other years. (Backup: [[file:roles/media/templates/]]
6. elisp:emacsconf-publish-talks-json-to-files
7. [[elisp:emacsconf-publish-backstage-index]]

* Upload service
:PROPERTIES:
:CUSTOM_ID: upload-service
:END:

1. [ ] Check [[file:group_vars/all.yml]] to make sure emacsconf_year is set.
2. [ ] =ansible-playbook -i inventory.yml prod-playbook.yml --tags upload --ask-become-pass= (or =ansible-playbook -i inventory.yml prod-playbook.yml --tags upload --ask-vault-pass= if you've [[#vault][stored it in a vault]], or =ansible-playbook -i inventory.yml prod-playbook.yml --tags upload= if the vault password is already in the environment variable =VAULT_PASSWORD=.)
3. [ ] Upload a test file through https://upload.emacsconf.org/. The password is defined in the upload_password Ansible variable.
4. [ ] Check that [[file:~/proj/emacsconf/emacsconf-ansible/README.org::#setting-up-the-backstage-area][the backstage area has been set up]]. Note that files are not automatically copied to the backstage area.
5. [ ] Send speakers the e-mail about the upload service
   - elisp:emacsconf-mail-upload-and-backstage-info-to-waiting-for-prerecs
   - elisp:emacsconf-mail-upload-and-backstage-info if mailing individually

Other notes:

- /ssh:media|sudo:upload@media:~upload
- /ssh:media|sudo::/etc/nginx/sites-available
- sudo service upload start

* Publishing
:PROPERTIES:
:CUSTOM_ID: publishing
:END:

Goals:
- [X] Set up Emacs 28.2 or a newer one
- [X] Check out the repositories
- [X] Load the configuration
- [X] Publish the backstage index
- [X] Publish the watchpages
- [X] Publish schedule to the wiki and push
- [ ] Have nice interactive setup
- [ ] Publish backstage index on a hook
- [ ] Connect to IRC and announce talks
- [ ] Push talk info the text files on the stream
- [ ] Start mpv in the right display
- [ ] Publish the prerec files
- [ ] Publish the prerec on the page

** Prod
:PROPERTIES:
:CUSTOM_ID: publishing-prod
:END:
To run the playbook and publish the main schedule:

#+begin_src sh
ansible-playbook -i inventory.yml prod-playbook.yml --tags wiki-publish --extra-vars='{"force_publish": true}'
#+end_src

#+RESULTS:
:results:
:end:


Update a specific talk's before/nav and the main schedule: (ex: wayland)
ansible-playbook playbook.yml -e '{"slug": "wayland"}' -i inventory.yml --tags publish

Force-publish the schedule:
ansible-playbook -i inventory.yml prod-playbook.yml --tags publish -e force_publish=true
** Development
:PROPERTIES:
:CUSTOM_ID: publishing-development
:END:
** Docker
:PROPERTIES:
:CUSTOM_ID: publishing-docker
:END:
Creating:
ansible-playbook -i docker-inventory.yml docker-playbook.yml --tags wiki,publish

Reusing:
ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags publish

With docker:
https://stackoverflow.com/questions/24738264/how-to-test-ansible-playbook-using-docker
* Pad
:PROPERTIES:
:CUSTOM_ID: pad
:END:

Before generating pads, use elisp:emacsconf-publish-talks-json-to-files to create the talks.json used.

** Production
:PROPERTIES:
:CUSTOM_ID: pad-production
:END:
ansible-playbook -i inventory.yml prod-playbook.yml --tags pad,proxy

To fall back to wikimedia rewrite:
ansible-playbook -i inventory.yml prod-playbook.yml --tags proxy --extra-vars='{"use_wikimedia": true}'
You can still access pads directly with direct/p like this:
https://pad.emacsconf.org/direct/p/2022-journalism

To undo wikimedia rewrite:
ansible-playbook -i inventory.yml prod-playbook.yml --tags proxy 

To prepare for a load test:
ansible-playbook -i inventory.yml prod-playbook.yml --tags pad --extra-vars='{"etherpad_load_test": true}'

To create pads:
ansible-playbook -i inventory.yml prod-playbook.yml --tags create-pads
** Docker
:PROPERTIES:
:CUSTOM_ID: pad-docker
:END:
Creating:
ansible-playbook -i docker-inventory.yml docker-playbook.yml --tags pad

Reusing an existing container:
ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags pad

Connecting:
docker exec -it emacsconf-pad /bin/bash

Creating pads
ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags create-pads

file:/docker:emacsconf-pad:/home/etherpad/etherpad/

Getting the API key

#+NAME: pad-key
#+begin_src sh
docker exec  emacsconf-pad cat /home/etherpad/etherpad/APIKEY.txt
#+end_src

#+RESULTS: pad-key
:results:
b7a15dc34cc7f6917cca6cd9a2b4b92145af7c7cd9b341af34869ab8cd3568be
:end:


#+begin_src sh :var padkey=pad-key
echo curl "http://localhost:9001/api/1/createPad?apikey=$padkey&padID=emacsconf-2022"
curl "http://localhost:9001/api/1/createPad?apikey=$padkey&padID=emacsconf-2022"
#+end_src

#+RESULTS:
:results:
curl http://localhost:9001/api/1/createPad?apikey=b7a15dc34cc7f6917cca6cd9a2b4b92145af7c7cd9b341af34869ab8cd3568be&padID=emacsconf-2022
{"code":0,"message":"ok","data":null}
:end:
** Useful
:PROPERTIES:
:CUSTOM_ID: pad-useful
:END:
https://github.com/systemli/ansible-role-etherpad
https://gist.github.com/aaronpk/7307172
* Pad proxy
:PROPERTIES:
:CUSTOM_ID: pad-proxy
:END:

ansible-playbook -i inventory.yml prod-playbook.yml --tags proxy --extra-vars='{"use_wikimedia": false}'

ansible-playbook -i inventory.yml prod-playbook.yml --tags proxy --extra-vars='{"use_wikimedia": true}'

* Stream
:PROPERTIES:
:CUSTOM_ID: stream
:END:
** Prod
:PROPERTIES:
:CUSTOM_ID: stream-prod
:END:
Setting up icecast:

ansible-playbook -i inventory.yml prod-playbook.yml --tags stream



** Testing
:PROPERTIES:
:CUSTOM_ID: stream-testing
:END:

Runs the ffmpeg command on res

ansible-playbook -i inventory.yml prod-playbook.yml --tags test -e icecast_test_file=/home/orga/test.webm -e icecast_test=file -e icecast_test_track=dev

Play the stream with MPV:

ansible-playbook -i inventory.yml prod-playbook.yml --tags test -e icecast_test_track=dev -e icecast_test=mpv

Use a test pattern (don't know if this works)

ansible-playbook -i inventory.yml prod-playbook.yml --tags test -e icecast_test_track=dev -e icecast_test=pattern
** Creating the fallback files
:PROPERTIES:
:CUSTOM_ID: stream-creating-the-fallback-files
:END:
ffmpeg -y -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -loop 1 -r 20 -t 10 -i sorry.png -c:v libvpx -c:a libvorbis -color_primaries 1 -color_trc 1 -colorspace 1 -crf 30 -g 120 -minrate 1.5M -b:v 1500 -g 120 -maxrate 1.5M -cluster_time_limit 5100 -shortest sorry.webm

ffmpeg -y -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -loop 1 -r 20 -t 10 -i sorry.png -vf scale=854:480 -c:v libvpx -c:a libvorbis -color_primaries 1 -color_trc 1 -colorspace 1 -crf 30 -g 120 -minrate 1.5M -b:v 1500 -g 120 -maxrate 1.5M -cluster_time_limit 5100 -shortest sorry-480p.webm

* OBS
:PROPERTIES:
:CUSTOM_ID: obs
:END:

ansible-playbook -i inventory.yml prod-playbook.yml --tags obs


Resizing VNC after connection
xrandr -s 1280x720
** Firefox
:PROPERTIES:
:CUSTOM_ID: obs-firefox
:END:
*** Firefox profiles like to be created in an X environment
:PROPERTIES:
:CUSTOM_ID: obs-firefox-firefox-profiles-like-to-be-created-in-an-x-environment
:END:
firefox -no-remote -CreateProfile "{{ emacsconf_id }}-{{ item.item.id }}
*** Install Tampermonkey extension and scripts                     :manual:
:PROPERTIES:
:CUSTOM_ID: obs-firefox-install-tampermonkey-extension-and-scripts
:END:

For each track:

1. Install the Tampermonkey extension by going to https://addons.mozilla.org/en-US/firefox/addon/tampermonkey/ .

2. Install the BBB script by clicking on the Tampermonkey extension, choosing *Install New Script*, and using this as the URL. Enable checking for updates.

   https://live.emacsconf.org/emacsconf-tampermonkey-bbb.js

   (these are updated by the Ansible playbook tag "static")

   Press =Ctrl+s= to save.

3. Add this script for IRC:

   https://live.emacsconf.org/emacsconf-tampermonkey-irc.js

4. Join an BBB meeting and switch out of full-screen with F11. Check the address bar to see if autoplay is disabled (crossed-out autoplay icon). If it is, click on it and change *Block audio* to *Allow audio and video*.



** How to update scenes from the gen copy
:PROPERTIES:
:CUSTOM_ID: obs-how-to-update-scenes-from-the-gen-copy
:END:
ssh emacsconf-gen@res.emacsconf.org -p 46668 "cat ~/.config/obs-studio/basic/scenes/emacsconf.json" | jq 'walk(if type == "string" then gsub("emacsconf"; "{{ emacsconf_id }}") else . end)' > roles/obs/templates/scenes.json
ansible-playbook -i inventory.yml prod-playbook.yml --tags obs-scene

* Media
:PROPERTIES:
:CUSTOM_ID: media
:END:

ansible-playbook -i inventory.yml prod-playbook.yml --tags media


* Captioning
:PROPERTIES:
:CUSTOM_ID: captioning
:END:

Set up whisper:

ansible-playbook -i inventory.yml prod-playbook.yml --tags caption

Update caption script:

ansible-playbook -i inventory.yml prod-playbook.yml --tags process-captions

ffmpeg -y -i handwritten/reencode.webm -t 60 -vcodec copy -acodec copy test.webm 
* Other useful things
:PROPERTIES:
:CUSTOM_ID: other-useful-things
:END:

nodemon -w . -e yml -x 'ansible-playbook -i inventory.yml prod-playbook.yml --tags vnc; true'

* Restreaming
:PROPERTIES:
:ID:       20251122T182719.845166
:CUSTOM_ID: restreaming
:END:

Add something like this to your ~prod-vars.yml~:

#+begin_src emacs-lisp
restreaming_platforms:
  - name: youtube
    streams:
      - name: gen
        key: xxxx-xxxx-xxxx-xxxx-xxxx
        url: https://www.youtube.com/watch?v=xxxxxxxxxxx
        studio: https://studio.youtube.com/video/xxxxxxxxxxx/livestreaming
        source: gen.webm
      - name: dev
        key: xxxx-xxxx-xxxx-xxxx-xxxx
        url: https://www.youtube.com/watch?v=xxxxx-xxxxx
        studio: https://studio.youtube.com/video/xxxxx-xxxxx/livestreaming
        source: dev.webm
      - name: test
        key: xxxx-xxxx-xxxx-xxxx-xxxx
        studio: https://studio.youtube.com/video/xxxxxxxxxxx/livestreaming
        url: https://youtu.be/xxxxxxxxxxx
        source: gen.webm
    stream_url: rtmp://a.rtmp.youtube.com/live2
    backup_stream: rtmp://b.rtmp.youtube.com/live2?backup=1
  - name: toobnix
    stream_url: rtmp://toobnix.org:1935/live
    streams:
      - name: gen
        key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
        url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx
        source: gen.webm
      - name: dev
        key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
        url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx
        source: dev.webm
      - name: test
        key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
        url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx
        source: gen.webm
#+end_src

This will set up scripts on the live0 server.

It doesn't get automatically started, so you'll also need to call ~screen -S restream-$SHIFT_ID-youtube~ and ~screen -S restream-$TRACK_ID-toobnix~.
* BBB
:PROPERTIES:
:CUSTOM_ID: bbb
:END:
ansible-playbook -i inventory.yml prod-playbook.yml --tags bbb
* Local testing with vagrant
:PROPERTIES:
:CUSTOM_ID: local-testing-with-vagrant
:END:

Rough notes on local testing

#+begin_src ruby :tangle test/front/Vagrantfile
# -*- mode: ruby -*-
Vagrant.configure("2") do |config|
  config.vm.box = "debian/bookworm64"
  config.vm.define "pad" do |pad|
  end
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "2048"
  end
  config.vm.network "private_network", ip: "192.168.56.2"
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "../../vagrant-playbook.yml"
  end
end
#+end_src

In ~test/front~: ~vagrant up --provision~

ansible-galaxy role install systemli.etherpad

nodemon -e '*' -w ../../roles/pad/ -x "vagrant destroy -f && vagrant up --provision"

Backing up:
ansible-playbook -i inventory.yml backup-playbook.yml