diff options
83 files changed, 1567 insertions, 1175 deletions
@@ -4,3 +4,4 @@ vnc-passwd-dev vnc-passwd-gen vnc-password vnc-password-gen +backups @@ -37,9 +37,14 @@ vaulted_become_pass: "yourpasswordhere" To set the password for this console session: #+begin_src sh :eval no - export ANSIBLE_PASSWORD=... + export VAULT_PASSWORD=... #+end_src +To change the password for a file: + +#+begin_src sh :eval no +ansible-vault rekey $FILE --ask-vault-pass +#+end_src * Processes @@ -52,7 +57,7 @@ To start a local copy of the wiki for testing, see [[#wiki-docker][Ikiwiki - Doc ** Ikiwiki *** Prod -When you update htmlscrubber.pm in wiki/templates: +When you update htmlscrubber.pm in wiki/templates or emacsconf.setup: ansible-playbook -i inventory.yml prod-playbook.yml --tags wiki-plugins ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags wiki-plugins @@ -109,6 +114,7 @@ ssh ikiwiki@localhost -p 2222 ikiwiki --setup /home/ikiwiki/emacsconf.setup -v 3. Export the talks.json with =M-x emacsconf-ansible-export-talks=. 4. ansible-playbook -i inventory.yml prod-playbook.yml --tags prerec 5. ansible-playbook -i inventory.yml prod-playbook.yml --tags caption +6. ansible-playbook -i inventory.yml local-playbook.yml When you receive a file, create a directory for it named =~/current/files/$slug=. Copy the uploaded file as =$video_slug--original.$extension=, or use =rename-original.sh $slug $file=. @@ -123,6 +129,9 @@ Then call =process-prerec.sh $file=. It will launch some screen sessions for ree 3. Update the following variables in your Emacs configuration: - emacsconf-backstage-dir - emacsconf-backstage-phase +4. Create ~/proj/emacsconf/{year}/cache +5. elisp:emacsconf-publish-talks-json-to-files +6. [[elisp:emacsconf-publish-backstage-index]] * Upload service =ansible-playbook -i inventory.yml prod-playbook.yml --tags upload --ask-become-pass= @@ -169,7 +178,7 @@ ansible-playbook playbook.yml -e '{"slug": "wayland"}' -i inventory.yml --tags p Force-publish the schedule: ansible-playbook -i inventory.yml prod-playbook.yml --tags publish -e force_publish=true - +** Development ** Docker Creating: ansible-playbook -i docker-inventory.yml docker-playbook.yml --tags wiki,publish @@ -180,6 +189,9 @@ ansible-playbook -i docker-inventory.yml docker-reuse-playbook.yml --tags publis With docker: https://stackoverflow.com/questions/24738264/how-to-test-ansible-playbook-using-docker * Pad + +Before generating pads, use elisp:emacsconf-publish-talks-json-to-files to create the talks.json used. + ** Production ansible-playbook -i inventory.yml prod-playbook.yml --tags pad,proxy @@ -194,6 +206,8 @@ ansible-playbook -i inventory.yml prod-playbook.yml --tags proxy To prepare for a load test: ansible-playbook -i inventory.yml prod-playbook.yml --tags pad --extra-vars='{"etherpad_load_test": true}' +To create pads: +ansible-playbook -i inventory.yml prod-playbook.yml --tags create-pads ** Docker Creating: ansible-playbook -i docker-inventory.yml docker-playbook.yml --tags pad @@ -283,78 +297,17 @@ For each track: 1. Install the Tampermonkey extension by going to https://addons.mozilla.org/en-US/firefox/addon/tampermonkey/ . -2. Install the BBB script by clicking on the Tampermonkey extension, choosing *Install New Script*, and pasting in the following: - - #+begin_src js :eval no - // ==UserScript== - // @name Emacsconf BBB setup - // @namespace https://emacsconf.org/ - // @version 0.1 - // @description Join BBB and set things up - // @author You - // @match https://bbb.emacsverse.org/* - // @icon https://www.google.com/s2/favicons?sz=64&domain=emacsverse.org - // @grant none - // ==/UserScript== - ( - async function() { - 'use strict'; - const NAME = 'emacsconf'; - async function waitUntil(conditionFunc, interval=500, timeout=null) { - let initResult = conditionFunc(); - if (initResult) return initResult; - return new Promise((resolve, reject) => { - let timeSoFar = 0; - let timer = setInterval(() => { - let result = conditionFunc(); - if (result) { - clearInterval(timer); - resolve(result); - } - timeSoFar += interval; - if (timeout && timeSoFar > timeout) { - clearInterval(timer); - reject(); - } - }, interval); - }); - } - if (document.querySelector('input.join-form')) { - document.querySelector('input.join-form').value = NAME; - document.querySelector('#room-join').click(); - return; - } - await waitUntil(() => document.querySelector('.icon-bbb-listen')).then((e) => e.closest('button').click()); - await waitUntil(() => document.querySelector('.icon-bbb-user')).then((e) => e.closest('button').click()); - })(); - #+end_src +2. Install the BBB script by clicking on the Tampermonkey extension, choosing *Install New Script*, and using this as the URL. Enable checking for updates. + + https://live.emacsconf.org/emacsconf-tampermonkey-bbb.js + + (these are updated by the Ansible playbook tag "static") Press =Ctrl+s= to save. 3. Add this script for IRC: - #+begin_src js :eval no -// ==UserScript== -// @name Connect to EmacsConf chat automatically -// @namespace https://emacsconf.org/ -// @version 0.1 -// @description try to take over the world! -// @author You -// @match https://chat.emacsconf.org/* -// @icon https://www.google.com/s2/favicons?sz=64&domain=emacsconf.org -// @grant none -// ==/UserScript== - -(function() { - 'use strict'; - setTimeout(() => { - if (document.querySelector('.connect-row')) { - document.querySelector('.connect-row').closest('form').querySelector('button').click(); - } - }, 1000); -})(); - #+end_src - + https://live.emacsconf.org/emacsconf-tampermonkey-irc.js 4. Join an BBB meeting and switch out of full-screen with F11. Check the address bar to see if autoplay is disabled (crossed-out autoplay icon). If it is, click on it and change *Block audio* to *Allow audio and video*. @@ -367,6 +320,8 @@ ansible-playbook -i inventory.yml prod-playbook.yml --tags obs-scene * Media ansible-playbook -i inventory.yml prod-playbook.yml --tags media + + * Captioning Set up whisper: @@ -381,3 +336,82 @@ ffmpeg -y -i handwritten/reencode.webm -t 60 -vcodec copy -acodec copy test.webm * Other useful things nodemon -w . -e yml -x 'ansible-playbook -i inventory.yml prod-playbook.yml --tags vnc; true' + +* Restreaming +:PROPERTIES: +:ID: 20251122T182719.845166 +:END: + +Add something like this to your ~prod-vars.yml~: + +#+begin_src emacs-lisp +restreaming_platforms: + - name: youtube + streams: + - name: gen + key: xxxx-xxxx-xxxx-xxxx-xxxx + url: https://www.youtube.com/watch?v=xxxxxxxxxxx + studio: https://studio.youtube.com/video/xxxxxxxxxxx/livestreaming + source: gen.webm + - name: dev + key: xxxx-xxxx-xxxx-xxxx-xxxx + url: https://www.youtube.com/watch?v=xxxxx-xxxxx + studio: https://studio.youtube.com/video/xxxxx-xxxxx/livestreaming + source: dev.webm + - name: test + key: xxxx-xxxx-xxxx-xxxx-xxxx + studio: https://studio.youtube.com/video/xxxxxxxxxxx/livestreaming + url: https://youtu.be/xxxxxxxxxxx + source: gen.webm + stream_url: rtmp://a.rtmp.youtube.com/live2 + backup_stream: rtmp://b.rtmp.youtube.com/live2?backup=1 + - name: toobnix + stream_url: rtmp://toobnix.org:1935/live + streams: + - name: gen + key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx + source: gen.webm + - name: dev + key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx + source: dev.webm + - name: test + key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + url: https://toobnix.org/w/xxxxxxxxxxxxxxxxxxxxxx + source: gen.webm +#+end_src + +This will set up scripts on the live0 server. + +It doesn't get automatically started, so you'll also need to call ~screen -S restream-$SHIFT_ID-youtube~ and ~screen -S restream-$TRACK_ID-toobnix~. +* BBB +ansible-playbook -i inventory.yml prod-playbook.yml --tags bbb +* Local testing with vagrant + +Rough notes on local testing + +#+begin_src ruby :tangle test/front/Vagrantfile +# -*- mode: ruby -*- +Vagrant.configure("2") do |config| + config.vm.box = "debian/bookworm64" + config.vm.define "pad" do |pad| + end + config.vm.provider "virtualbox" do |vb| + vb.memory = "2048" + end + config.vm.network "private_network", ip: "192.168.56.2" + config.vm.provision "ansible" do |ansible| + ansible.playbook = "../../vagrant-playbook.yml" + end +end +#+end_src + +In ~test/front~: ~vagrant up --provision~ + +ansible-galaxy role install systemli.etherpad + +nodemon -e '*' -w ../../roles/pad/ -x "vagrant destroy -f && vagrant up --provision" + +Backing up: +ansible-playbook -i inventory.yml backup-playbook.yml diff --git a/backup-playbook.yml b/backup-playbook.yml new file mode 100644 index 0000000..5ac234d --- /dev/null +++ b/backup-playbook.yml @@ -0,0 +1,60 @@ +# backup_playbook.yml +--- +- name: Load vars + hosts: all + tags: always + tasks: + - include_vars: + file: prod-vars.yml +- name: Backup Etherpad + hosts: pad + remote_user: "{{ emacsconf_user }}" + become: false + vars_files: + - roles/pad/vars/main.yml + tasks: + - name: "Ensure the backup directory exists on the remote server" + ansible.builtin.file: + path: "{{ backup_dir_on_server }}" + state: directory + mode: '0755' + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + - name: "Backup MySQL database" + community.mysql.mysql_db: + name: "{{ etherpad_mysql_database_name }}" + state: dump + target: "{{ backup_dir_on_server }}{{ etherpad_mysql_database_name }}.sql" + login_user: "{{ etherpad_mysql_database_user }}" + login_password: "{{ etherpad_mysql_database_password }}" + register: db_backup_result + - name: "Change ownership of the backup file" + ansible.builtin.file: + path: "{{ backup_dir_on_server }}{{ etherpad_mysql_database_name }}.sql" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + mode: '0644' + become: true + - name: "Log the backup status" + ansible.builtin.debug: + msg: "Database backup completed successfully. File is at {{ backup_dir_on_server }}{{ etherpad_mysql_database_name }}.sql" + +- name: "Retrieve backup from prod and save locally" + hosts: localhost # This part runs on your local machine + connection: local + + tasks: + - name: "Ensure the local backup directory exists" + ansible.builtin.file: + path: "{{ local_backup_dir }}" + state: directory + mode: '0755' + + - name: "Fetch the database backup file" + ansible.posix.synchronize: + src: "{{ hostvars['pad'].ansible_ssh_user }}@{{ hostvars['pad'].ansible_host }}:{{ backup_dir_on_server }}{{ etherpad_mysql_database_name }}.sql" + dest: "{{ local_backup_dir }}" + mode: pull + archive: yes + compress: yes + rsync_path: "rsync" diff --git a/common-playbook.yml b/common-playbook.yml index 2822e41..9cb44a3 100644 --- a/common-playbook.yml +++ b/common-playbook.yml @@ -1,5 +1,5 @@ - name: Set up wiki - hosts: front + hosts: wiki tags: wiki roles: - wiki @@ -13,16 +13,16 @@ tags: edit roles: - edit +- name: Set up proxy + hosts: pad + tags: proxy, pad-proxy + roles: + - pad-proxy - name: Set up pad hosts: pad tags: pad roles: - pad -- name: Set up proxy - hosts: pad - tags: proxy - roles: - - pad-proxy - name: Set up upload container hosts: upload tags: upload @@ -56,6 +56,11 @@ tags: media roles: - media +# - name: Set up BigBlueButton +# hosts: bbb +# tags: bbb +# roles: +# - bbb - name: Set up OBS hosts: obs tags: obs @@ -66,3 +71,8 @@ tags: live roles: - live +- name: Set up lounge + hosts: front + tags: lounge + roles: + - lounge diff --git a/group_vars/all.yml b/group_vars/all.yml index 82d4c77..7865420 100644 --- a/group_vars/all.yml +++ b/group_vars/all.yml @@ -2,9 +2,10 @@ docker: false res_x: 1280 res_y: 720 fps: 30 -emacsconf_year: 2023 +emacsconf_year: 2025 emacsconf_name: EmacsConf emacsconf_id: emacsconf +emacsconf_domain: emacsconf.org emacsconf_user: orga emacsconf_group: orga emacsconf_tracks: @@ -34,8 +35,15 @@ emacs_config_dir: ~{{ emacsconf_user }}/.emacs.d emacsconf_el_dir: ~{{ emacsconf_user }}/emacsconf-el emacsconf_edit_wiki_dir: ~{{ emacsconf_user }}/emacsconf-wiki emacsconf_private_dir: ~{{ emacsconf_user }}/emacsconf-{{ emacsconf_year }}-private -emacsconf_caption_dir: /data/emacsconf/{{ emacsconf_year }} +emacsconf_caption_dir: /data/emacsconf/shared/{{ emacsconf_year }} emacsconf_timezone: US/Eastern etherpad_server_name: pad.emacsconf.org +emacsconf_qa_start_open: true test_mode: false media_protect_root: true +protect_stream_with_password: false +restreaming_platforms: [] + +update_cache: true +backup_dir_on_server: "~{{emacsconf_user}}/backups/" +local_backup_dir: "backups/" diff --git a/host_vars/media/plain b/host_vars/media/plain new file mode 100644 index 0000000..e9f3e32 --- /dev/null +++ b/host_vars/media/plain @@ -0,0 +1 @@ +ansible_become_pass: "{{ vaulted_become_pass }}" diff --git a/host_vars/upload/plain b/host_vars/upload/plain new file mode 100644 index 0000000..e9f3e32 --- /dev/null +++ b/host_vars/upload/plain @@ -0,0 +1 @@ +ansible_become_pass: "{{ vaulted_become_pass }}" diff --git a/inventory.yml b/inventory.yml index 6d87c8f..aa8732d 100644 --- a/inventory.yml +++ b/inventory.yml @@ -6,6 +6,13 @@ prod: ansible_become: true emacsconf_group: org cpus: 12 + bbb: + ansible_host: bbb.emacsverse.org + ansible_python_interpreter: /usr/bin/python3 + ansible_become: true + ansible_ssh_user: root + emacsconf_group: org + cpus: 4 publish: ansible_host: res.emacsconf.org ansible_python_interpreter: /usr/bin/python3 @@ -24,12 +31,20 @@ prod: ansible_ssh_user: orga ansible_python_interpreter: /usr/bin/python3 ansible_become: true + update_cache: false + wiki: + ansible_host: front0.emacsconf.org + remote_user: orga + ansible_ssh_user: orga + ansible_python_interpreter: /usr/bin/python3 + ansible_become: true pad: ansible_host: front0.emacsconf.org remote_user: orga ansible_ssh_user: orga ansible_python_interpreter: /usr/bin/python3 ansible_become: true + update_cache: false media: ansible_host: media.emacsconf.org remote_user: sachac diff --git a/local-playbook.yml b/local-playbook.yml index 37cd294..8875a2b 100644 --- a/local-playbook.yml +++ b/local-playbook.yml @@ -10,7 +10,11 @@ tasks: - include_vars: file: local-vars.yml -- name: Set up Emacs +# - name: Set up Emacs +# hosts: localhost +# roles: +# - wiki-publish +- name: Set up cache directory hosts: localhost roles: - - wiki-publish + - local diff --git a/roles/base/files/keys/sachac b/roles/base/files/keys/sachac index 999b59c..643bd03 100644 --- a/roles/base/files/keys/sachac +++ b/roles/base/files/keys/sachac @@ -1,2 +1,3 @@ ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK0Vg112xS0SAuCutincht2LWs+2jC8EWC19Irotv8M0ztzLf6wmXEw0xoB8D78LKzXGC/gFcIvYzsNezHFpU5PmlxYBRJkdOYH2zYfnlWQFpJKmk1OelTrugaRE4HywXurf6q6Sot5hzbzPmCWgOlBZshnkDXMAyPCfYvL+RcwTRJWiaiGwwDHlfHCkebr4cwypRQ7Nl2kKajdp4wZXwbuP64pPNMmftZEMEM910w3zPnzQTil4IuLSiVC8K7TSk6xsnrsk10Y6zfoaHkZ71OD58rqPPFqeHYDj8SAvp6W4hHwakbf+r8nfRfr8Tc+gtCf0B6a4Y050OI5FxHlmjh +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCM41Zid5BjgXwEEnuSSLDvuWDqs3FXPAGwWxV9aY4uHb21+05rsbgAddXtxfj3kJd9tOz97nz5zEyet5bMiOxrh0w7R/LRMCRtiCerVd8ABpDnRJ1INXgAO0tOudVpmBwwDPp2njUbNW+POPBD6s5TXINPFK/V2bjTXkuYhmO4/6QS4OJZYMjugkqxk+JjMtF/e2+HR6UPAWXXyKRTPOQlfSQre/+bWSkU41oONuy4kXeXjiB+zQxRvcHptH+bk0v37jUWPN6PKSCoAlKPXxvGM86eDRF+Rs2fr7WJpkhsuVHaiVla6kBhaEKb9tDpSPg9twJZtl5si4cCEG8kP3Cv sachac-surface ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyMf4V8eCzYNEde8xG4tIJPBv8NwoTzyRG9O5+Bl69osaHV7OZQz81wXil1qZ/xrUu6fc5jMkxq7j5KCCs2MF6gMq12UKe9ESKYe5i+jFL7+V6JNQqcjLcyaEfEFtFCJ95nWCQWpXrMPijvpB3+YxLspFOTz8ZJsGENXU+Rkz5EIdx2VTgHUbddCjE5jndIO58uPKmR4EpMeUWxb20xYLpOwM14aGF/ERVjI++dIwu7mc21kxg42HJjRA/NRV48IxrGl57KKzl7qtMrqwp+ucoLWw4PdqHk4/tApjmrgLiJzLpSZx/4LL3mHTg3I6w9fC5yTgk3k6rJFomb2Jbboxx diff --git a/roles/bbb/README.md b/roles/bbb/README.md new file mode 100644 index 0000000..225dd44 --- /dev/null +++ b/roles/bbb/README.md @@ -0,0 +1,38 @@ +Role Name +========= + +A brief description of the role goes here. + +Requirements +------------ + +Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required. + +Role Variables +-------------- + +A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well. + +Dependencies +------------ + +A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles. + +Example Playbook +---------------- + +Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too: + + - hosts: servers + roles: + - { role: username.rolename, x: 42 } + +License +------- + +BSD + +Author Information +------------------ + +An optional section for the role authors to include contact information, or a website (HTML is not allowed). diff --git a/roles/bbb/defaults/main.yml b/roles/bbb/defaults/main.yml new file mode 100644 index 0000000..17c44b0 --- /dev/null +++ b/roles/bbb/defaults/main.yml @@ -0,0 +1,4 @@ +bbb_docker_repo_dir: /data/emacsconf/shared/bbb-docker-repo +bbb_docker_dir: /data/emacsconf/shared/bbb-docker +bbb_domain: bbb.emacsverse.org +bbb_ip: 207.66.177.26 diff --git a/roles/bbb/handlers/main.yml b/roles/bbb/handlers/main.yml new file mode 100644 index 0000000..40aff70 --- /dev/null +++ b/roles/bbb/handlers/main.yml @@ -0,0 +1,2 @@ +--- +# handlers file for bbb diff --git a/roles/bbb/meta/main.yml b/roles/bbb/meta/main.yml new file mode 100644 index 0000000..c572acc --- /dev/null +++ b/roles/bbb/meta/main.yml @@ -0,0 +1,52 @@ +galaxy_info: + author: your name + description: your role description + company: your company (optional) + + # If the issue tracker for your role is not on github, uncomment the + # next line and provide a value + # issue_tracker_url: http://example.com/issue/tracker + + # Choose a valid license ID from https://spdx.org - some suggested licenses: + # - BSD-3-Clause (default) + # - MIT + # - GPL-2.0-or-later + # - GPL-3.0-only + # - Apache-2.0 + # - CC-BY-4.0 + license: license (GPL-2.0-or-later, MIT, etc) + + min_ansible_version: 2.1 + + # If this a Container Enabled role, provide the minimum Ansible Container version. + # min_ansible_container_version: + + # + # Provide a list of supported platforms, and for each platform a list of versions. + # If you don't wish to enumerate all versions for a particular platform, use 'all'. + # To view available platforms and versions (or releases), visit: + # https://galaxy.ansible.com/api/v1/platforms/ + # + # platforms: + # - name: Fedora + # versions: + # - all + # - 25 + # - name: SomePlatform + # versions: + # - all + # - 1.0 + # - 7 + # - 99.99 + + galaxy_tags: [] + # List tags for your role here, one per line. A tag is a keyword that describes + # and categorizes the role. Users find roles by searching for tags. Be sure to + # remove the '[]' above, if you add tags to this list. + # + # NOTE: A tag is limited to a single word comprised of alphanumeric characters. + # Maximum 20 tags per role. + +dependencies: [] + # List your role dependencies here, one per line. Be sure to remove the '[]' above, + # if you add dependencies to this list. diff --git a/roles/bbb/tasks/main.yml b/roles/bbb/tasks/main.yml new file mode 100644 index 0000000..1a4b115 --- /dev/null +++ b/roles/bbb/tasks/main.yml @@ -0,0 +1,18 @@ +--- +- name: Create group + group: + name: "{{ emacsconf_group }}" + state: present +- name: Create user + user: + name: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + state: present +- name: Add public key for authorized access + ansible.posix.authorized_key: + user: "{{ emacsconf_user }}" + state: present + key: '{{ item }}' + with_file: + - ../../base/files/keys/sachac + - ../../base/files/keys/orga diff --git a/roles/bbb/templates/env.template b/roles/bbb/templates/env.template new file mode 100644 index 0000000..65e8770 --- /dev/null +++ b/roles/bbb/templates/env.template @@ -0,0 +1,202 @@ +# ==================================== +# ADDITIONS to BigBlueButton +# ==================================== +# (place a '#' before to disable them) + +# HTTPS Proxy +# fully automated Lets Encrypt certificates +ENABLE_HTTPS_PROXY=true +# If your network doesn't allow access to DNS at 8.8.8.8 specify your own resolvers +#RESOLVER_ADDRESS=x.x.x.x + +# coturn (a TURN Server) +# requires either the abhove HTTPS Proxy to be enabled +# or TLS certificates to be mounted to container +ENABLE_COTURN=true +#COTURN_TLS_CERT_PATH= +#COTURN_TLS_KEY_PATH= + +# Greenlight Frontend +# https://docs.bigbluebutton.org/greenlight/gl-overview.html +ENABLE_GREENLIGHT=true + +# Enable Webhooks +# used by some integrations +#ENABLE_WEBHOOKS=true + +# Prometheus Exporter +# serves the bigbluebutton-exporter under following URL: +# https://yourdomain/bbb-exporter +#ENABLE_PROMETHEUS_EXPORTER=true +#ENABLE_PROMETHEUS_EXPORTER_OPTIMIZATION=true + +# Recording +# IMPORTANT: this is currently a big privacy issues, because it will +# record everything which happens in the conference, even when the button +# suggets, that it does not. +# https://github.com/bigbluebutton/bigbluebutton/issues/9202 +# make sure that you get peoples consent, before they join a room +ENABLE_RECORDING=true +#REMOVE_OLD_RECORDING=false +#RECORDING_MAX_AGE_DAYS=14 + +# ==================================== +# SECRETS +# ==================================== +# important! change these to any random values +SHARED_SECRET={{ bbb_shared_secret }} +ETHERPAD_API_KEY={{ bbb_etherpad_api_key }} +RAILS_SECRET={{ bbb_rails_secret }} +POSTGRESQL_SECRET={{ bbb_postgresql_secret }} +FSESL_PASSWORD={{ bbb_fsesl_password }} + + + +# ==================================== +# CONNECTION +# ==================================== + +DOMAIN={{ bbb_domain }} + +EXTERNAL_IPv4={{ bbb_ip }} +EXTERNAL_IPv6= + +# STUN SERVER +# stun.freeswitch.org +STUN_IP={{ bbb_ip }} +STUN_PORT=3478 + +# TURN SERVER +# uncomment and adjust following two lines to add an external TURN server +TURN_SERVER=turns:{{ bbb_domain }}:5349?transport=tcp +TURN_SECRET={{ bbb_turn_secret }} + +# Allowed SIP IPs +# due to high traffic caused by bots, by default the SIP port is blocked. +# but you can allow access by your providers IP or IP ranges (comma seperated) +# Hint: if you want to allow requests from every IP, you can use 0.0.0.0/0 +SIP_IP_ALLOWLIST= + + +# ==================================== +# CUSTOMIZATION +# ==================================== + +CLIENT_TITLE=BigBlueButton + +# use following lines to replace the default welcome message and footer +WELCOME_MESSAGE="Welcome to <b>%%CONFNAME%%</b>!<br><br>For help on using BigBlueButton see these (short) <a href='https://www.bigbluebutton.org/html5' target='_blank'><u>tutorial videos</u></a>.<br><br>To join the audio bridge click the speaker button. Use a headset to avoid causing background noise for others." +WELCOME_FOOTER="This server is running <a href='https://docs.bigbluebutton.org/'' target='_blank'><u>BigBlueButton</u></a>." + +# use following line for an additional SIP dial-in message +#WELCOME_FOOTER="This server is running <a href='https://docs.bigbluebutton.org/' target='_blank'><u>BigBlueButton</u></a>. <br><br>To join this meeting by phone, dial:<br> INSERT_YOUR_PHONE_NUMBER_HERE<br>Then enter %%CONFNUM%% as the conference PIN number." + +# for a different default presentation, place the pdf file in ./conf/ and +# adjust the following path +DEFAULT_PRESENTATION=./mod/nginx/default.pdf + +# language of sound announcements +# options: +# - en-ca-june - EN Canadian June +# - en-us-allison - US English Allison +# - en-us-callie - US English Callie (default) +# - de-de-daedalus3 - German by Daedalus3 (https://github.com/Daedalus3/freeswitch-german-soundfiles) +# - es-ar-mario - Spanish/Argentina Mario +# - fr-ca-june - FR Canadian June +# - pt-br-karina - Brazilian Portuguese Karina +# - ru-RU-elena - RU Russian Elena +# - ru-RU-kirill - RU Russian Kirill +# - ru-RU-vika - RU Russian Viktoriya +# - sv-se-jakob - Swedish (Sweden) Jakob +# - zh-cn-sinmei - Chinese/China Sinmei +# - zh-hk-sinmei - Chinese/Hong Kong Sinmei +SOUNDS_LANGUAGE=en-us-callie + +# set to false to disable listenOnlyMode +LISTEN_ONLY_MODE=true + +# set to true to disable echo test +DISABLE_ECHO_TEST=false + +# set to true to automatically share webcam +AUTO_SHARE_WEBCAM=false + +# set to true to disable video preview for webcam sharing +DISABLE_VIDEO_PREVIEW=false + +# set to false to disable chat +CHAT_ENABLED=true + +# set to true to start chat closed +CHAT_START_CLOSED=false + +# set to true to disable announcements "You are now (un-)muted" +DISABLE_SOUND_MUTED=false + +# set to true to disable announcement "You are the only person in this conference" +DISABLE_SOUND_ALONE=false + +# maximum count of breakout rooms per meeting +# Warning: increasing the limit of breakout rooms per meeting +# can generate excessive overhead to the server. We recommend +# this value to be kept under 12. +BREAKOUTROOM_LIMIT=8 + +# set to false to disable the learning dashboard +ENABLE_LEARNING_DASHBOARD=true + +# ==================================== +# Tuning +# ==================================== +# Default = 2; Min = 1; Max = 4 +# On powerful systems with high number of meetings you can set values up to 4 to accelerate handling of events +NUMBER_OF_BACKEND_NODEJS_PROCESSES=2 + +# Default = 2; Min = 1; Max = 8 +# Set a number between 1 and 4 times the value of NUMBER_OF_BACKEND_NODEJS_PROCESSES where higher number helps with meetings +# stretching the recommended number of users in BigBlueButton +NUMBER_OF_FRONTEND_NODEJS_PROCESSES=2 + + +# ==================================== +# GREENLIGHT CONFIGURATION +# ==================================== + +### SMTP CONFIGURATION +# Emails are required for the basic features of Greenlight to function. +# Please refer to your SMTP provider to get the values for the variables below +#SMTP_SENDER_EMAIL= +#SMTP_SENDER_NAME= +#SMTP_SERVER= +#SMTP_PORT= +#SMTP_DOMAIN=bbb.emacsverse.org +#SMTP_USERNAME= +#SMTP_PASSWORD= +#SMTP_AUTH= +#SMTP_STARTTLS_AUTO=true +#SMTP_STARTTLS=false +#SMTP_TLS=false +#SMTP_SSL_VERIFY=true + +### EXTERNAL AUTHENTICATION METHODS +# +#OPENID_CONNECT_CLIENT_ID= +#OPENID_CONNECT_CLIENT_SECRET= +#OPENID_CONNECT_ISSUER= +#OPENID_CONNECT_REDIRECT= + +# To enable hCaptcha on the user sign up and sign in, define these 2 keys +#HCAPTCHA_SITE_KEY= +#HCAPTCHA_SECRET_KEY= + +# Set these if you are using a Simple Storage Service (S3) +# Uncomment S3_ENDPOINT only if you are using a S3 OTHER than Amazon Web Service (AWS) S3. +#S3_ACCESS_KEY_ID= +#S3_SECRET_ACCESS_KEY= +#S3_REGION= +#S3_BUCKET= +#S3_ENDPOINT= + +# Define the default locale language code (i.e. 'en' for English) from the fallowing list: +# [en, ar, fr, es] +#DEFAULT_LOCALE=en diff --git a/roles/bbb/tests/inventory b/roles/bbb/tests/inventory new file mode 100644 index 0000000..878877b --- /dev/null +++ b/roles/bbb/tests/inventory @@ -0,0 +1,2 @@ +localhost + diff --git a/roles/bbb/tests/test.yml b/roles/bbb/tests/test.yml new file mode 100644 index 0000000..72d74f0 --- /dev/null +++ b/roles/bbb/tests/test.yml @@ -0,0 +1,5 @@ +--- +- hosts: localhost + remote_user: root + roles: + - bbb diff --git a/roles/bbb/vars/main.yml b/roles/bbb/vars/main.yml new file mode 100644 index 0000000..de701b8 --- /dev/null +++ b/roles/bbb/vars/main.yml @@ -0,0 +1,2 @@ +--- +# vars file for bbb diff --git a/roles/caption/tasks/main.yml b/roles/caption/tasks/main.yml index fea78f4..96b3198 100644 --- a/roles/caption/tasks/main.yml +++ b/roles/caption/tasks/main.yml @@ -6,19 +6,22 @@ - cmake - jq - inotify-tools + - pipx - name: Install whisper - ansible.builtin.pip: + community.general.pipx: name: git+https://github.com/openai/whisper.git - name: Install Python packages - ansible.builtin.pip: - name: - - lhotse - - webvtt-py - - tqdm - - torchaudio - - num2words + community.general.pipx: + name: "{{ item }}" + install_deps: true + loop: + - lhotse + - webvtt-py + - tqdm + - torchaudio + - num2words - name: Set up aeneas - include: aeneas.yml + include_tasks: aeneas.yml - name: Create group group: name: "{{ emacsconf_group }}" diff --git a/roles/caption/templates/process-captions.py b/roles/caption/templates/process-captions.py index a42439b..bdbfa1e 100755 --- a/roles/caption/templates/process-captions.py +++ b/roles/caption/templates/process-captions.py @@ -40,7 +40,7 @@ import json import torch THREADS = {{ cpus }} -VIDEO_REGEXP = '\\.(webm|mov|mp4|mkv)$' +VIDEO_REGEXP = '\\.(webm|mov|mp4|mkv|mpv)$' AUDIO_REGEXP = '\\.(ogg|opus)$' ALWAYS = False TRIM_AUDIO = False @@ -114,8 +114,8 @@ def extract_audio(work): log("Extracting audio from %s acodec %s" % (work['video'], acodec)) output = subprocess.check_output(['ffmpeg', '-y', '-i', work['video'], '-acodec', acodec, '-vn', new_file], stderr=subprocess.STDOUT) work['audio'] = new_file - if os.path.isfile("/data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh"): - subprocess.call(["/data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh", work['audio']]) + if os.path.isfile("/data/emacsconf/admin/{{ emacsconf_year }}/scripts/upload.sh"): + subprocess.call(["/data/emacsconf/admin/{{ emacsconf_year }}/scripts/upload.sh", work['audio']]) return work def to_sec(time_str): @@ -150,8 +150,8 @@ def generate_captions(work): txt_writer(result, work['audio'], {'max_line_width': 60, 'max_line_count': None, 'highlight_words': None}) work['vtt'] = new_file work['txt'] = work['base'] + '.txt' - if os.path.isfile("/data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh"): - subprocess.call(["/data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh", work['vtt'], work['txt']]) + if os.path.isfile("/data/emacsconf/admin/{{ emacsconf_year }}/scripts/upload.sh"): + subprocess.call(["/data/emacsconf/admin/{{ emacsconf_year }}/scripts/upload.sh", work['vtt'], work['txt']]) if 'srv2' in work: del work['srv2'] return work diff --git a/roles/edit/templates/emacsconf-edit.el b/roles/edit/templates/emacsconf-edit.el index 195170a..603d0a3 100644 --- a/roles/edit/templates/emacsconf-edit.el +++ b/roles/edit/templates/emacsconf-edit.el @@ -84,6 +84,9 @@ (unless (and (boundp 'server-clients) server-clients) (server-start)) (find-file "{{ emacsconf_private_dir }}/conf.org") +(setq emacsconf-cache-dir "{{ emacsconf_caption_dir }}/cache") +(setq case-fold-search t) (emacsconf-add-org-after-todo-state-change-hook) +(setq emacsconf-publishing-phase 'conference) (unless noninteractive (emacsconf-erc-connect)) (setq tab-width 2) diff --git a/roles/live/tasks/main.yml b/roles/live/tasks/main.yml index 3e51765..f49b2fd 100644 --- a/roles/live/tasks/main.yml +++ b/roles/live/tasks/main.yml @@ -1,23 +1,25 @@ - name: Create the directory for this year file: - path: /var/www/live.emacsconf.org/{{ item }} + path: /var/www/{{ emacsconf_live_domain }}/{{ item }} owner: "{{ emacsconf_user }}" group: "{{ emacsconf_user }}" state: directory loop: - "{{ emacsconf_year }}" - "{{ emacsconf_year }}/watch" -- name: Configure Nginx +- name: Set up nginx.conf template: src: live.emacsconf.org.conf - dest: /etc/nginx/sites-available/live.emacsconf.org -- name: Make sure main configuration is enabled - file: - src: /etc/nginx/sites-available/live.emacsconf.org - dest: /etc/nginx/sites-enabled/live.emacsconf.org - force: no - state: link -- name: Reload configuration + dest: /etc/nginx/sites-available/{{ emacsconf_live_domain }} +- name: Reload nginx service: name: nginx state: reloaded +- name: Static files + tags: static + template: + src: "{{ item }}" + dest: "/var/www/{{ emacsconf_live_domain }}/{{ item }}" + loop: + - emacsconf-tampermonkey-bbb.js + - emacsconf-tampermonkey-irc.js diff --git a/roles/lounge/defaults/main.yml b/roles/lounge/defaults/main.yml new file mode 100644 index 0000000..26fbaaa --- /dev/null +++ b/roles/lounge/defaults/main.yml @@ -0,0 +1,4 @@ +lounge_domain: chat.emacsconf.org +irc_network: irc.libera.chat +irc_port: 6697 +irc_channels: "#emacsconf, #emacsconf-gen, #emacsconf-dev, and #emacsconf-org" diff --git a/roles/lounge/tasks/main.yml b/roles/lounge/tasks/main.yml new file mode 100644 index 0000000..8c6593f --- /dev/null +++ b/roles/lounge/tasks/main.yml @@ -0,0 +1,40 @@ +- name: Create directory + file: + path: /var/www/{{ lounge_domain }} + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + state: directory +- name: Set up error page + template: + src: error.html + dest: /var/www/{{ lounge_domain }}/error.html + mode: 0755 + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" +- name: Create main configuration if needed + template: + src: nginx-site-config + dest: /etc/nginx/sites-available/{{ lounge_domain }} +- name: Make sure main configuration is enabled + file: + src: /etc/nginx/sites-available/{{ lounge_domain }} + dest: /etc/nginx/sites-enabled/{{ lounge_domain }} + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + force: no + state: link +- name: Reload configuration + become: true + service: + name: nginx + state: reloaded +- name: Install systemd configuration + tags: system + become: true + template: + src: thelounge.service + dest: /etc/systemd/system/thelounge.service + owner: root + group: root + mode: 0755 + when: not use_initd diff --git a/roles/lounge/templates/error.html b/roles/lounge/templates/error.html new file mode 100644 index 0000000..5e2a093 --- /dev/null +++ b/roles/lounge/templates/error.html @@ -0,0 +1,37 @@ +<!DOCTYPE html> +<html lang="en"> +<head> + <meta charset="UTF-8"> + <title>{{lounge_domain}} currently unavailable</title> + <style> + body { padding: 20px; font-family: Arial, sans-serif } + </style> +</head> +<body> + <h1>Web-based chat is currently unavailable (code 502)</h1> + + <p>The web-based IRC interface will be available during <a href="https://{{emacsconf_domain}}/{{emacsconf_year}}">{{emacsconf_name}} {{emacsconf_year}}</a>. Please check back then. In the meantime, you can also connect to {{irc_network}}:{{irc_port}} with your favorite IRC client and join {{irc_channels}}. + + <p>This page will refresh when the site is back. You can also check <a href="https://status.emacsconf.org">status.emacsconf.org</a> for updates.</p> + + <noscript>Your browser doesn’t support javascript. Please try refreshing the page manually every few minutes.</noscript> + + <script src="//ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script> + <script> + var retryCurrent = 30, retryMax = 60 * 5; + async function check() { + let result = await fetch(window.location.href, { method: HEAD }).then(resp => { + if (resp.status == 200) { + window.location.reload(true); + } else if (resp.status == 502) { + if (retryCurrent < retryMax) { + retryCurrent = Math.min(retryCurrent * 2, retryMax); + } + setTimeout(check, retryCurrent * 1000); + } + }); + } + setTimeout(check_response, retryCurrent * 1000); + </script> +</body> +</html> diff --git a/roles/lounge/templates/nginx-site-config b/roles/lounge/templates/nginx-site-config new file mode 100644 index 0000000..cc6aa4d --- /dev/null +++ b/roles/lounge/templates/nginx-site-config @@ -0,0 +1,63 @@ +upstream backend { + server 127.0.0.1:9000; +} + +proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=thelounge_cache:10m max_size=3g inactive=120m use_temp_path=off; + + +server { + listen 80; + listen [::]:80; + server_name chat.emacsconf.org; + + include snippets/well-known-acme-challenge.conf; + + location /js-licenses.html { root /var/www/{{lounge_domain}}; } + + location / { + return 302 https://$server_name$request_uri; + } +} + +server { + listen 443 ssl http2; + listen [::]:443 ssl http2; + server_name {{lounge_domain}}; + + include /etc/nginx/tls/{{emacsconf_domain}}.conf; + + location /js-licenses.html { root /var/www/{{lounge_domain}}; } + error_page 502 /error.html; + location = /error.html { + root /var/www/{{lounge_domain}}; + } + location / { + proxy_pass http://backend; + proxy_http_version 1.1; + proxy_set_header Connection "upgrade"; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header X-Forwarded-For $remote_addr; + proxy_set_header X-Forwarded-Proto $scheme; + + # by default nginx times out connections in one minute + proxy_read_timeout 1d; + } + +} + +#server { +# listen 443 ssl http2; +# listen [::]:443 ssl http2; +# server_name chat.emacsconf.org; +# +# include /etc/nginx/tls/emacsconf.org.conf; +# root /var/www/chat.emacsconf.org; +#} + +server { + listen 80; + listen [::]:80; + server_name chat.emacsconf.com chat.emacsconf.net; + + return 302 $scheme://{{lounge_domain}}$request_uri; +} diff --git a/roles/lounge/templates/thelounge.service b/roles/lounge/templates/thelounge.service new file mode 100644 index 0000000..abc4868 --- /dev/null +++ b/roles/lounge/templates/thelounge.service @@ -0,0 +1,11 @@ +[Unit] +Description=The Lounge IRC Client +After=network.target + +[Service] +ExecStart=/usr/bin/thelounge start +User=thelounge +WorkingDirectory=/var/lib/thelounge + +[Install] +WantedBy=multi-user.target diff --git a/roles/media/tasks/main.yml b/roles/media/tasks/main.yml index 8cd854d..e9daaef 100644 --- a/roles/media/tasks/main.yml +++ b/roles/media/tasks/main.yml @@ -11,6 +11,10 @@ file: path: /var/www/{{ media_server_name }}/{{ emacsconf_year }}/backstage state: directory +- name: Ensure current directory exists + file: + path: /var/www/{{ media_server_name }}/{{ emacsconf_year }}/current + state: directory - name: Create group group: name: "{{ emacsconf_group }}" @@ -68,3 +72,48 @@ service: name: nginx state: reloaded +- name: Symlink the current year's backstage directory + file: + src: /var/www/{{ media_server_name }}/{{ emacsconf_year }}/backstage + dest: "~{{ emacsconf_user }}/backstage" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + state: link +- name: Symlink the current year's directory + file: + src: /var/www/{{ media_server_name }}/{{ emacsconf_year }} + dest: "~{{ emacsconf_user }}/{{ emacsconf_year }}" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + state: link +- name: Symlink the current year's directory as current + file: + src: /var/www/{{ media_server_name }}/{{ emacsconf_year }} + dest: "~{{ emacsconf_user }}/current" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + state: link +- name: Create the bin directory + tags: media-scripts + file: + state: directory + path: "~{{ emacsconf_user }}/bin" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" +- name: Add the bin directory to the path + tags: media-scripts + lineinfile: + dest: "~{{ emacsconf_user }}/.bashrc" + state: present + line: "export PATH=$PATH:~/bin" +- name: Create batch scripts + tags: media-scripts + template: + src: "{{ item }}" + dest: "~{{ emacsconf_user }}/bin/{{ item }}" + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + mode: 0755 + loop: + - bbb-open + - bbb-before diff --git a/roles/media/templates/nginx-include b/roles/media/templates/nginx-include index b42cacd..4d1eabe 100644 --- a/roles/media/templates/nginx-include +++ b/roles/media/templates/nginx-include @@ -1,22 +1,41 @@ rewrite ^/current/bbb-open.html$ {{ bbb_open_url }} redirect; location /{{ emacsconf_year }}/backstage { - auth_basic "Restricted"; - auth_basic_user_file /etc/nginx/sites-available/{{ media_server_name }}-{{ emacsconf_year }}-htpasswd; - autoindex on; - rewrite ^/{{ emacsconf_year }}/backstage/current/pad/([^/]*)$ https://{{ etherpad_server_name }}/{{ emacsconf_year }}-$1 redirect; - rewrite ^/{{ emacsconf_year }}/backstage/current/room/([^/]*)$ https://{{ media_server_name }}/{{ emacsconf_year }}/backstage/assets/redirects/open/bbb-$1.html redirect; - rewrite ^/{{ emacsconf_year }}/backstage/current/([^/]*)/pad/?$ https://{{ etherpad_server_name }}/{{ emacsconf_year }}-$1 redirect; - rewrite ^/{{ emacsconf_year }}/backstage/current/([^/]*)/room/?$ https://{{ media_server_name }}/{{ emacsconf_year }}/backstage/assets/redirects/open/bbb-$1.html redirect; - } + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/sites-available/{{ media_server_name }}-{{ emacsconf_year }}-htpasswd; + autoindex on; + rewrite ^/{{ emacsconf_year }}/backstage/current/pad/([^/]*)$ https://{{ etherpad_server_name }}/{{ emacsconf_year }}-$1 redirect; + rewrite ^/{{ emacsconf_year }}/backstage/current/room/([^/]*)$ https://{{ media_server_name }}/{{ emacsconf_year }}/backstage/assets/redirects/open/bbb-$1.html redirect; + rewrite ^/{{ emacsconf_year }}/backstage/current/([^/]*)/pad/?$ https://{{ etherpad_server_name }}/{{ emacsconf_year }}-$1 redirect; + rewrite ^/{{ emacsconf_year }}/backstage/current/([^/]*)/room/?$ https://{{ media_server_name }}/{{ emacsconf_year }}/backstage/assets/redirects/open/bbb-$1.html redirect; + add_header Cache-Control no-cache; + if_modified_since off; + expires off; + etag off; +} + location ~* \.vtt$ { + add_header Cache-Control "no-cache, no-store, must-revalidate"; + add_header Pragma "no-cache"; + add_header Expires "0"; + } + location /{{ emacsconf_year }}/{{ emacsconf_id }}.ics { - auth_basic off; - } + auth_basic off; + add_header Cache-Control "no-cache, no-store, must-revalidate"; + add_header Pragma "no-cache"; + add_header Expires "0"; +} location /{{ emacsconf_year }}/schedules/ { auth_basic off; + add_header Cache-Control "no-cache, no-store, must-revalidate"; + add_header Pragma "no-cache"; + add_header Expires "0"; } {% for track in emacsconf_tracks %} location /{{ emacsconf_year }}/{{ emacsconf_id }}-{{ track.id }}.ics { auth_basic off; + add_header Cache-Control "no-cache, no-store, must-revalidate"; + add_header Pragma "no-cache"; + add_header Expires "0"; } {% endfor %} {% if media_protect_root %} diff --git a/roles/media/templates/nginx-site-config b/roles/media/templates/nginx-site-config index 4dbdaaa..ba25cba 100644 --- a/roles/media/templates/nginx-site-config +++ b/roles/media/templates/nginx-site-config @@ -2,4 +2,9 @@ server { listen 80; server_name {{ media_server_name }}; root /var/www/{{ media_server_name }}; -}
\ No newline at end of file + location ~* \.vtt$ { + add_header Cache-Control "no-cache, no-store, must-revalidate"; + add_header Pragma "no-cache"; + add_header Expires "0"; + } +} diff --git a/roles/obs/defaults/main.yml b/roles/obs/defaults/main.yml index 6eb9451..d95b900 100644 --- a/roles/obs/defaults/main.yml +++ b/roles/obs/defaults/main.yml @@ -3,5 +3,7 @@ ff_vcustom: rt cpu-used=5 threads=2 error-resilient=1 crf=30 g=120 minrate=1.5M ff_vbitrate: 1500 ff_vgopsize: 120 obs_profile_path: /home/{{ emacsconf_user }}/.config/obs-studio/basic/profiles -emacsconf_asset_dir: /data/{{ emacsconf_id }}/assets +emacsconf_asset_dir: /data/{{ emacsconf_id }}/shared/{{ emacsconf_year }}/assets mumble_server: mumble.emacsconf.org +background_music_dir: "{{ emacsconf_asset_dir }}/music" +background_music_volume: 30 diff --git a/roles/obs/tasks/main.yml b/roles/obs/tasks/main.yml index 249a0bb..95f12b6 100644 --- a/roles/obs/tasks/main.yml +++ b/roles/obs/tasks/main.yml @@ -47,11 +47,21 @@ template: src: xorg.conf dest: /etc/X11/xorg.conf +- name: Create directories + file: + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + path: "{{ item }}" + state: directory + mode: 0775 + loop: + - "{{ emacsconf_asset_dir }}" + - "{{ emacsconf_asset_dir }}/stream" - name: Set up MPV and MPVC tags: mpv - include: mpv.yml + include_tasks: mpv.yml - name: Set up track-specific things - include: track.yml + include_tasks: track.yml loop: "{{ emacsconf_tracks }}" - debug: var: emacsconf_home @@ -82,21 +92,29 @@ line: export TZ={{ emacsconf_timezone }} dest: "/home/{{ emacsconf_user }}/.bashrc" - name: Allow sudo from {{ emacsconf_user }} to the stream users + tags: obs-sudo become: true become_user: root - copy: - content: | - {% for item in emacsconf_tracks %} - {{ emacsconf_user }} ALL=({{ emacsconf_id }}-{{ item.id }}) NOPASSWD: ALL - {% endfor %} - dest: /etc/sudoers.d/50_emacsconf -- name: Create directories - file: - owner: "{{ emacsconf_user }}" - group: "{{ emacsconf_group }}" - path: "{{ emacsconf_asset_dir }}" - state: directory - mode: 0775 + community.general.sudoers: + name: "{{ emacsconf_user }}-{{ emacsconf_id }}-{{ item.id }}" + user: "{{ emacsconf_user }}" + runas: "{{emacsconf_id }}-{{ item.id }}" + commands: ALL + nopassword: true + with_items: + - "{{ emacsconf_tracks }}" +- name: Allow sudo from the stream users to the {{ emacsconf_user }} + tags: obs-sudo + become: true + become_user: root + community.general.sudoers: + name: "{{ emacsconf_id }}-{{ item.id }}-{{ emacsconf_user }}" + user: "{{emacsconf_id }}-{{ item.id }}" + runas: "{{ emacsconf_user }}" + commands: ALL + nopassword: true + with_items: + - "{{ emacsconf_tracks }}" - name: Set up scripts tags: wip, obs-scripts template: @@ -106,9 +124,13 @@ loop: - overlay - music + - stop-music - play - play-with-intro - intro - bbb - pad + - handle-qa - handle-session + - reset-state + - rebroadcast diff --git a/roles/obs/tasks/obs-setup.yml b/roles/obs/tasks/obs-setup.yml index 5b0fba4..4e53e8c 100644 --- a/roles/obs/tasks/obs-setup.yml +++ b/roles/obs/tasks/obs-setup.yml @@ -71,11 +71,3 @@ loop: - obs-track - obs-cli-track -- name: Fix permissions - become: true - become_user: root - file: - path: "~{{ emacsconf_user }}" - owner: "{{ emacsconf_user }}" - group: "{{ emacsconf_group }}" - recurse: true diff --git a/roles/obs/tasks/track.yml b/roles/obs/tasks/track.yml index d615336..cbd4b39 100644 --- a/roles/obs/tasks/track.yml +++ b/roles/obs/tasks/track.yml @@ -9,10 +9,15 @@ var: emacsconf_home - name: Set up user tags: obs-track-scripts - include: user.yml + include_tasks: user.yml - name: Set up user-related things become_user: "{{ emacsconf_user }}" block: + - name: Set up symlink to current + file: + src: "~{{ old_emacsconf_user }}/current" + dest: "~{{ emacsconf_user }}/current" + state: link - name: Set up track bins for addition to paths file: path: "~{{ emacsconf_user }}/bin/{{ item.id }}" @@ -20,7 +25,7 @@ group: "{{ emacsconf_group }}" state: directory - name: Set up VNC - include: tigervnc.yml + include_tasks: tigervnc.yml - name: Create MPV profile directory file: path: "~{{ emacsconf_user }}/.config/mpv" @@ -65,7 +70,7 @@ # shell: jackd -r -ddummy # async: 2592000 - name: Set up pulse - include: pulse.yml + include_tasks: pulse.yml tags: pulse - name: Set up I3 directory file: @@ -95,10 +100,10 @@ group: "{{ emacsconf_group }}" - name: Set up Emacs configuration tags: wip - include: emacs.yml + include_tasks: emacs.yml - name: Set up symbolic links file: - src: "/data/{{ emacsconf_id }}/assets/stream" + src: "/data/{{ emacsconf_id }}/shared/{{ emacsconf_year }}/assets/stream" dest: "~{{ emacsconf_user }}/stream" state: link diff --git a/roles/obs/templates/bbb b/roles/obs/templates/bbb index c12702c..b7c508f 100755 --- a/roles/obs/templates/bbb +++ b/roles/obs/templates/bbb @@ -3,17 +3,13 @@ # {{ ansible_managed }} # Kill the background music if playing -if screen -list | grep -q background; then - screen -S background -X quit -fi +/usr/local/bin/reset-state # Update the overlay SLUG=$1 overlay $SLUG -killall -s TERM firefox-esr firefox https://media.emacsconf.org/{{ emacsconf_year }}/backstage/assets/redirects/open/bbb-$SLUG.html & sleep 5 -xdotool search --class firefox windowactivate --sync xdotool key Return xdotool key F11 wait diff --git a/roles/obs/templates/emacsconf-stream-config.el b/roles/obs/templates/emacsconf-stream-config.el index bd5ffd7..2aa886b 100644 --- a/roles/obs/templates/emacsconf-stream-config.el +++ b/roles/obs/templates/emacsconf-stream-config.el @@ -13,7 +13,9 @@ (setq emacsconf-stream-track "{{ item.name }}") (autoload 'text-property-search-forward "text-property-search") (add-to-list 'load-path "~/emacsconf-el") +(setq tab-width 2) +(setq case-fold-search t) +(setq emacsconf-cache-dir "{{ emacsconf_caption_dir }}/cache") (require 'emacsconf) (require 'emacsconf-stream) (emacsconf-stream-display-clock-and-countdown) -(setq tab-width 2) diff --git a/roles/obs/templates/i3-config b/roles/obs/templates/i3-config index 1787b64..31a7b40 100644 --- a/roles/obs/templates/i3-config +++ b/roles/obs/templates/i3-config @@ -23,7 +23,7 @@ font pango:monospace 8 # xss-lock grabs a logind suspend inhibit lock and will use i3lock to lock the # screen before suspend. Use loginctl lock-session to lock your screen. -exec --no-startup-id xss-lock --transfer-sleep-lock -- i3lock --nofork +# exec --no-startup-id xss-lock --transfer-sleep-lock -- i3lock --nofork # NetworkManager is the most popular way to manage wireless networks on Linux, # and nm-applet is a desktop environment-independent system tray GUI for it. diff --git a/roles/obs/templates/intro b/roles/obs/templates/intro index bf02dc3..9bf1161 100755 --- a/roles/obs/templates/intro +++ b/roles/obs/templates/intro @@ -8,7 +8,7 @@ fi SLUG=$1 FILE=$1 if [[ ! -f $FILE ]]; then - LIST=({{ emacsconf_caption_dir }}/assets/stream/emacsconf-{{ emacsconf_year }}-$FILE--*.webm) + LIST=({{ emacsconf_caption_dir }}/cache/emacsconf-{{ emacsconf_year }}-$FILE--*--intro.webm) FILE="${LIST[0]}" BY_SLUG=1 else @@ -16,8 +16,10 @@ else fi shift overlay $SLUG -if [[ -f {{ emacsconf_caption_dir }}/assets/intros/$SLUG.webm ]]; then - mpv {{ emacsconf_caption_dir }}/assets/intros/$SLUG.webm +if [[ -f "$FILE" ]]; then + echo "Playing $FILE" + mpv "$FILE" else - firefox {{ emacsconf_caption_dir }}/assets/in-between/$SLUG.png + echo "Opening backup image {{ emacsconf_caption_dir }}/assets/in-between/$SLUG.png" + firefox {{ emacsconf_caption_dir }}/assets/in-between/$SLUG.png fi diff --git a/roles/obs/templates/music b/roles/obs/templates/music index 29c5ddb..a5a6a31 100755 --- a/roles/obs/templates/music +++ b/roles/obs/templates/music @@ -1,5 +1,5 @@ if screen -list | grep -q background; then echo "Already running in screen, attach with screen -x background" else - screen -dmS background /bin/bash -c "mpv ~/stream/background.wav --loop=yes" + screen -dmS background /bin/bash -c "mpv {{ background_music_dir }}/* --shuffle --loop-playlist=inf --volume={{ background_music_volume }}" fi diff --git a/roles/obs/templates/overlay b/roles/obs/templates/overlay index 1361884..0836a59 100755 --- a/roles/obs/templates/overlay +++ b/roles/obs/templates/overlay @@ -3,20 +3,20 @@ SLUG=$(echo "$1" | perl -ne 'if (/emacsconf-[0-9]*-(.*?)--/) { print $1; } else { print; }') -if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png ]]; then +if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png ]]; then echo "Found other overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png ~/other.png -elif [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ]]; then + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png ~/other.png +elif [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ]]; then echo "Found video overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ~/other.png + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ~/other.png else - echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png, please override ~/other.png manually" - cp {{ emacsconf_caption_dir }}/assets/overlays/blank-other.svg.png ~/other.png + echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png, please override ~/other.png manually" + cp {{ emacsconf_caption_dir }}/assets/overlays/blank-other.png ~/other.png fi -if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ]]; then +if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ]]; then echo "Found video overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ~/video.png + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ~/video.png else - echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png, override ~/video.png manually" - cp {{ emacsconf_caption_dir }}/assets/overlays/blank-video.svg.png ~/video.png + echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png, override ~/video.png manually" + cp {{ emacsconf_caption_dir }}/assets/overlays/blank-video.png ~/video.png fi diff --git a/roles/obs/templates/play b/roles/obs/templates/play index 20fd24c..69632d4 100755 --- a/roles/obs/templates/play +++ b/roles/obs/templates/play @@ -2,19 +2,60 @@ # Play intro if recorded, then play files # {{ ansible_managed }} -# Kill the background music if playing -if screen -list | grep -q background; then - screen -S background -X quit -fi +shopt -s nullglob + +# play-with-intro $SLUG +YEAR="{{ emacsconf_year }}" +BASE_DIR="{{ emacsconf_caption_dir }}" +CACHE_DIR="{{ emacsconf_caption_dir }}/cache" +FIREFOX_NAME=firefox-esr +SLUG=$1 +PREFIX=$(get-file-prefix $SLUG) +PREFER_LIVE=$(jq -r ".talks[] | select(.slug == \"$SLUG\") | .\"prefer-live\" " < "$BASE_DIR/talks.json") +/usr/local/bin/reset-state # Update the overlay -FILE=$1 -if [[ ! -f $FILE ]]; then - LIST=({{ emacsconf_caption_dir }}/assets/stream/emacsconf-{{ emacsconf_year }}-$FILE*--main.webm) +overlay $SLUG + +if [ "$PREFER_LIVE" = "t" ]; then + /usr/local/bin/bbb $SLUG + exit 0 +fi + +# Play the video if it exists. If it doesn't exist, switch to the BBB room and stop processing. +if [ "x$TEST_MODE" = "x" ]; then + LIST=($CACHE_DIR/{{ emacsconf_id }}-{{ emacsconf_year }}-$SLUG*--main.webm) +else + LIST=($BASE_DIR/assets/test/{{ emacsconf_id }}-{{ emacsconf_year }}-$SLUG*--main.webm) +fi + + +FILE="${LIST[0]}" +# No file in the stream directory; check for original files in the stream directory, then check the cache +NOSUB="" +if [[ ! -f "$FILE" ]]; then + # Is there a cache file or an original file? + LIST=($CACHE_DIR/{{ emacsconf_id }}-{{ emacsconf_year }}-$SLUG*--main.webm) FILE="${LIST[0]}" - BY_SLUG=1 + if [[ ! -f $FILE ]]; then + LIST=($CACHE_DIR/{{ emacsconf_id }}-{{ emacsconf_year }}-$SLUG*--original.*) + FILE="${LIST[0]}" + fi + echo "Candidate: " $FILE + SUBS=($CACHE_DIR/{{ emacsconf_id }}-{{ emacsconf_year }}-$SLUG*--main.vtt) + if [[ -f "${SUBS[0]}" ]]; then + if ! cat ${SUBS[0]} | head -1 | grep -q captioned ; then + echo "Skipping subtitles because they're not edited" + NOSUB="--sub-visibility=no" + else + NOSUB="--sub-visibility=yes" + fi + fi +fi + +if [[ -f "$FILE" ]]; then + screen -mS talk /bin/bash -c "mpv $NOSUB $FILE" +else + /usr/local/bin/bbb $SLUG + exit 0 fi -shift -SLUG=$(echo "$FILE" | perl -ne 'if (/emacsconf-[0-9]*-(.*?)--/) { print $1; } else { print; }') -overlay $SLUG -mpv $FILE $* & diff --git a/roles/obs/templates/play-with-intro b/roles/obs/templates/play-with-intro index 1b1b9a5..0390f8f 100755 --- a/roles/obs/templates/play-with-intro +++ b/roles/obs/templates/play-with-intro @@ -2,23 +2,21 @@ # Play intro if recorded, then play files # {{ ansible_managed }} -# Kill the background music if playing -if screen -list | grep -q background; then - screen -S background -X quit -fi +# play-with-intro $SLUG +YEAR="{{ emacsconf_year }}" +BASE_DIR="{{ emacsconf_caption_dir }}" +CACHE_DIR="{{ emacsconf_caption_dir }}/cache" +FIREFOX_NAME=firefox-esr +SLUG=$1 +PREFIX=$(get-file-prefix $SLUG) + +/usr/local/bin/reset-state # Update the overlay -FILE=$1 -if [[ ! -f $FILE ]]; then - LIST=({{ emacsconf_caption_dir }}/assets/stream/emacsconf-{{ emacsconf_year }}-$FILE*.webm) - FILE="${LIST[0]}" - BY_SLUG=1 -fi -shift -SLUG=$(echo "$FILE" | perl -ne 'if (/emacsconf-[0-9]*-(.*?)--/) { print $1; } else { print; }') -overlay $SLUG -# Play the video -if [[ -f {{ emacsconf_caption_dir }}/assets/intros/$SLUG.webm ]]; then - intro $SLUG -fi -mpv $FILE $* & +/usr/local/bin/overlay $SLUG + +# Play the intro if it exists. If it doesn't exist, switch to the intro slide and stop processing. + +/usr/local/bin/intro $SLUG + +/usr/local/bin/play $SLUG diff --git a/roles/obs/templates/set-overlay b/roles/obs/templates/set-overlay index 0b6712b..8a408bf 100644 --- a/roles/obs/templates/set-overlay +++ b/roles/obs/templates/set-overlay @@ -6,20 +6,20 @@ if [[ ! -f $FILE ]]; then fi shift SLUG=$(echo "$FILE" | perl -ne 'if (/^emacsconf-[0-9]*-(.*?)--/) { print $1; } else { print; }') -if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png ]]; then +if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png ]]; then echo "Found other overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png ~/other.png -elif [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ]]; then + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png ~/other.png +elif [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ]]; then echo "Found video overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ~/other.png + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ~/other.png else - echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.svg.png, please override ~/other.png manually" - cp {{ emacsconf_caption_dir }}/assets/overlays/blank-other.svg.png ~/other.png + echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-other.png, please override ~/other.png manually" + cp {{ emacsconf_caption_dir }}/assets/overlays/blank-other.png ~/other.png fi -if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ]]; then +if [[ -f {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ]]; then echo "Found video overlay for $SLUG, copying" - cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png ~/video.png + cp {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png ~/video.png else - echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.svg.png, override ~/video.png manually" - cp {{ emacsconf_caption_dir }}/assets/overlays/blank-video.svg.png ~/video.png + echo "Could not find {{ emacsconf_caption_dir }}/assets/overlays/$SLUG-video.png, override ~/video.png manually" + cp {{ emacsconf_caption_dir }}/assets/overlays/blank-video.png ~/video.png fi diff --git a/roles/obs/templates/xstartup-track b/roles/obs/templates/xstartup-track index e09c081..f2f6716 100755 --- a/roles/obs/templates/xstartup-track +++ b/roles/obs/templates/xstartup-track @@ -5,7 +5,7 @@ export PATH="/usr/local/bin:/usr/bin:/bin:{{ emacsconf_home }}/bin" xrdb $HOME/.Xresources pulseaudio --start pacmd set-default-sink qa -firefox file:///data/emacsconf/{{ emacsconf_year }}/index-{{ item.id }}.html & +firefox file:///data/emacsconf/admin/{{ emacsconf_year }}/index-{{ item.id }}.html & $HOME/bin/track-obs & mumble mumble://{{ emacsconf_id }}-{{ item.id }}@{{ mumble_server }}/{{ item.mumble_channel }} & termit & diff --git a/roles/pad-proxy/tasks/main.yml b/roles/pad-proxy/tasks/main.yml index 8de72df..1e4518a 100644 --- a/roles/pad-proxy/tasks/main.yml +++ b/roles/pad-proxy/tasks/main.yml @@ -2,6 +2,10 @@ - name: Set up Nginx as root become: true block: + - name: Remove apache + apt: + name: apache2 + state: absent - name: Install Nginx apt: name: nginx diff --git a/roles/pad-proxy/templates/etherpad.nginx.conf b/roles/pad-proxy/templates/etherpad.nginx.conf index 144185f..b657b98 100644 --- a/roles/pad-proxy/templates/etherpad.nginx.conf +++ b/roles/pad-proxy/templates/etherpad.nginx.conf @@ -20,7 +20,7 @@ server { include {{ etherpad_tls }}; {% endif %} access_log /var/log/nginx/{{ etherpad_server_name }}.access.log; - location ~ ^/(locales/|locales.json|admin/|static/|pluginfw/|javascripts/|socket.io/|ep/|minified/|api/|ro/|error/|jserror/|favicon.ico|robots.txt) { + location ~ ^/(locales/|locales.json|admin/|static/|pluginfw/|javascripts/|socket.io/|ep/|minified/|api/|ro/|error/|jserror/|favicon.ico|robots.txt|.*\.js) { proxy_buffering off; proxy_pass http://etherpad_upstream; } @@ -49,7 +49,7 @@ server { proxy_pass http://etherpad_upstream; } location / { - rewrite ^/?$ https://pad.emacsconf.org/{{ emacsconf_year }} redirect; + rewrite ^/?$ {{ etherpad_url }}/{{ emacsconf_year }} redirect; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; diff --git a/roles/pad/defaults/main.yml b/roles/pad/defaults/main.yml index 7538489..7562410 100644 --- a/roles/pad/defaults/main.yml +++ b/roles/pad/defaults/main.yml @@ -1,10 +1,2 @@ --- # defaults file for pad -etherpad_path: /home/etherpad/etherpad -etherpad_database_name: emacsconf_pad -etherpad_database_user: etherpad -etherpad_user: etherpad -etherpad_group: etherpad -etherpad_base: emacsconf -etherpad_year: "{{ emacsconf_year }}" -etherpad_load_test: false diff --git a/roles/pad/tasks/main.yml b/roles/pad/tasks/main.yml index 9851d66..6e32ff6 100644 --- a/roles/pad/tasks/main.yml +++ b/roles/pad/tasks/main.yml @@ -1,10 +1,11 @@ --- # tasks file for pad - name: Set up packages as root + become: true block: - name: Add GPG apt: - update_cache: yes + update_cache: "{{ update_cache }}" name: - gpg - sudo @@ -16,15 +17,15 @@ - name: Add nodesource repository ansible.builtin.apt_repository: repo: deb https://deb.nodesource.com/node_20.x nodistro main - update_cache: yes + update_cache: "{{ update_cache }}" - name: Install packages apt: - update_cache: yes + update_cache: "{{ update_cache }}" name: - git - systemd - sudo - - nodejs + - nodejs=20.19.5-1nodesource1 - mariadb-server - mariadb-client state: present @@ -34,65 +35,106 @@ home: /home/etherpad shell: /bin/bash state: present -- include: mariadb.yml - become: true -- name: Set up etherpad as the etherpad user - tags: etherpad-src - become: true - become_user: "{{ etherpad_user }}" - block: - - name: Install etherpad - git: - repo: https://github.com/ether/etherpad-lite.git - dest: "{{ etherpad_path }}" - depth: 1 - - name: Configure etherpad - template: - src: templates/settings.json - dest: "{{ etherpad_path }}/settings.json" - - name: Install dependencies - shell: cd {{ etherpad_path }}; . src/bin/functions.sh; src/bin/installDeps.sh - # - name: Install etherpad plugins - # npm: - # name: ep_scrolltoanchor - # path: "{{ etherpad_path }}" - - name: Change ownership - file: - dest: /home/etherpad/etherpad - owner: "{{ etherpad_user }}" - group: "{{ etherpad_group }}" - recurse: true - - name: Set etherpad API key - copy: - content: "{{ etherpad_api_key }}" - dest: "{{ etherpad_path }}/APIKEY.txt" - owner: "{{ etherpad_user }}" - mode: "0600" -- name: Install init.d configuration - tags: system - become: true - template: - src: etherpad.init.d - dest: /etc/init.d/etherpad - owner: root - group: root - mode: 0755 - when: use_initd -- name: Install systemd configuration - tags: system - become: true - template: - src: etherpad.service - dest: /etc/systemd/system/etherpad.service - owner: root - group: root - mode: 0755 - when: not use_initd + - name: Install pnpm + community.general.npm: + name: pnpm + global: true + - name: Install Etherpad + include_role: + name: systemli.etherpad + - name: Install MySQL packages + apt: + name: + - mariadb-server + - mariadb-client + - python3-mysqldb + state: present + - name: Ensure mysql is configured to bind only to localhost + ini_file: + dest: /etc/mysql/my.cnf + section: mysqld + option: "bind-address" + value: "127.0.0.1" + - name: Ensure anonymous users are not in the database + mysql_user: + name: "" + host: "{{ item }}" + state: absent + with_items: + - localhost + - 127.0.0.1 + - ::1 + - "%" + - name: Start MariaDB + service: + name: mysql + state: restarted + enabled: yes + runlevel: + - 3 + - 5 + +# - import_tasks: mariadb.yml +# become: true +# - name: Set up etherpad as the etherpad user +# tags: etherpad-src +# become: true +# become_user: "{{ etherpad_user }}" +# block: +# - name: Install etherpad +# git: +# repo: https://github.com/ether/etherpad-lite.git +# dest: "{{ etherpad_path }}" +# version: v1.9.7 +# depth: 1 +# - name: Configure etherpad +# template: +# src: templates/settings.json +# dest: "{{ etherpad_path }}/settings.json" +# - name: Install dependencies +# shell: cd {{ etherpad_path }}; . src/bin/functions.sh; src/bin/installDeps.sh +# # - name: Install etherpad plugins +# # npm: +# # name: ep_scrolltoanchor +# # path: "{{ etherpad_path }}" +# - name: Change ownership +# file: +# dest: /home/etherpad/etherpad +# owner: "{{ etherpad_user }}" +# group: "{{ etherpad_group }}" +# recurse: true +# - name: Set etherpad API key +# copy: +# content: "{{ etherpad_api_key }}" +# dest: "{{ etherpad_path }}/APIKEY.txt" +# owner: "{{ etherpad_user }}" +# mode: "0600" +# - name: Install init.d configuration +# tags: system +# become: true +# template: +# src: etherpad.init.d +# dest: /etc/init.d/etherpad +# owner: root +# group: root +# mode: 0755 +# when: use_initd +# - name: Install systemd configuration +# tags: system +# become: true +# template: +# src: etherpad.service +# dest: /etc/systemd/system/etherpad.service +# owner: root +# group: root +# mode: 0755 +# when: not use_initd - name: Restart Etherpad become: true service: - name: etherpad + name: etherpad-lite state: restarted + enabled: yes - tags: create-pads include_vars: file: talks.json @@ -104,7 +146,7 @@ - name: Wait for OK tags: create-pads uri: - url: "https://{{ etherpad_server_name }}/api/1/createPad?apikey={{ etherpad_api_key }}&padID={{etherpad_year}}" + url: "{{ etherpad_url }}/api/1/createPad?apikey={{ etherpad_api_key }}&padID={{etherpad_year}}" register: _result until: _result.status == 200 retries: 720 @@ -115,5 +157,5 @@ - name: Create pads tags: create-pads uri: - url: "https://{{ etherpad_server_name }}/api/1/createPad?apikey={{ etherpad_api_key }}&padID={{etherpad_year}}-{{ item.slug }}" + url: "{{ etherpad_url }}/api/1/createPad?apikey={{ etherpad_api_key }}&padID={{etherpad_year}}-{{ item.slug }}" loop: "{{ talks | json_query('talks[*]') }}" diff --git a/roles/pad/tasks/mariadb.yml b/roles/pad/tasks/mariadb.yml deleted file mode 100644 index ec81430..0000000 --- a/roles/pad/tasks/mariadb.yml +++ /dev/null @@ -1,42 +0,0 @@ ---- -- name: Install MySQL packages - apt: - name: - - mariadb-server - - mariadb-client - - python3-mysqldb -- name: Ensure mysql is configured to bind only to localhost - ini_file: - dest: /etc/mysql/my.cnf - section: mysqld - option: "bind-address" - value: "127.0.0.1" -- name: Start MariaDB - service: - name: mysql - state: restarted - enabled: yes - runlevel: - - 3 - - 5 -- name: Ensure anonymous users are not in the database - mysql_user: - name: "" - host: "{{ item }}" - state: absent - with_items: - - localhost - - 127.0.0.1 - - ::1 - - "%" -- name: Ensure emacsconf-pad database exists - mysql_db: - name: "{{ etherpad_database_name }}" - collation: utf8mb4_general_ci - state: present -- name: Grant permissions to user - mysql_user: - name: "{{ etherpad_database_user }}" - state: present - priv: "{{ etherpad_database_name }}.*:ALL" - password: "{{ etherpad_database_password }}" diff --git a/roles/pad/templates/etherpad.init.d b/roles/pad/templates/etherpad.init.d deleted file mode 100755 index 299d5fb..0000000 --- a/roles/pad/templates/etherpad.init.d +++ /dev/null @@ -1,80 +0,0 @@ -#!/bin/sh -# {{ ansible_managed }} - -### BEGIN INIT INFO -# Provides: etherpad-lite -# Required-Start: $local_fs $remote_fs $network $syslog -# Required-Stop: $local_fs $remote_fs $network $syslog -# Default-Start: 2 3 4 5 -# Default-Stop: 0 1 6 -# Short-Description: starts etherpad lite -# Description: starts etherpad lite using start-stop-daemon -### END INIT INFO - -PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" -LOGFILE="{{ etherpad_path }}/etherpad-lite.log" -EPLITE_DIR="{{ etherpad_path }}" -EPLITE_BIN="src/bin/safeRun.sh" -USER="{{ etherpad_user }}" -GROUP="{{ etherpad_group }}" -DESC="Etherpad Lite" -NAME="etherpad" - -set -e - -. /lib/lsb/init-functions - -start() { - echo "Starting $DESC... " - - start-stop-daemon --start --chuid "$USER:$GROUP" -d $EPLITE_DIR --background --make-pidfile --pidfile /var/run/$NAME.pid --exec $EPLITE_DIR/$EPLITE_BIN -- $LOGFILE || true - echo "done" -} - -#We need this function to ensure the whole process tree will be killed -killtree() { - local _pid=$1 - local _sig=${2-TERM} - for _child in $(ps -o pid --no-headers --ppid ${_pid}); do - killtree ${_child} ${_sig} - done - kill -${_sig} ${_pid} -} - -stop() { - echo "Stopping $DESC... " - if test -f /var/run/$NAME.pid; then - while test -d /proc/$(cat /var/run/$NAME.pid); do - killtree $(cat /var/run/$NAME.pid) 15 - sleep 0.5 - done - rm /var/run/$NAME.pid - fi - echo "done" -} - -status() { - status_of_proc -p /var/run/$NAME.pid "" "etherpad-lite" && exit 0 || exit $? -} - -case "$1" in - start) - start - ;; - stop) - stop - ;; - restart) - stop - start - ;; - status) - status - ;; - *) - echo "Usage: $NAME {start|stop|restart|status}" >&2 - exit 1 - ;; -esac - -exit 0 diff --git a/roles/pad/templates/etherpad.service b/roles/pad/templates/etherpad.service deleted file mode 100644 index bcc45d1..0000000 --- a/roles/pad/templates/etherpad.service +++ /dev/null @@ -1,17 +0,0 @@ -# {{ ansible_managed }} - -[Unit] -Description=Etherpad -After=syslog.target network.target - -[Service] -Type=simple -User={{ etherpad_user }} -Group={{ etherpad_group }} -WorkingDirectory={{ etherpad_path }} -ExecStart=node {{ etherpad_path }}/node_modules/ep_etherpad-lite/node/server.js -Environment=NODE_ENV=production -Restart=always - -[Install] -WantedBy=multi-user.target diff --git a/roles/pad/templates/settings.json b/roles/pad/templates/settings.json deleted file mode 100644 index 977a562..0000000 --- a/roles/pad/templates/settings.json +++ /dev/null @@ -1,634 +0,0 @@ -/* - * {{ ansible_managed }} - * - * This file must be valid JSON. But comments are allowed - * - * Please edit settings.json, not settings.json.template - * - * Please note that starting from Etherpad 1.6.0 you can store DB credentials in - * a separate file (credentials.json). - * - * - * ENVIRONMENT VARIABLE SUBSTITUTION - * ================================= - * - * All the configuration values can be read from environment variables using the - * syntax "${ENV_VAR}" or "${ENV_VAR:default_value}". - * - * This is useful, for example, when running in a Docker container. - * - * DETAILED RULES: - * - If the environment variable is set to the string "true" or "false", the - * value becomes Boolean true or false. - * - If the environment variable is set to the string "null", the value - * becomes null. - * - If the environment variable is set to the string "undefined", the setting - * is removed entirely, except when used as the member of an array in which - * case it becomes null. - * - If the environment variable is set to a string representation of a finite - * number, the string is converted to that number. - * - If the environment variable is set to any other string, including the - * empty string, the value is that string. - * - If the environment variable is unset and a default value is provided, the - * value is as if the environment variable was set to the provided default: - * - "${UNSET_VAR:}" becomes the empty string. - * - "${UNSET_VAR:foo}" becomes the string "foo". - * - "${UNSET_VAR:true}" and "${UNSET_VAR:false}" become true and false. - * - "${UNSET_VAR:null}" becomes null. - * - "${UNSET_VAR:undefined}" causes the setting to be removed (or be set - * to null, if used as a member of an array). - * - If the environment variable is unset and no default value is provided, - * the value becomes null. THIS BEHAVIOR MAY CHANGE IN A FUTURE VERSION OF - * ETHERPAD; if you want the default value to be null, you should explicitly - * specify "null" as the default value. - * - * EXAMPLE: - * "port": "${PORT:9001}" - * "minify": "${MINIFY}" - * "skinName": "${SKIN_NAME:colibris}" - * - * Would read the configuration values for those items from the environment - * variables PORT, MINIFY and SKIN_NAME. - * - * If PORT and SKIN_NAME variables were not defined, the default values 9001 and - * "colibris" would be used. - * The configuration value "minify", on the other hand, does not have a - * designated default value. Thus, if the environment variable MINIFY were - * undefined, "minify" would be null. - * - * REMARKS: - * 1) please note that variable substitution always needs to be quoted. - * - * "port": 9001, <-- Literal values. When not using - * "minify": false substitution, only strings must be - * "skinName": "colibris" quoted. Booleans and numbers must not. - * - * "port": "${PORT:9001}" <-- CORRECT: if you want to use a variable - * "minify": "${MINIFY:true}" substitution, put quotes around its name, - * "skinName": "${SKIN_NAME}" even if the required value is a number or - * a boolean. - * Etherpad will take care of rewriting it - * to the proper type if necessary. - * - * "port": ${PORT:9001} <-- ERROR: this is not valid json. Quotes - * "minify": ${MINIFY} around variable names are missing. - * "skinName": ${SKIN_NAME} - * - * 2) Beware of undefined variables and default values: nulls and empty strings - * are different! - * - * This is particularly important for user's passwords (see the relevant - * section): - * - * "password": "${PASSW}" // if PASSW is not defined would result in password === null - * "password": "${PASSW:}" // if PASSW is not defined would result in password === '' - * - * If you want to use an empty value (null) as default value for a variable, - * simply do not set it, without putting any colons: "${ABIWORD}". - * - * 3) if you want to use newlines in the default value of a string parameter, - * use "\n" as usual. - * - * "defaultPadText" : "${DEFAULT_PAD_TEXT}Line 1\nLine 2" - */ -{ - /* - * Name your instance! - */ - "title": "EmacsConf Etherpad", - - /* - * Pathname of the favicon you want to use. If null, the skin's favicon is - * used if one is provided by the skin, otherwise the default Etherpad favicon - * is used. If this is a relative path it is interpreted as relative to the - * Etherpad root directory. - */ - "favicon": null, - - /* - * Skin name. - * - * Its value has to be an existing directory under src/static/skins. - * You can write your own, or use one of the included ones: - * - * - "no-skin": an empty skin (default). This yields the unmodified, - * traditional Etherpad theme. - * - "colibris": the new experimental skin (since Etherpad 1.8), candidate to - * become the default in Etherpad 2.0 - */ - "skinName": "colibris", - - /* - * Skin Variants - * - * Use the UI skin variants builder at /p/test#skinvariantsbuilder - * - * For the colibris skin only, you can choose how to render the three main - * containers: - * - toolbar (top menu with icons) - * - editor (containing the text of the pad) - * - background (area outside of editor, mostly visible when using page style) - * - * For each of the 3 containers you can choose 4 color combinations: - * super-light, light, dark, super-dark. - * - * For example, to make the toolbar dark, you will include "dark-toolbar" into - * skinVariants. - * - * You can provide multiple skin variants separated by spaces. Default - * skinVariant is "super-light-toolbar super-light-editor light-background". - * - * For the editor container, you can also make it full width by adding - * "full-width-editor" variant (by default editor is rendered as a page, with - * a max-width of 900px). - */ - "skinVariants": "super-light-toolbar super-light-editor light-background", - - /* - * IP and port which Etherpad should bind at. - * - * Binding to a Unix socket is also supported: just use an empty string for - * the ip, and put the full path to the socket in the port parameter. - * - * EXAMPLE USING UNIX SOCKET: - * "ip": "", // <-- has to be an empty string - * "port" : "/somepath/etherpad.socket", // <-- path to a Unix socket - */ - "ip": "127.0.0.1", - "port": 9001, - - /* - * Option to hide/show the settings.json in admin page. - * - * Default option is set to true - */ - "showSettingsInAdminPage": true, - - /* - * Node native SSL support - * - * This is disabled by default. - * Make sure to have the minimum and correct file access permissions set so - * that the Etherpad server can access them - */ - - /* - "ssl" : { - "key" : "/path-to-your/epl-server.key", - "cert" : "/path-to-your/epl-server.crt", - "ca": ["/path-to-your/epl-intermediate-cert1.crt", "/path-to-your/epl-intermediate-cert2.crt"] - }, - */ - - /* - * The type of the database. - * - * You can choose between many DB drivers, for example: dirty, postgres, - * sqlite, mysql. - * - * You shouldn't use "dirty" for for anything else than testing or - * development. - * - * - * Database specific settings are dependent on dbType, and go in dbSettings. - * Remember that since Etherpad 1.6.0 you can also store this information in - * credentials.json. - * - * For a complete list of the supported drivers, please refer to: - * https://www.npmjs.com/package/ueberdb2 - */ - - /* - * An Example of MySQL Configuration (commented out). - * - * See: https://github.com/ether/etherpad-lite/wiki/How-to-use-Etherpad-Lite-with-MySQL - */ - - "dbType" : "mysql", - "dbSettings" : { - "user": "{{ etherpad_database_user }}", - "host": "localhost", - "port": 3306, - "password": "{{ etherpad_database_password }}", - "database": "{{ etherpad_database_name }}", - "charset": "utf8mb4" - }, - - /* - * The default text of a pad - */ - "defaultPadText" : "Conference info, how to watch/participate: https://emacsconf.org/{{ emacsconf_year }}/\nGuidelines for conduct: https://emacsconf.org/conduct/\nSee end of file for license (CC Attribution-ShareAlike 4.0 + GPLv3 or later)\n----------------------------------------------------------------\nQuestions and discussion go here:\n- \n- \n- \n- \n- \n- \n- \n\n----------------------------------------------------------------\nExcept where otherwise noted, the material on the EmacsConf pad are dual-licensed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International Public License; and the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) an later version. Copies of these two licenses are included in the EmacsConf wiki repository, in the COPYING.GPL and COPYING.CC-BY-SA files (https://emacsconf.org/COPYING/)\n\nBy contributing to this pad, you agree to make your contributions available under the above licenses. You are also promising that you are the author of your changes, or that you copied them from a work in the public domain or a work released under a free license that is compatible with the above two licenses. DO NOT SUBMIT COPYRIGHTED WORK WITHOUT PERMISSION.", - - /* - * Default Pad behavior. - * - * Change them if you want to override. - */ - "padOptions": { - "noColors": false, - "showControls": true, - "showChat": false, - "showLineNumbers": true, - "useMonospaceFont": false, - "userName": null, - "userColor": null, - "rtl": false, - "alwaysShowChat": false, - "chatAndUsers": false, - "lang": null - }, - - /* - * Pad Shortcut Keys - */ - "padShortcutEnabled" : { - "altF9": true, /* focus on the File Menu and/or editbar */ - "altC": true, /* focus on the Chat window */ - "cmdShift2": true, /* shows a gritter popup showing a line author */ - "delete": true, - "return": true, - "esc": true, /* in mozilla versions 14-19 avoid reconnecting pad */ - "cmdS": true, /* save a revision */ - "tab": true, /* indent */ - "cmdZ": true, /* undo/redo */ - "cmdY": true, /* redo */ - "cmdI": true, /* italic */ - "cmdB": true, /* bold */ - "cmdU": true, /* underline */ - "cmd5": true, /* strike through */ - "cmdShiftL": true, /* unordered list */ - "cmdShiftN": true, /* ordered list */ - "cmdShift1": true, /* ordered list */ - "cmdShiftC": true, /* clear authorship */ - "cmdH": true, /* backspace */ - "ctrlHome": true, /* scroll to top of pad */ - "pageUp": true, - "pageDown": true - }, - - /* - * Should we suppress errors from being visible in the default Pad Text? - */ - "suppressErrorsInPadText": false, - - /* - * If this option is enabled, a user must have a session to access pads. - * This effectively allows only group pads to be accessed. - */ - "requireSession": false, - - /* - * Users may edit pads but not create new ones. - * - * Pad creation is only via the API. - * This applies both to group pads and regular pads. - */ - "editOnly": {{ (not etherpad_load_test) | bool | lower }}, - - /* - * If true, all css & js will be minified before sending to the client. - * - * This will improve the loading performance massively, but makes it difficult - * to debug the javascript/css - */ - "minify": true, - - /* - * How long may clients use served javascript code (in seconds)? - * - * Not setting this may cause problems during deployment. - * Set to 0 to disable caching. - */ - "maxAge": 21600, // 60 * 60 * 6 = 6 hours - - /* - * Absolute path to the Abiword executable. - * - * Abiword is needed to get advanced import/export features of pads. Setting - * it to null disables Abiword and will only allow plain text and HTML - * import/exports. - */ - "abiword": null, - - /* - * This is the absolute path to the soffice executable. - * - * LibreOffice can be used in lieu of Abiword to export pads. - * Setting it to null disables LibreOffice exporting. - */ - "soffice": null, - - /* - * Path to the Tidy executable. - * - * Tidy is used to improve the quality of exported pads. - * Setting it to null disables Tidy. - */ - "tidyHtml": null, - - /* - * Allow import of file types other than the supported ones: - * txt, doc, docx, rtf, odt, html & htm - */ - "allowUnknownFileEnds": false, - - /* - * This setting is used if you require authentication of all users. - * - * Note: "/admin" always requires authentication. - */ - "requireAuthentication": false, - - /* - * Require authorization by a module, or a user with is_admin set, see below. - */ - "requireAuthorization": false, - - /* - * When you use NGINX or another proxy/load-balancer set this to true. - * - * This is especially necessary when the reverse proxy performs SSL - * termination, otherwise the cookies will not have the "secure" flag. - * - * The other effect will be that the logs will contain the real client's IP, - * instead of the reverse proxy's IP. - */ - "trustProxy": true, - - /* - * Settings controlling the session cookie issued by Etherpad. - */ - "cookie": { - /* - * Value of the SameSite cookie property. "Lax" is recommended unless - * Etherpad will be embedded in an iframe from another site, in which case - * this must be set to "None". Note: "None" will not work (the browser will - * not send the cookie to Etherpad) unless https is used to access Etherpad - * (either directly or via a reverse proxy with "trustProxy" set to true). - * - * "Strict" is not recommended because it has few security benefits but - * significant usability drawbacks vs. "Lax". See - * https://stackoverflow.com/q/41841880 for discussion. - */ - "sameSite": "Lax", - - /* - * How long (in milliseconds) after navigating away from Etherpad before the - * user is required to log in again. (The express_sid cookie is set to - * expire at time now + sessionLifetime when first created, and its - * expiration time is periodically refreshed to a new now + sessionLifetime - * value.) If requireAuthentication is false then this value does not really - * matter. - * - * The "best" value depends on your users' usage patterns and the amount of - * convenience you desire. A long lifetime is more convenient (users won't - * have to log back in as often) but has some drawbacks: - * - It increases the amount of state kept in the database. - * - It might weaken security somewhat: The cookie expiration is refreshed - * indefinitely without consulting authentication or authorization - * hooks, so once a user has accessed a pad, the user can continue to - * use the pad until the user leaves for longer than sessionLifetime. - * - * Session lifetime can be set to infinity (not recommended) by setting this - * to null or 0. Note that if the session does not expire, most browsers - * will delete the cookie when the browser exits, but a session record is - * kept in the database forever. - */ - "sessionLifetime": 864000000, // = 10d * 24h/d * 60m/h * 60s/m * 1000ms/s - - /* - * How long (in milliseconds) before the expiration time of an active user's - * session is refreshed (to now + sessionLifetime). This setting affects the - * following: - * - How often a new session expiration time will be written to the - * database. - * - How often each user's browser will ping the Etherpad server to - * refresh the expiration time of the session cookie. - * - * High values reduce the load on the database and the load from browsers, - * but can shorten the effective session lifetime if Etherpad is restarted - * or the user navigates away. - * - * Automatic session refreshes can be disabled (not recommended) by setting - * this to null. - */ - "sessionRefreshInterval": 86400000 // = 1d * 24h/d * 60m/h * 60s/m * 1000ms/s - }, - - /* - * Privacy: disable IP logging - */ - "disableIPlogging": true, - - /* - * Time (in seconds) to automatically reconnect pad when a "Force reconnect" - * message is shown to user. - * - * Set to 0 to disable automatic reconnection. - */ - "automaticReconnectionTimeout": 0, - - /* - * By default, when caret is moved out of viewport, it scrolls the minimum - * height needed to make this line visible. - */ - "scrollWhenFocusLineIsOutOfViewport": { - - /* - * Percentage of viewport height to be additionally scrolled. - * - * E.g.: use "percentage.editionAboveViewport": 0.5, to place caret line in - * the middle of viewport, when user edits a line above of the - * viewport - * - * Set to 0 to disable extra scrolling - */ - "percentage": { - "editionAboveViewport": 0, - "editionBelowViewport": 0 - }, - - /* - * Time (in milliseconds) used to animate the scroll transition. - * Set to 0 to disable animation - */ - "duration": 0, - - /* - * Flag to control if it should scroll when user places the caret in the - * last line of the viewport - */ - "scrollWhenCaretIsInTheLastLineOfViewport": false, - - /* - * Percentage of viewport height to be additionally scrolled when user - * presses arrow up in the line of the top of the viewport. - * - * Set to 0 to let the scroll to be handled as default by Etherpad - */ - "percentageToScrollWhenUserPressesArrowUp": 0 - }, - - /* - * User accounts. These accounts are used by: - * - default HTTP basic authentication if no plugin handles authentication - * - some but not all authentication plugins - * - some but not all authorization plugins - * - * User properties: - * - password: The user's password. Some authentication plugins will ignore - * this. - * - is_admin: true gives access to /admin. Defaults to false. If you do not - * uncomment this, /admin will not be available! - * - readOnly: If true, this user will not be able to create new pads or - * modify existing pads. Defaults to false. - * - canCreate: If this is true and readOnly is false, this user can create - * new pads. Defaults to true. - * - * Authentication and authorization plugins may define additional properties. - * - * WARNING: passwords should not be stored in plaintext in this file. - * If you want to mitigate this, please install ep_hash_auth and - * follow the section "secure your installation" in README.md - */ - - /* - "users": { - "admin": { - // 1) "password" can be replaced with "hash" if you install ep_hash_auth - // 2) please note that if password is null, the user will not be created - "password": "changeme1", - "is_admin": true - }, - "user": { - // 1) "password" can be replaced with "hash" if you install ep_hash_auth - // 2) please note that if password is null, the user will not be created - "password": "changeme1", - "is_admin": false - } - }, - */ - - /* - * Restrict socket.io transport methods - */ - "socketTransportProtocols" : ["xhr-polling", "jsonp-polling", "htmlfile"], - - "socketIo": { - /* - * Maximum permitted client message size (in bytes). All messages from - * clients that are larger than this will be rejected. Large values make it - * possible to paste large amounts of text, and plugins may require a larger - * value to work properly, but increasing the value increases susceptibility - * to denial of service attacks (malicious clients can exhaust memory). - */ - "maxHttpBufferSize": 10000 - }, - - /* - * Allow Load Testing tools to hit the Etherpad Instance. - * - * WARNING: this will disable security on the instance. - */ - "loadTest": {{ etherpad_load_test | bool | lower }}, - - /** - * Disable dump of objects preventing a clean exit - */ - "dumpOnUncleanExit": false, - - /* - * Disable indentation on new line when previous line ends with some special - * chars (':', '[', '(', '{') - */ - - /* - "indentationOnNewLine": false, - */ - - /* - * From Etherpad 1.8.3 onwards, import and export of pads is always rate - * limited. - * - * The default is to allow at most 10 requests per IP in a 90 seconds window. - * After that the import/export request is rejected. - * - * See https://github.com/nfriedly/express-rate-limit for more options - */ - "importExportRateLimiting": { - // duration of the rate limit window (milliseconds) - "windowMs": 90000, - - // maximum number of requests per IP to allow during the rate limit window - "max": 10 - }, - - /* - * From Etherpad 1.8.3 onwards, the maximum allowed size for a single imported - * file is always bounded. - * - * File size is specified in bytes. Default is 50 MB. - */ - "importMaxFileSize": 52428800, // 50 * 1024 * 1024 - - /* - * From Etherpad 1.8.5 onwards, when Etherpad is in production mode commits from individual users are rate limited - * - * The default is to allow at most 10 changes per IP in a 1 second window. - * After that the change is rejected. - * - * See https://github.com/animir/node-rate-limiter-flexible/wiki/Overall-example#websocket-single-connection-prevent-flooding for more options - */ - "commitRateLimiting": { - // duration of the rate limit window (seconds) - "duration": 1, - - // maximum number of changes per IP to allow during the rate limit window - "points": 10 - }, - - /* - * Toolbar buttons configuration. - * - * Uncomment to customize. - */ - - /* - "toolbar": { - "left": [ - ["bold", "italic", "underline", "strikethrough"], - ["orderedlist", "unorderedlist", "indent", "outdent"], - ["undo", "redo"], - ["clearauthorship"] - ], - "right": [ - ["importexport", "timeslider", "savedrevision"], - ["settings", "embed"], - ["showusers"] - ], - "timeslider": [ - ["timeslider_export", "timeslider_returnToPad"] - ] - }, - */ - - /* - * Expose Etherpad version in the web interface and in the Server http header. - * - * Do not enable on production machines. - */ - "exposeVersion": false, - - /* - * The log level we are using. - * - * Valid values: DEBUG, INFO, WARN, ERROR - */ - "loglevel": "INFO", - - /* Override any strings found in locale directories */ - "customLocaleStrings": {}, - - /* Disable Admin UI tests */ - "enableAdminUITests": false -} diff --git a/roles/pad/vars/main.yml b/roles/pad/vars/main.yml index 981efa9..dfe2bfc 100644 --- a/roles/pad/vars/main.yml +++ b/roles/pad/vars/main.yml @@ -1,2 +1,20 @@ ---- -# vars file for pad +etherpad_user: etherpad +etherpad_group: etherpad +etherpad_base: emacsconf +etherpad_year: "{{ emacsconf_year }}" +etherpad_load_test: "false" +etherpad_title: "{{emacsconf_name}} {{emacsconf_year}}" +etherpad_enable_metrics: true +etherpad_default_text: "Conference info, how to watch/participate: https://emacsconf.org/{{ emacsconf_year }}/\\nGuidelines for conduct: https://emacsconf.org/conduct/\\nSee end of file for license (CC Attribution-ShareAlike 4.0 + GPLv3 or later)\\n----------------------------------------------------------------\\nQuestions and discussion go here:\\n- \\n- \\n- \\n- \\n- \\n- \\n- \\n\\n----------------------------------------------------------------\\nExcept where otherwise noted, the material on the EmacsConf pad are dual-licensed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International Public License; and the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) an later version. Copies of these two licenses are included in the EmacsConf wiki repository, in the COPYING.GPL and COPYING.CC-BY-SA files (https://emacsconf.org/COPYING/)\\n\\nBy contributing to this pad, you agree to make your contributions available under the above licenses. You are also promising that you are the author of your changes, or that you copied them from a work in the public domain or a work released under a free license that is compatible with the above two licenses. DO NOT SUBMIT COPYRIGHTED WORK WITHOUT PERMISSION." +etherpad_pad_options_no_colors: "false" +etherpad_pad_options_show_controls: "true" +etherpad_pad_options_show_chat: "false" +etherpad_abiword: null +etherpad_soffice: "null" +etherpad_edit_only: "true" +etherpad_mysql_database_user: etherpad +etherpad_mysql_database_name: etherpad +etherpad_db_type: mysql +etherpad_mysql_database_host: localhost +etherpad_mysql_database_password: "{{ etherpad_database_password }}" +etherpad_min_node_version: "18.18.2" diff --git a/roles/prerec/tasks/main.yml b/roles/prerec/tasks/main.yml index 8144e28..a0a3cac 100644 --- a/roles/prerec/tasks/main.yml +++ b/roles/prerec/tasks/main.yml @@ -33,7 +33,7 @@ owner: "{{ emacsconf_user }}" group: "{{ emacsconf_group }}" - name: Recreate encoding script and backup old one - tags: process-prerec + tags: process-prerec, prerec-scripts template: src: "reencode.sh" dest: "{{ emacsconf_caption_dir }}/scripts/reencode.sh" @@ -41,8 +41,12 @@ group: "{{ emacsconf_group }}" backup: yes mode: 0775 +- name: Get UIDs + tags: process-prerec, prerec-scripts + getent: + database: passwd - name: Copy scripts for processing - tags: process-prerec + tags: process-prerec, prerec-scripts template: src: "{{ item }}" dest: "/usr/local/bin/{{ item }}" @@ -59,13 +63,16 @@ - reencode.sh - run-aeneas.sh - rename-original.sh + - copy-original.sh - mux-subs.sh - verify-main.sh - remux.sh + - get-file-prefix + - reencode-in-screen.sh - name: Copy Makefile + tags: process-prerec, prerec-scripts, prerec-make template: src: Makefile dest: "{{ emacsconf_caption_dir }}/cache/Makefile" owner: "{{ emacsconf_user }}" group: "{{ emacsconf_group }}" - diff --git a/roles/prerec/templates/Makefile b/roles/prerec/templates/Makefile index 94aa412..f58c5d6 100644 --- a/roles/prerec/templates/Makefile +++ b/roles/prerec/templates/Makefile @@ -1,16 +1,118 @@ # {{ ansible_managed }} -PRERECS_FINAL := $(wildcard {{ emacsconf_id }}-*--final.webm) -PRERECS_MAIN := $(patsubst %--final.webm, %--main.webm, $(PRERECS_FINAL)) +# To recreate this file, use ansible-playbook -i inventory.yml prod-playbook.yml --tags prerec-make + +VIDEO_EXTS = mp4 mkv webm mov mpv +source_patterns = $(foreach ext,$(VIDEO_EXTS),$(1)--original.$(ext)) + +PRERECS_ORIGINAL := $(wildcard emacsconf-*--original.*) +PREFIXES := $(shell for f in $(PRERECS_ORIGINAL); do echo "$${f%--original.*}"; done) +PRERECS_REENCODED := $(addsuffix --reencoded.webm, $(PREFIXES)) +PRERECS_OPUS := $(addsuffix --reencoded.opus, $(PREFIXES)) +PRERECS_MAIN := $(addsuffix --main.webm, $(PREFIXES)) +PRERECS_CAPTIONS := $(addsuffix --reencoded.vtt, $(PREFIXES)) +PRERECS_FINAL := $(wildcard emacsconf-*--final.webm) +LINODE_BBB_ID := 67329098 .PHONY: all -all: $(PRERECS_MAIN) $(PRERECS_FINAL) -emacsconf-%--main.webm: {{ emacsconf_id }}-%--final.webm +all: reencoded opus main + +reencoded: $(PRERECS_REENCODED) + +opus: $(PRERECS_OPUS) + +captions: $(PRERECS_CAPTIONS) + +main: $(PRERECS_MAIN) + +emacsconf-%--reencoded.webm: SOURCES = $(call source_patterns, emacsconf-$*) +emacsconf-%--reencoded.webm: + $(eval SOURCE := $(lastword $(sort $(wildcard $(SOURCES))))) + @if [ -z "$(SOURCE)" ]; then \ + echo "No source file found for $@"; \ + echo "Tried: $(SOURCES)"; \ + exit 1; \ + fi + @echo "Using source: $(SOURCE)" + reencode-in-screen.sh "$(SOURCE)" + +emacsconf-%--reencoded.opus: emacsconf-%--reencoded.webm + ffmpeg -y -i "$<" -c:a copy "$@" + +emacsconf-%--answers.opus: emacsconf-%--answers.webm + ffmpeg -y -i "$<" -c:a copy "$@" + +# emacsconf-%--normalized.opus: emacsconf-%--reencoded.opus +# ffmpeg-normalize "$<" -ofmt opus -c:a libopus -o "$@" + +emacsconf-%--main.webm: emacsconf-%--reencoded.webm emacsconf-%--normalized.opus emacsconf-%--main.vtt + ffmpeg -i emacsconf-$*--reencoded.webm -i emacsconf-$*--normalized.opus -i emacsconf-$*--main.vtt \ + -map 0:v -map 1:a -c:v copy -c:a copy \ + -map 2 -c:s webvtt -y \ + $@ + +emacsconf-%--main.webm: emacsconf-%--reencoded.webm emacsconf-%--normalized.opus + ffmpeg -i emacsconf-$*--reencoded.webm -i emacsconf-$*--normalized.opus \ + -map 0:v -map 1:a -c:v copy -c:a copy \ + -y $@ + +emacsconf-%--main.webm: emacsconf-%--reencoded.webm cp "$<" "$@" -emacsconf-%--final.webm: {{ emacsconf_id }}-%--main.vtt +emacsconf-%--final.webm: emacsconf-%--main.vtt mux-subs.sh "$@" "$<" +emacsconf-%--main.opus: emacsconf-%--main.webm + ffmpeg -y -i "$<" -c:a copy "$@" + +emacsconf-%--reencoded.vtt: emacsconf-%--reencoded.opus + whisperx --model large-v3 --align_model WAV2VEC2_ASR_LARGE_LV60K_960H --compute_type int8 --print_progress True --max_line_width 50 --segment_resolution chunk --max_line_count 1 --language en --initial_prompt "Transcribe this talk about Emacs. It may mention Emacs keywords such as Org Mode, Org Roam, Magit, gptel, or chatgpt-shell, or tech keywords such as LLMs. Format function names and keyboard shortcut sequences according to Emacs conventions using Markdown syntax. For example: control h becomes \`C-h\`." "$<" + +emacsconf-%--backstage--silences.csv: emacsconf-%--reencoded.opus + ffmpeg -i "$<" -af silencedetect=noise=-30dB:d=0.5 -f null - 2>&1 | awk '/silence_start/ {start=$$NF} /silence_end/ {print start "," (start + $$NF)}' > "$@" + +show-files: + @echo "Original $(words $(PRERECS_ORIGINAL)):" + @echo "$(PRERECS_ORIGINAL)" + @echo "Prefixes $(words $(PREFIXES)):" + @echo "$(PREFIXES)" + @echo "Reencoded $(words $(PRERECS_REENCODED)):" + @echo "$(PRERECS_REENCODED)" + @echo "Opus $(words $(PRERECS_OPUS)):" + @echo "$(PRERECS_OPUS)" + +bbb-testing: + @echo "Resizing BBB node to 8GB 4 core for testing" + linode-cli linodes resize $(LINODE_BBB_ID) --type g6-standard-4 --allow_auto_disk_resize false + sleep 2m + @echo "Booting up" + linode-cli linodes boot $(LINODE_BBB_ID) + ssh root@bbb.emacsverse.org "cd ~/greenlight-v3; docker compose restart" + +bbb-dormant: + @echo "Shutting down" + ssh root@bbb.emacsverse.org "/usr/sbin/shutdown -h now &" || true + sleep 30 + @echo "Powering off BBB node" + linode-cli linodes shutdown $(LINODE_BBB_ID) + sleep 30 + @echo "Resizing BBB node to nanode, dormant" + linode-cli linodes resize $(LINODE_BBB_ID) --type g6-nanode-1 --allow_auto_disk_resize false + +bbb-prod: + @echo "Resizing BBB node to production size" + linode-cli linodes resize $(LINODE_BBB_ID) --type g6-dedicated-8 --allow_auto_disk_resize true + sleep 2m + @echo "Booting up" + linode-cli linodes boot $(LINODE_BBB_ID) + +live-dormant: + @echo "Resizing live0 node to nanode, dormant" + linode-cli linodes resize 17921960 --type g6-nanode-1 --allow_auto_disk_resize false + sleep 120 + linode-cli linodes boot 17921960 + + rsync: rsync -avzue ssh {{ emacsconf_caption_dir }}/cache/ orga@media.emacsconf.org:/var/www/media.emacsconf.org/{{ emacsconf_year }}/backstage/ diff --git a/roles/prerec/templates/copy-original.sh b/roles/prerec/templates/copy-original.sh new file mode 100755 index 0000000..854bdb8 --- /dev/null +++ b/roles/prerec/templates/copy-original.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# {{ ansible_managed }} +# Usage: rename-original.sh $slug $file [$extra] +SLUG=$1 +FILE=$2 +EXTRA="" +if [ -z ${3-unset} ]; then + EXTRA="" +elif [ -n "$3" ]; then + EXTRA="$3" +elif echo "$FILE" | grep -e '\(webm\|mp4\|mov\)'; then + EXTRA="--original" +fi +filename=$(basename -- "$FILE") +extension="${filename##*.}" +filename="${filename%.*}" +FILE_PREFIX=$(get-file-prefix $SLUG) +if echo "$FILE" | grep -q \\. ; then + cp "$FILE" $FILE_PREFIX$EXTRA.$extension + echo $FILE_PREFIX$EXTRA.$extension +else + cp "$FILE" $FILE_PREFIX$EXTRA + echo $FILE_PREFIX$EXTRA +fi diff --git a/roles/prerec/templates/get-file-prefix b/roles/prerec/templates/get-file-prefix new file mode 100755 index 0000000..0240b99 --- /dev/null +++ b/roles/prerec/templates/get-file-prefix @@ -0,0 +1,10 @@ +#!/bin/bash +# {{ ansible_managed }} +# Usage: get-file-prefix $slug [talks.json] +# You can also set the TALKS_JSON environment variable + +SLUG=$1 +if [ -z "$TALKS_JSON" ]; then + TALKS_JSON=${2:-~orga/current/talks.json} +fi +jq -r '.talks[] | select(.slug=="'$SLUG'")["file-prefix"]' < $TALKS_JSON diff --git a/roles/prerec/templates/process-prerec.sh b/roles/prerec/templates/process-prerec.sh index 795753d..4347576 100755 --- a/roles/prerec/templates/process-prerec.sh +++ b/roles/prerec/templates/process-prerec.sh @@ -5,14 +5,11 @@ ORIGINAL=$1 REENCODED=$(echo "$ORIGINAL" | perl -pe 's/^(emacsconf-[0-9]*-.*?--.*?--.*?--).*/$1reencoded.webm/') SLUG=$(echo "$ORIGINAL" | perl -ne '/^emacsconf-[0-9]*-(.*?)--/ && print $1') MAIN=$(echo "$ORIGINAL" | perl -pe 's/^(emacsconf-[0-9]*-.*?--.*?--.*?--).*/$1main.webm/') -SCREEN=reencode-$SLUG -if ! ( screen -ls | grep -q $SLUG ); then - screen -dmS $SCREEN -fi +VTT=$(echo "$ORIGINAL" | perl -pe 's/^(emacsconf-[0-9]*-.*?--.*?--.*?--).*/$1reencoded.vtt/') # ( cd /data/emacsconf/cache; ./update-cache ) -# /data/emacsconf/{{ emacsconf_year }}/scripts/talk $SLUG "WAITING_FOR_PREREC" "PROCESSING" +# /data/emacsconf/admin/{{ emacsconf_year }}/scripts/talk $SLUG "WAITING_FOR_PREREC" "PROCESSING" if [[ ! -f "$REENCODED" ]]; then - screen -S $SCREEN -X screen -t reencode-$SLUG /bin/bash -c "/data/emacsconf/{{ emacsconf_year }}/scripts/reencode.sh \"$ORIGINAL\" \"$REENCODED\" && /data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh $REENCODED $MAIN && /data/emacsconf/{{ emacsconf_year }}/scripts/thumbnail.sh \"$MAIN\" && /data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh $(echo "$MAIN" | sed s/webm$/png/) exec /bin/bash" & + screen -dmS reencode-$SLUG /bin/bash -c "reencode.sh \"$ORIGINAL\" \"$REENCODED\" && make $MAIN && upload.sh $REENCODED $MAIN && thumbnail.sh \"$MAIN\" && upload.sh $(echo "$MAIN" | sed s/webm$/png/); echo $(date -Iminutes) $SLUG reencoded >> ~/emacsconf.log && make $VTT; echo $(date -Iminutes) $SLUG captioned >> ~/emacsconf.log; exec /bin/bash" & +else + screen -dmS captions-$SLUG /bin/bash -c "make $VTT; echo $(date -Iminutes) $SLUG captioned >> ~/emacsconf.log; exec /bin/bash" fi -screen -S $SCREEN -X screen -t captions-$SLUG /bin/bash -c "/data/emacsconf/{{ emacsconf_year }}/scripts/process-captions.py $(dirname $ORIGINAL); exec /bin/bash" -screen -x $SCREEN diff --git a/roles/prerec/templates/reencode-in-screen.sh b/roles/prerec/templates/reencode-in-screen.sh new file mode 100755 index 0000000..5c12203 --- /dev/null +++ b/roles/prerec/templates/reencode-in-screen.sh @@ -0,0 +1,16 @@ +#!/bin/bash +ORIGINAL=$1 +BASE="${ORIGINAL%--original.*}" +REENCODED="${BASE}--reencoded.webm" +SLUG=$(echo "$ORIGINAL" | perl -ne '/^emacsconf-[0-9]*-(.*?)--/ && print $1') +LOCK=".lock-$SLUG" + +if [ ! -f "$REENCODED" ]; then + if [ -f "$LOCK" ]; then + echo "$LOCK already exists, waiting for it" + else + touch "$LOCK" + screen -dmS reencode-$SLUG /bin/bash -c "reencode.sh \"$ORIGINAL\" \"$REENCODED\" && thumbnail.sh \"$MAIN\" && rm \"$LOCK\"" + echo "Processing $REENCODED in reencode-$SLUG" + fi +fi diff --git a/roles/prerec/templates/reencode.sh b/roles/prerec/templates/reencode.sh index b0bdfc7..ac5309a 100755 --- a/roles/prerec/templates/reencode.sh +++ b/roles/prerec/templates/reencode.sh @@ -10,6 +10,9 @@ print_only=false limit_resolution={{ res_y }} limit_fps={{ fps }} +output_final="${2:-$(echo $input | sed 's/--original.*/--reencoded.webm/')}" +output_temp="partial--${output_final}" # <-- New temporary name + while getopts :q:c:t:s OPT; do case $OPT in q|+q) @@ -33,7 +36,7 @@ shift `expr $OPTIND - 1` OPTIND=1 input="$1" -output="$2" +output="${2:-$(echo $input | sed 's/--original.*/--reencoded.webm/')}" command="$(cat<<EOF ffmpeg -y -i "$input" $time_limit \ @@ -47,7 +50,7 @@ ffmpeg -y -i "$input" $time_limit \ fps='$limit_fps'" \ -c:v libvpx-vp9 -b:v 0 -crf $q -c:a libopus \ -row-mt 1 -tile-columns 2 -tile-rows 2 -cpu-used $cpu \ - -pass 2 -threads $cpu -- "$output" + -pass 2 -threads $cpu -- "$output_temp" EOF )" @@ -55,4 +58,5 @@ if [ $print_only == true ]; then echo "$command" else eval "$command" + mv "$output_temp" "$output_final" fi diff --git a/roles/prerec/templates/remux.sh b/roles/prerec/templates/remux.sh index c378133..2d2320b 100755 --- a/roles/prerec/templates/remux.sh +++ b/roles/prerec/templates/remux.sh @@ -1,20 +1,29 @@ #!/usr/bin/env bash # {{ ansible_managed }} # Mix in the normalized audio -# Usage: remux.sh $input_video +# Usage: remux.sh $input_video_or_slug with_suffix() { - echo "$input_video" | sed "s/--\(reencoded\|original\).webm\$/--$1/" + echo "$input_video" | sed "s/--\(main\|reencoded\|original\).webm\$/--$1/" } input_video="$1" +if [ ! -f $input_video ]; then + # treat it as a slug + input_video=$(get-file-prefix $1)--reencoded.webm +fi + input_audio="$(with_suffix "normalized.opus")" output_video="$(with_suffix "final.webm")" main_video="$(with_suffix "main.webm")" main_subs="$(with_suffix "main.vtt")" +if cat $main_subs | head -1 | grep captioned; then + $subs = "-i $main_subs" +fi + command="$(cat<<EOF -ffmpeg -i "$input_video" -i "$input_audio" -c:v copy -c:a copy -map 0:v:0 -map 1:a:0 "$output_video" && +ffmpeg -y -i "$input_video" -i "$input_audio" $subs -c:v copy -c:a copy -map 0:v:0 -map 1:a:0 "$output_video" && cp "$output_video" "$main_video" && if [[ -f "$main_subs" ]]; then touch -m "$main_subs"; fi EOF @@ -23,14 +32,18 @@ EOF printf "input: %s\ncomputed output: %s\nrelated main: %s\n\n" "$input_video" "$output_video" "$main_video" printf "Produced incantation:\n%s\n\n" "$command" -while true; -do - read -r -p "Run it? y/n " -n 1 -r response - if [[ $response =~ ^([yY])$ ]]; then - eval "$command" - exit 0 - else - printf "\nExiting\n" - exit 3 - fi -done +if [ -z "$CONFIRMED" ]; then + eval "$command" +else + while true; + do + read -r -p "Run it? y/n " -n 1 -r response + if [[ $response =~ ^([yY])$ ]]; then + eval "$command" + exit 0 + else + printf "\nExiting\n" + exit 3 + fi + done +fi diff --git a/roles/prerec/templates/rename-original.sh b/roles/prerec/templates/rename-original.sh index 0499ae8..ed085ed 100755 --- a/roles/prerec/templates/rename-original.sh +++ b/roles/prerec/templates/rename-original.sh @@ -8,7 +8,7 @@ EXTRA="" if [ -z ${3-unset} ]; then EXTRA="" elif [ -n "$3" ]; then - EXTRA="--$3" + EXTRA="$3" elif echo "$FILE" | grep -e '\(webm\|mp4\|mov\)'; then EXTRA="--original" fi @@ -16,8 +16,13 @@ filename=$(basename -- "$FILE") extension="${filename##*.}" filename="${filename%.*}" FILE_PREFIX=$(jq -r '.talks[] | select(.slug=="'$SLUG'")["file-prefix"]' < $TALKS_JSON) -mv "$FILE" $FILE_PREFIX$EXTRA.$extension -echo $FILE_PREFIX$EXTRA.$extension +if echo "$FILE" | grep -q \\. ; then + mv "$FILE" $FILE_PREFIX$EXTRA.$extension + echo $FILE_PREFIX$EXTRA.$extension +else + mv "$FILE" $FILE_PREFIX$EXTRA + echo $FILE_PREFIX$EXTRA +fi # Copy to original if needed if [ -f $FILE_PREFIX--original.webm ] && [ ! -f $FILE_PREFIX--main.$extension ]; then cp $FILE_PREFIX--original.$extension $FILE_PREFIX--main.webm diff --git a/roles/prerec/templates/run-aeneas.sh b/roles/prerec/templates/run-aeneas.sh index 8246c85..a9e27f4 100755 --- a/roles/prerec/templates/run-aeneas.sh +++ b/roles/prerec/templates/run-aeneas.sh @@ -11,4 +11,4 @@ if [ ! -f $BASE--whisper.vtt ]; then fi python3 -m aeneas.tools.execute_task $AUDIO *.txt "task_language=eng|os_task_file_format=vtt|is_text_type=plain" ${BASE}--aeneas.vtt cp ${BASE}--aeneas.vtt ${BASE}--main.vtt -/data/emacsconf/{{ emacsconf_year }}/scripts/upload.sh ${BASE}--main.vtt +/data/emacsconf/admin/{{ emacsconf_year }}/scripts/upload.sh ${BASE}--main.vtt diff --git a/roles/prerec/templates/talk b/roles/prerec/templates/talk index 0531cda..fa2e5f0 100755 --- a/roles/prerec/templates/talk +++ b/roles/prerec/templates/talk @@ -6,6 +6,8 @@ SLUG="$1" FROM_STATUS="$2" TO_STATUS="$3" +XDG_RUNTIME_DIR=/run/user/{{ getent_passwd[emacsconf_user].1 }} + if [ "x$TO_STATUS" == "x" ]; then FROM_STATUS=. TO_STATUS="$2" @@ -14,7 +16,7 @@ cd {{ emacsconf_private_dir }} #echo "Pulling conf.org..." #git pull echo "Updating status..." -emacsclient --eval "(emacsconf-with-todo-hooks (emacsconf-update-talk-status \"$SLUG\" \"$FROM_STATUS\" \"$TO_STATUS\"))" -a emacs +XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR emacsclient --eval "(emacsconf-with-todo-hooks (emacsconf-update-talk-status \"$SLUG\" \"$FROM_STATUS\" \"$TO_STATUS\"))" -a emacs #echo "Committing and pushing in the background" #git commit -m "Update task status for $SLUG from $FROM_STATUS to $TO_STATUS" conf.org #git push & diff --git a/roles/prerec/templates/upload.sh b/roles/prerec/templates/upload.sh index f3dc9c5..03cc66a 100755 --- a/roles/prerec/templates/upload.sh +++ b/roles/prerec/templates/upload.sh @@ -2,5 +2,5 @@ # {{ ansible_managed }} scp $* orga@media.emacsconf.org:~/backstage -/data/emacsconf/{{ emacsconf_year }}/scripts/publish-backstage-index.sh -rsync -avze ssh orga@media.emacsconf.org:~/backstage/ /data/emacsconf/cache/ +# /usr/local/bin/publish-backstage-index.sh +rsync -avze ssh orga@media.emacsconf.org:~/backstage/ /data/emacsconf/shared/{{ emacsconf_year }}/cache/ diff --git a/roles/publish/defaults/main.yml b/roles/publish/defaults/main.yml index 52ec596..19a12d8 100644 --- a/roles/publish/defaults/main.yml +++ b/roles/publish/defaults/main.yml @@ -1,6 +1,6 @@ emacsconf_org_file: "{{ emacsconf_private_dir }}/conf.org" emacsconf_publishing_phase: schedule -emacs_version: 28.2 +emacs_version: 29.1 emacs_build_parent: /usr/src/emacs emacs_build_dir: "{{ emacs_build_parent }}/emacs-{{ emacs_version }}" emacsconf_wiki_branch: master diff --git a/roles/publish/tasks/emacs.yml b/roles/publish/tasks/emacs.yml index 3944876..d961878 100644 --- a/roles/publish/tasks/emacs.yml +++ b/roles/publish/tasks/emacs.yml @@ -8,6 +8,8 @@ - pkg-config - libjansson-dev - libgnutls28-dev + - libtiff-dev + - libtiff5-dev - automake - texinfo state: present diff --git a/roles/publish/tasks/main.yml b/roles/publish/tasks/main.yml index 0724bdd..797ac7f 100644 --- a/roles/publish/tasks/main.yml +++ b/roles/publish/tasks/main.yml @@ -25,8 +25,8 @@ register: emacs - name: Set up Emacs become: yes - include: emacs.yml - when: not emacs.stat.exists + import_tasks: emacs.yml + # when: not emacs.stat.exists - name: Configure git template: src: git-config @@ -68,7 +68,7 @@ dest: "~{{ emacsconf_user }}/subed" - name: Check out wiki repository ansible.builtin.git: - repo: anon@git.emacsconf.org:emacsconf-wiki + repo: "{{ emacsconf_publishing_source }}" dest: "{{ emacsconf_edit_wiki_dir }}" ssh_opts: "-i /home/{{ emacsconf_user }}/.ssh/id_rsa_anon_git_emacsconf" register: wiki_clone diff --git a/roles/publish/templates/emacsconf-config.el b/roles/publish/templates/emacsconf-config.el index 0c7a80b..767de85 100644 --- a/roles/publish/templates/emacsconf-config.el +++ b/roles/publish/templates/emacsconf-config.el @@ -15,22 +15,24 @@ (setq emacsconf-ansible-directory "{{ emacsconf_ansible_directory }}") {% endif %} (setq emacsconf-pad-api-key "{{ etherpad_api_key }}") -(setq emacsconf-publishing-phase '{{ emacsconf_publishing_phase }}) +(setq emacsconf-publishing-phase 'conference) (setq emacsconf-backstage-password "{{ emacsconf_backstage_password }}") +(setq emacsconf-public-media-directory "/ssh:orga@media.emacsconf.org:/var/www/media.emacsconf.org/{{ emacsconf_year }}") (setq emacsconf-backstage-dir "/ssh:orga@media.emacsconf.org:/var/www/media.emacsconf.org/{{ emacsconf_year }}/backstage") (setq emacsconf-upload-dir "/ssh:orga@media.emacsconf.org:/srv/upload") {% if ansible_host == "res.emacsconf.org" %} -(setq emacsconf-res-dir "/data/{{ emacsconf_id }}/{{ emacsconf_year}}") -(setq emacsconf-cache-dir "/data/{{ emacsconf_id }}/cache") +(setq emacsconf-res-dir "/data/{{ emacsconf_id }}/shared/{{ emacsconf_year}}") +(setq emacsconf-cache-dir "/data/{{ emacsconf_id }}/shared/{{ emacsconf_year}}/cache") (setq emacsconf-stream-host "localhost") (setq emacsconf-stream-overlay-dir "{{ emacsconf_caption_dir }}/assets/overlays") (setq emacsconf-stream-asset-dir "{{ emacsconf_caption_dir }}/assets") {% else %} -(setq emacsconf-res-dir (format "/ssh:orga@res.emacsconf.org:/data/emacsconf/%s" emacsconf-year)) +(setq emacsconf-res-dir (format "/ssh:orga@res.emacsconf.org:/data/emacsconf/admin/%s" emacsconf-year)) {% endif %} +(setq emacsconf-publish-autocommit-wiki t) (add-to-list 'load-path "~/compile-media") (add-to-list 'load-path "~/subed/subed") (require 'compile-media) diff --git a/roles/stream/tasks/main.yml b/roles/stream/tasks/main.yml index d821d5e..eaf2fc8 100644 --- a/roles/stream/tasks/main.yml +++ b/roles/stream/tasks/main.yml @@ -8,40 +8,55 @@ package: name: icecast2 state: present +- name: Install package for setting htpasswd + package: + name: python3-passlib + when: protect_stream_with_password +- name: Create htpasswd entry + htpasswd: + create: yes + name: "{{ emacsconf_backstage_user }}" + password: "{{ emacsconf_backstage_password }}" + path: /etc/nginx/{{ emacsconf_id }}-htpasswd + when: protect_stream_with_password - name: Set up config become: true template: src: icecast.xml - dest: /etc/icecast2/icecast-emacsconf.xml + dest: /etc/icecast2/icecast-{{ emacsconf_id }}.xml - name: Create restream dir file: path: "{{ icecast_restream_dir }}" owner: "{{ icecast_user }}" state: directory - name: Set up track-specific things - include: track.yml + include_tasks: track.yml loop: "{{ emacsconf_tracks }}" - name: Set up init file become: true template: src: icecast-emacsconf.init.d - dest: /etc/init.d/emacsconf + dest: /etc/init.d/{{ emacsconf_id }} mode: 0755 - name: Set up nginx config tags: stream-nginx become: true template: src: emacsconf.nginx.conf - dest: /etc/nginx/emacsconf.nginx.conf + dest: /etc/nginx/{{ emacsconf_id }}.nginx.conf mode: 0644 - name: Include emacsconf in nginx config become: true lineinfile: - line: include /etc/nginx/emacsconf.nginx.conf; - regexp: '^\s*include /etc/nginx/emacsconf.nginx.conf;' + line: include /etc/nginx/{{ emacsconf_id }}.nginx.conf; + regexp: '^\s*include /etc/nginx/{{ emacsconf_id }}.nginx.conf;' insertafter: '.*tls/live0.conf.*' backup: yes - path: /etc/nginx/sites-available/live0.emacsconf.org + path: /etc/nginx/sites-available/live0.{{ emacsconf_id }}.org +- name: Reload nginx config + service: + name: nginx + state: reloaded - name: Enable icecast become: true service: @@ -57,7 +72,7 @@ register: port_check ignore_errors: yes - name: Try to restart icecast if not started - service: name=emacsconf state=started enabled=yes + service: name={{ emacsconf_id }} state=started enabled=yes when: port_check.failed == true - name: Set up restream scripts tags: restream @@ -67,7 +82,7 @@ owner: orga mode: 0755 loop: "{{ restreaming_platforms | subelements('streams') | list }}" - no_log: True + # no_log: True - name: Set up restream scripts tags: restream template: @@ -76,7 +91,7 @@ owner: orga mode: 0755 loop: "{{ restreaming_platforms | subelements('streams') | list }}" - no_log: True + # no_log: True - name: Copy fallback files copy: src: sorry.webm diff --git a/roles/stream/templates/emacsconf.nginx.conf b/roles/stream/templates/emacsconf.nginx.conf index ac84ef5..cf5e2b8 100644 --- a/roles/stream/templates/emacsconf.nginx.conf +++ b/roles/stream/templates/emacsconf.nginx.conf @@ -11,6 +11,10 @@ location ~ ^/((gen|dev)(-480p|-fallback)?.webm)$ { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Real-IP $remote_addr; + {% if protect_stream_with_password %} + auth_basic {{ emacsconf_name }}; + auth_basic_user_file /etc/nginx/{{ emacsconf_id }}-htpasswd; + {% endif %} } location ~ ^/emacsconf/(.*)$ { @@ -26,5 +30,8 @@ location ~ ^/emacsconf/(.*)$ { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Real-IP $remote_addr; + {% if protect_stream_with_password %} + auth_basic {{ emacsconf_name }}; + auth_basic_user_file /etc/nginx/{{ emacsconf_id }}-htpasswd; + {% endif %} } -
\ No newline at end of file diff --git a/roles/stream/templates/icecast.xml b/roles/stream/templates/icecast.xml index 06830d3..b01bc48 100644 --- a/roles/stream/templates/icecast.xml +++ b/roles/stream/templates/icecast.xml @@ -192,7 +192,6 @@ <mount-name>/{{ track.id }}.webm</mount-name> <username>{{ icecast_emacsconf_user }}</username> <password>{{ icecast_emacsconf_password }}</password> - {% if not test_mode %}<dump-file>/data/{{ emacsconf_id }}-{{ emacsconf_year }}-{{ track.id }}_%Y-%m-%d_%H-%M-%S.webm</dump-file>{% endif %} <stream-name>{{ emacsconf_name }} {{ emacsconf_year }} - {{ track.name }} track</stream-name> <stream-description>The livestream for the {{ track.name }} track of {{ emacsconf_name }} {{ emacsconf_year }}</stream-description> <stream-url>{{ track.watch }}</stream-url> @@ -202,6 +201,7 @@ <on-disconnect>/usr/local/bin/{{ emacsconf_id }}-lowres-{{ track.id }}-on-disconnect</on-disconnect> <fallback-mount>/{{ track.id }}-sorry.webm</fallback-mount> <fallback-override>1</fallback-override> + {% if not test_mode %}<dump-file>/data/{{ emacsconf_id }}-{{ emacsconf_year }}-{{ track.id }}_%Y-%m-%d_%H-%M-%S.webm</dump-file>{% endif %} </mount> <mount type="normal"> <mount-name>/{{ track.id }}-host.webm</mount-name> diff --git a/roles/stream/templates/lowres.sh b/roles/stream/templates/lowres.sh index 502fcb3..13d6a81 100755 --- a/roles/stream/templates/lowres.sh +++ b/roles/stream/templates/lowres.sh @@ -1,6 +1,6 @@ #!/bin/bash # {{ ansible_managed }} sleep 10 -for i in 1 2 3 4 5; do +while true; do ffmpeg -loglevel 24 -f webm -reconnect_at_eof 1 -reconnect_streamed 1 -re -i "http://localhost:{{ icecast_port }}/{{ item.id }}.webm" -vf scale="{{ icecast_lowres_scale }}" -f webm -c:a copy -b:v 500k -maxrate 1M -bufsize 1M -content_type video/webm -c:v libvpx "icecast://{{ icecast_emacsconf_user }}:{{ icecast_emacsconf_password }}@localhost:{{ icecast_port }}/{{ item.id }}-480p.webm" >> {{ icecast_restream_dir }}/{{ emacsconf_id }}-lowres-{{ item.id }}.log || sleep 5 done diff --git a/roles/upload/tasks/main.yml b/roles/upload/tasks/main.yml index 3e7e615..b63e3f1 100644 --- a/roles/upload/tasks/main.yml +++ b/roles/upload/tasks/main.yml @@ -21,6 +21,14 @@ name: - nodejs state: present +- name: Add to dehydrated.conf + become: true + lineinfile: + line: "{{ upload_server_name }}" + path: /etc/dehydrated/domains.txt +- name: Create or renew cert + command: "dehydrated --cron" + become: true - name: Create upload user become: true user: @@ -99,6 +107,19 @@ owner: "{{ upload_user }}" group: "{{ upload_group }}" recurse: true +- name: Create backup dir to use when the conference is done + file: + path: "{{ upload_done_dir }}" + owner: "{{ upload_user }}" + group: "{{ upload_group }}" + state: directory +- name: Create backup file to use when the conference is done + template: + src: index.html + owner: "{{ upload_user }}" + group: "{{ upload_group }}" + mode: 0755 + dest: "{{ upload_done_dir }}/index.html" - name: Install systemd configuration tags: system become: true @@ -109,9 +130,34 @@ group: root mode: 0755 when: not use_initd +- name: Create main configuration if needed + template: + src: nginx-site-config + dest: /etc/nginx/sites-available/{{ upload_server_name }} +- name: Make sure main configuration is enabled + file: + src: /etc/nginx/sites-available/{{ upload_server_name }} + dest: /etc/nginx/sites-enabled/{{ upload_server_name }} + owner: "{{ emacsconf_user }}" + group: "{{ emacsconf_group }}" + force: no + state: link +- name: Reload configuration + become: true + service: + name: nginx + state: reloaded - name: Restart Upload become: true + when: upload_enabled service: name: upload state: restarted - enabled: yes + enabled: true +- name: Stop upload + become: true + when: not upload_enabled + service: + name: upload + state: stopped + enabled: false diff --git a/roles/upload/templates/index.html b/roles/upload/templates/index.html new file mode 100644 index 0000000..e5d8e6e --- /dev/null +++ b/roles/upload/templates/index.html @@ -0,0 +1,7 @@ +<html> + <head> + </head> + <body> + EmacsConf is done for now, so we've turned off the file upload service. Let us know at <a href="mailto:emacsconf-org-private@gnu.org">emacsconf-org-private@gnu.org</a> if you need it back to upload something! + </body> +</html> diff --git a/roles/upload/templates/nginx-site-config b/roles/upload/templates/nginx-site-config new file mode 100644 index 0000000..48842d1 --- /dev/null +++ b/roles/upload/templates/nginx-site-config @@ -0,0 +1,46 @@ +upstream upload_emacsconf { + server 127.0.0.1:3000; +} + +server { + listen 80; + listen [::]:80; + server_name {{ upload_server_name }}; + + include snippets/well-known-acme-challenge.conf; + + location / { + return 302 https://$server_name$request_uri; + } +} + +server { + listen 443 ssl http2; + listen [::]:443 ssl http2; + server_name {{ upload_server_name }}; + + ssl_certificate /var/local/dehydrated/certs/{{ upload_server_name }}/fullchain.pem; + ssl_certificate_key /var/local/dehydrated/certs/{{ upload_server_name }}/privkey.pem; + ssl_trusted_certificate /var/local/dehydrated/certs/{{ upload_server_name }}/fullchain.pem; + include ssl_params.local; + include snippets/well-known-acme-challenge.conf; +{% if upload_enabled %} + location @upload_emacsconf { + proxy_pass http://upload_emacsconf; + proxy_http_version 1.1; + proxy_buffering off; + proxy_request_buffering off; # needs nginx version >= 1.7.11 + proxy_set_header Host $http_host; + } + location / { + #try_files $uri $uri/ @upload_emacsconf; + proxy_pass http://upload_emacsconf; + proxy_http_version 1.1; + proxy_buffering off; + proxy_request_buffering off; # needs nginx version >= 1.7.11 + proxy_set_header Host $http_host; + } + {% else %} + root {{ upload_done_dir }}; + {% endif %} +} diff --git a/roles/wiki/defaults/main.yml b/roles/wiki/defaults/main.yml index fedf090..66e570a 100644 --- a/roles/wiki/defaults/main.yml +++ b/roles/wiki/defaults/main.yml @@ -20,4 +20,4 @@ ikiwiki_git_wrapper: "{{ ikiwiki_path }}/hooks/emacsconf" ikiwiki_git_test_receive_wrapper: "{{ ikiwiki_path }}/hooks/emacsconf-pre" ikiwiki_git_base_url: //git.emacsconf.org/emacsconf-wiki ikiwiki_cgi_wrapper: "{{ ikiwiki_path }}/ikiwiki.cgi" - +ikiwiki_bare_git_dir: git://git.emacsconf.org/emacsconf-wiki diff --git a/roles/wiki/tasks/docker.yml b/roles/wiki/tasks/docker.yml index 4c7fd06..2bfe613 100644 --- a/roles/wiki/tasks/docker.yml +++ b/roles/wiki/tasks/docker.yml @@ -4,50 +4,6 @@ - lighttpd - supervisor - sudo -- name: Create the anon user - user: - name: anon - state: present - when: docker -- name: Set up Ikiwiki setup - template: - src: emacsconf.setup - dest: "{{ ikiwiki_path }}/emacsconf.setup" - owner: www-data - group: www-data -- name: Set up the ikiwiki directories - file: - dest: "{{ ikiwiki_dest }}" - state: directory - owner: ikiwiki - group: www-data - recurse: true -- name: Clone the bare git repo - git: - bare: true - repo: "{{ ikiwiki_git_source_mount }}" - dest: "{{ ikiwiki_bare_git_dir }}" - version: "{{ ikiwiki_git_branch }}" -- name: Set up post-update hook - template: - src: post-update - dest: "{{ ikiwiki_bare_git_dir }}/hooks/post-update" - mode: 0755 -- name: Remove sample - file: - path: "{{ ikiwiki_bare_git_dir }}/hooks/post-update.sample" - state: absent -- name: Set up ikiwiki post-update hook - template: - src: post-update.h00-ikiwiki-wrapper - dest: "{{ ikiwiki_bare_git_dir }}/hooks/post-update.h00-ikiwiki-wrapper" - mode: 0755 -- name: Change owner - file: - dest: "{{ ikiwiki_bare_git_dir }}" - recurse: true - owner: ikiwiki - group: www-data - name: Clone the working git repo git: repo: "{{ ikiwiki_bare_git_dir }}" @@ -67,19 +23,6 @@ service: name: lighttpd state: started -- name: Start ssh - tags: ssh - service: - name: ssh - state: started -- name: Set up SSH directory - tags: ssh - file: - path: /home/ikiwiki/.ssh - owner: ikiwiki - group: ikiwiki - state: directory - mode: 0700 - name: Set up SSH authentication tags: ssh block: diff --git a/roles/wiki/tasks/main.yml b/roles/wiki/tasks/main.yml index cd7cbdc..3ee7d4f 100644 --- a/roles/wiki/tasks/main.yml +++ b/roles/wiki/tasks/main.yml @@ -13,6 +13,11 @@ - nginx - wget state: present +- name: Start ssh + tags: ssh + service: + name: ssh + state: started - name: Create ikiwiki group group: name: ikiwiki @@ -22,11 +27,63 @@ name: ikiwiki group: ikiwiki state: present +- name: Create anon group + group: + name: anon + state: present +- name: Create anon user + user: + name: anon + group: anon + state: present +- include_tasks: bare-git.yml + when: docker or test_mode +- name: Configure safe directory + shell: git config --global --add safe.directory "{{ ikiwiki_src_dir }}" +- name: Clone the working git repo + git: + repo: "{{ ikiwiki_bare_git_dir }}" + dest: "{{ ikiwiki_src_dir }}" +- name: Switch the source git repo to the branch + shell: git checkout "{{ ikiwiki_git_branch }}" 2>/dev/null || git checkout -b "{{ ikiwiki_git_branch }}" + args: + chdir: "{{ ikiwiki_src_dir }}" +- name: Set the default branch policy + shell: git config --global pull.rebase false + become: true + become_user: ikiwiki +- name: Set up SSH directory + tags: ssh + file: + path: /home/ikiwiki/.ssh + owner: ikiwiki + group: ikiwiki + state: directory + mode: 0700 +- name: Set up SSH key access + tags: wiki-test-keys + template: + src: authorized_keys + dest: "/home/ikiwiki/.ssh/authorized_keys" + mode: 0600 + owner: ikiwiki + group: ikiwiki + when: test_mode +- name: Set up the ikiwiki directories + file: + dest: "{{ ikiwiki_dest }}" + state: directory + owner: ikiwiki + group: ikiwiki + recurse: true - name: Template the config ansible.builtin.template: src: emacsconf.setup dest: "{{ ikiwiki_path }}/emacsconf.setup" owner: ikiwiki + group: ikiwiki +- include_tasks: nginx.yml + when: test_mode - name: Create the plugin directory file: path: "{{ ikiwiki_plugin_path }}/IkiWiki/Plugin" @@ -40,7 +97,7 @@ - copyright.pm - htmlscrubber.pm - license.pm -- include: docker.yml +- include_tasks: docker.yml when: docker - name: Chown all the files to ikiwiki tags: wiki-plugins @@ -48,7 +105,12 @@ dest: "{{ ikiwiki_path }}" owner: ikiwiki group: ikiwiki + state: directory recurse: true +- name: Debug + tags: dev-wiki + debug: + var: ikiwiki_path - name: Regenerate all the files tags: wiki-regenerate, wiki-plugins become: true diff --git a/roles/wiki/templates/emacsconf.setup b/roles/wiki/templates/emacsconf.setup index d74fbb0..a7d2377 100644 --- a/roles/wiki/templates/emacsconf.setup +++ b/roles/wiki/templates/emacsconf.setup @@ -150,7 +150,7 @@ diffurl: {{ ikiwiki_git_base_url }}/diff/[[file]]?id=[[sha1_commit]]&id2=[[sha1_ # where to pull and push changes (set to empty string to disable) #gitorigin_branch: origin # branch that the wiki is stored in -gitmaster_branch: {{ ikiwiki_git_branch }} +#gitorigin_branch: {{ ikiwiki_git_branch }} # htmlscrubber plugin # PageSpec specifying pages not to scrub @@ -200,7 +200,7 @@ atom: 1 # PageSpec controlling which pages are locked #locked_pages: '!*/Discussion' #locked_pages: 'index or edit' -locked_pages: 'edit' +locked_pages: 'edit or donate or donors or script.js or local.css or templates/page.tmpl' # moderatedcomments plugin # PageSpec matching users or comment locations to moderate diff --git a/vagrant-inventory.yml b/vagrant-inventory.yml new file mode 100644 index 0000000..7e58cd3 --- /dev/null +++ b/vagrant-inventory.yml @@ -0,0 +1,10 @@ +vagrant: + hosts: + pad: + ansible_host: 127.0.0.1 + ansible_port: 2200 + ansible_user: vagrant + ansible_private_key_file: /home/sacha/proj/emacsconf/emacsconf-ansible/test/front/.vagrant/machines/pad/virtualbox/private_key + ansible_python_interpreter: /usr/bin/python3 + ansible_become: true + emacsconf_group: org diff --git a/vagrant-playbook.yml b/vagrant-playbook.yml new file mode 100644 index 0000000..1060b61 --- /dev/null +++ b/vagrant-playbook.yml @@ -0,0 +1,33 @@ +- name: Pre-flight checks and package installation + hosts: pad + become: true + gather_facts: false + pre_tasks: + - name: Ensure ntpdate is installed for time sync + ansible.builtin.apt: + name: ntpdate + state: present + update_cache: yes + - name: Synchronize system clock + ansible.builtin.command: ntpdate pool.ntp.org + changed_when: true + - name: Ensure ACL package is installed + ansible.builtin.apt: + name: acl + state: present +- name: Load vars + hosts: pad + tags: always + tasks: + - include_vars: + file: vagrant-vars.yml +- name: Set up pad proxy + hosts: pad + tags: proxy + roles: + - pad-proxy +- name: Set up pad + hosts: pad + tags: pad + roles: + - pad |