# Building reproducible Emacs
Andrew Tropin
[[!template id=vid src="https://mirror.csclub.uwaterloo.ca/emacsconf/2020/emacsconf-2020--08-building-reproducible-emacs--andrew-tropin.webm" subtitles="/2020/subtitles/emacsconf-2020--08-building-reproducible-emacs--andrew-tropin.vtt"]]
[Download compressed .webm video (29.4M)](https://media.emacsconf.org/2020/emacsconf-2020--08-building-reproducible-emacs--andrew-tropin--compressed32.webm)
[Download compressed .webm video (18.4M, highly compressed)](https://mirror.csclub.uwaterloo.ca/emacsconf/2020/smaller/emacsconf-2020--08-building-reproducible-emacs--andrew-tropin--vp9-q56-video-original-audio.webm)
[View transcript](#transcript)
It's not always easy to take part of someone's configuration and make
it work, it's almost never easy to move your configuration to fresh OS
installation or hardware. Not sure that this snippet is enough to
make package work? Forgot to install ripgrep in your system for
rg.el? Got a broken version of package on package-install?
There is a way to make an Emacs configuration reliable, composable and
self-contained. It's possible to freeze package versions, create
systemd unit for emacs daemon, maintain system dependencies and
package subconfigurations in one place with one tool.
The talk explains how to leverage the power of nix package manager and
use-package to make pretty good emacs configuration.
There is a stream record on the same topic:
<https://youtu.be/2_e3kPJQ93s>. It lacks few interesting points about
composability of such configuration approach, but already have enough
interesting information. The talk will be a little more structured
and more Emacs-users oriented.
<!-- from the pad --->
- Actual start and end time (EST): Start: 2020-11-28T11.26.34; Q&A:
2020-11-28T11.40.48; End 2020-11-28T11.43.33
# Questions
## Do you deal with config files such as emacs-custom.el, some which have sensitive data?
Sensitive data is in other directories that aren't shared, and
emacs-custom.el is completely avoided, as it prevents
reproducible/system independent behaviour.
## How did you learn Nix language basics? Just from the the manual?
He referred to the Nix IRC channel.
## What are the main advantages besides switching computers (which most people rarely do)?
Make parts of config available for projects - sharing with other
people.
## Have you tried Guix in place of Nix? (more parens! :) :)
Currently trying it, and also in-process of switching from Nix to Guix.
# Notes
- Emacs configuration is entangled with the system configuration
(dired uses ls, grep.el uses grep).
- Reproducible behaviour is therefore not only dependent of Emacs
compilation/configuration, but also system configuration.
- "config.el" files configure Emacs, and accompanying "default.nix"
files make sure that the correct packages/fonts/libraries/etc are
installed.
- Reproducible development environment: <https://github.com/abcdw/rde>
- Using Org-roam to demo how to config a Nix layer(?)
- custom.el conflicts with Nix(?)
# Related talks
- [[/2022/talks/rde|rde Emacs introduction]] - Andrew's 2022 talk
<a name="transcript"></a>
# Transcript
00:00:00.000 --> 00:00:24.056
Hello, everyone. I am Andrew Tropin. I
am a professional software engineer I
was playing with NixOS It's an operating
system based on the Nix package manager.
I came up with this interesting approach
for configuring Emacs. I want to share
it with you.
00:00:24.056 --> 00:01:31.389
I will start with the bold statement
that Emacs configuration is almost the
same as system configuration. It's not
related to that Emacs joke about Emacs
being an operating system. It's more
about Emacs being integrated with so
many tools inside the environment. For
example, if you don't even use any fancy
workflows, you use only plain Emacs
without any configuration, dired uses
ls, grep.el uses grep, and info files
placed somewhere in your system. Also
Emacs can interact with gpg, git, make,
and other stuff. When you grow your
Emacs Lisp init.el file or other files
in your .emacs.d directory, you get much
more integration with underlying
operating system.
00:01:31.389 --> 00:02:08.622
The question is: how to manage such
configuration? Because you can't just
take a bunch of .el files and move to a
different machine and be sure that
everything will work. Because you didn't
move your executables. You didn't move
configuration of other programs. You
didn't move your service configurations.
And you can't even just create dotfiles
for each program and move it with your
.el files. The approach would be a
little broader.
00:02:08.622 --> 00:02:23.722
Everything that I am showing today is
available on Github. Any source code,
you can find here. but my copy of the
repository is on my local machine.
00:02:23.722 --> 00:03:45.889
As you can see, the font is a little
small. And also, my terminal font is
also a little small. I can do a quick
fix and increase the font. But imagine
how cool it will be if you can have a
file which contains the configuration
for a system. You change some value.
Here, for example, fontSize = 16 and run
some command and based on this file and
some other includes your operating
system is built and all your environment
is set up and ready for use. For example
here, we already built the new operating
system, and everything is already
installed in my SSD. Now I can run the
program and you can see that my alacrity
terminal has much bigger font and also
if I restart my Emacs instance it by
default uses a much bigger font for any
buffer. Practical, and as you can see,
it's already working, thanks to Nix and
NixOS.
00:03:45.889 --> 00:04:44.556
I will explain a little later how it
works inside, but for now, let's specify
a little more what happened right now. I
fed my... Oh. It doesn't work. Sorry. I
want... I have my whole operating system
defined in a few Nix files. For example,
here you saw the file which defines some
variables for my environment and then a
few more files for different programs.
There is a folder which contains all
Emacs-related configuration. Also, there
are package definitions defined in Nix
package repositories which is also
included for the function which
generates the operating system.
00:04:44.556 --> 00:05:11.689
Getting all my configurations written in
Nix language and a few firewalls in ??
languages, everything is gathered
together, and from that input and only
from that input, the new operating
system is built. Emacs now is a part of
this operating system. I can distribute
this Emacs configuration with all the
environment that I want.
00:05:11.689 --> 00:05:50.789
Practical so far. Let's clarify which
problems does it solve. First of all,
the integration problem. For example, a
few minutes ago, you saw that I changed
one variable. That was to update... The
first one, for my terminal, and the
second one, for my Emacs. It's pretty
good that a few different programs can
share some data. For example, you can
have one of them for every application,
or something like that and you change
only one value in one place and the
whole operating system is updated.
00:05:50.789 --> 00:06:20.856
Also, another problem is
reproducibility. For example, when you
install your new instance of Emacs on
your laptop or something like that, you
can be sure that you will get the same
package versions and you can be sure
that the configuration of your work
results in newly-updated or
newly-installed packages.
00:06:20.856 --> 00:06:43.256
Also, if you update packages, sometimes
it's hard to revert, because it's the
way your package manager almost every
time works. You're just getting the
latest available packages. If they are
broken, you need to wait for the
maintainer to update them.
00:06:43.256 --> 00:07:39.656
And also, your basic configuration
almost always doesn't contain any native
dependencies, like executables or
something else. Recently, I saw some
attempts to make it possible to use
use-package for those needs, like
ensuring native dependencies or
something like that. It's obviously...
If your configuration isn't reproducible
and it doesn't have your whole
environment, placed in one repository,
it's very hard to share such
configuration. You can share part of
your configuration and some instruction
how to get a similar environment, but it
doesn't always work. Let's go closer to
actually Emacs configuration itself.
00:07:39.656 --> 00:08:10.839
I had some experience with Spacemacs and
Doom Emacs distributions. I also watched
a lot of videos and articles by
Protesilaos and a lot of other custom
configurations of many different cool
people. And also I was inspired by
use-package and decided that I will
create a folding structure for my Emacs
configuration.
00:08:10.839 --> 00:09:01.306
I will be using subconfigs. It's almost
the same as layers in Spacemacs, or
modules in Doom Emacs, which are
self-contained. They contain Emacs Lisp
code which configures all packages
necessary for this part of
configuration. It contains all Emacs
dependencies like Emacs packages. It
contains all native dependencies like
binaries or maybe info pages or
something like that. It also contains
variables that can be shared between
Emacs and other applications, and it can
contain service or system definitions
which configure your systemd service or
something like that that you use in your
workflow. For example, for synchronizing
your e-mails.
00:09:01.306 --> 00:10:06.922
Let's start from just the example that I
already am... I have a folding structure
for my configuration. I have some files
here. early-init just has this. Nothing
changes. It will be copied to that
.emacs.d directory later with some
exceptions that it will replace the Nix
dir and a symlink will be created to it.
I have use-package-init.el. It's part of
configuration that will be on top of
everything to be able to use use-package
in my subconfigurations. And actually
some Nix code to glue everything up and
config dirs which contain all my
subconfigs.
00:10:06.922 --> 00:11:10.105
Let's start from faces subconfig. Let's
start from config.el which can be
familiar for many people. Just
use-package definition for faces package
and some configuration for it which are
setting some attributes. It reads some
variables. Those variables are actually
defined in a different place. If I open
default.nix file, you can see that it
contains the definition or subconfig,
and it should contain a definition of
variables that it uses by... I forgot to
move it from my original default.nix
file somewhere here. You probably can
find definition of those variables just
right here.
00:11:10.105 --> 00:11:38.422
I took values from my Nix expressions.
Those values will be shared across my
alacrity, Emacs, and other applications.
Later, they will be placed in generated
Emacs configuration. They will be
available for faces config. Here I will
be referencing them just like Emacs
variables.
00:11:38.422 --> 00:12:39.222
Let's take a look at another more
complicated example. For example,
org-roam package. Just a basic
use-package configuration which uses a
variable and the definition. It's a
little more complex than the previous
one. Elisp configuration in the same
file. emacsPackages specified here.
Those two packages: org-roam and
company-org-roam. systemPackages: it's
something that should be available on
your host operating system. And for
emacsPackages, you need sqlite package,
and also the definition of the variable
which will be passed in my Emacs
configuration later. It's equal to my
workDir, which is defined in my
environment, and a subdirectory of it.
00:12:39.222 --> 00:12:43.222
([Amin:] Andrew, you have about five
minutes including questions.)
00:12:43.222 --> 00:13:26.222
Oh, okay. I'm almost finished. It was
last example. Let me open my Org file.
Okay. Right here. I won't give you an
introduction to Nix itself and the
underlying mechanism, but I can say that
there's already a proof of concept
framework for utilizing Nix and NixOS
for configuring Emacs and making a very
complex workflow reproducible on other
machines. It gives everything that we
saw right now.
00:13:26.222 --> 00:14:05.389
For the future work, I plan to
reimplement it in Guile, which is a
Scheme dialect, which is another Lisp
language, for the GNU Guix operating
system, because I like Lisp languages a
little more than Nix languages and I
want to make this project from proof of
concept to some state which will be
user-friendly and available for other
people. If I will have a lot of time, I
will make an operating system which will
be inspired by Lisp machines to make the
whole experience very Lispy.
00:14:05.389 --> 00:14:22.622
Thank you for your attention and now I
will answer questions. Oh. There is a
lot of... Okay. I see some questions.
00:14:22.622 --> 00:14:29.222
Did you release some config files such
as Emacs custom.el, some of which have
sensitive data?
00:14:29.222 --> 00:14:59.456
Ideally, in the folding way, I create a
separate directory called
local/share/emacs, and I place custom el
files here. It's not synchronized in any
way, and it will be just lost in case
you move to a separate machine. I do it
for a purpose, because I don't use
custom.el. It's hard to make it
reproducible if you're using such
mechanism as custom.el.
00:14:59.456 --> 00:15:06.656
How do you learn the Nix language
basics? Just from the manual?
00:15:06.656 --> 00:15:32.989
I read a lot of documentation. Also, I
saw the course like Learn Nix in 15
minutes. And also there was another
resource. Better to ask this question in
Nix or NixOS channel in IRC, which will
be treated in more details.
00:15:32.989 --> 00:15:38.909
What are the main advantages besides
switching computers, which most people
rarely do?
00:15:38.909 --> 00:16:10.556
For example, the original idea was to
make part of configurations available
for projects. For example, you have some
project, you made the setup, and want
other developers to use the same setup
on their machine, but you implement only
the part of stuff, like one subconfig
especially for this language for this
project. With such approach, you can
easily share such subconfig with other
people.
00:16:10.556 --> 00:16:15.239
Have you tried Guix in place of Nix?
00:16:15.239 --> 00:16:41.239
Yes, I tried it, and currently I am in
the state of switching from Nix to Guix.
You can follow my Youtube channel, I
think, I do streams twice in a month
talking about reproducibility and
related stuff. Probably soon I will be
talking about installation of Guix and
configuration of it.
00:16:41.239 --> 00:16:50.406
In case you're watching this video
later, you can find me somewhere on the
network using those contacts. It's my
nickname and my e-mail address.
00:16:54.072 --> 00:17:04.622
([Amin:] Awesome. I think we're wrapping
up just on time. Thank you so much,
Andrew, for your great talk, and for
hanging out to answer the questions
live.)
00:17:04.622 --> 00:17:18.000
[Andrew:] Thank you for organizing the
conference and thank you all
participants for questions and
participation. See you soon!
