path: root/2020/subtitles/emacsconf-2020--08-building-reproducible-emacs--andrew-tropin.vtt

WEBVTT

00:00:00.000 --> 00:00:04.556
Hello, everyone. I am Andrew Tropin.

00:00:04.556 --> 00:00:06.622
I am a professional software engineer

00:00:06.622 --> 00:00:11.622
I was playing with NixOS

00:00:11.622 --> 00:00:15.322
It's an operating system based on the
Nix package manager.

00:00:15.322 --> 00:00:21.089
I came up with this interesting
approach for configuring Emacs.

00:00:21.089 --> 00:00:24.056
I want to share it with you.

00:00:24.056 --> 00:00:27.756
I will start with the bold statement that

00:00:27.756 --> 00:00:30.822
Emacs configuration is almost the same

00:00:30.822 --> 00:00:33.022
as system configuration.

00:00:33.022 --> 00:00:37.262
It's not related to that Emacs joke

00:00:37.262 --> 00:00:39.922
about Emacs being an operating system.

00:00:39.922 --> 00:00:44.489
It's more about Emacs being integrated

00:00:44.489 --> 00:00:48.589
with so many tools inside the environment.

00:00:48.589 --> 00:00:53.089
For example, if you don't even use any
fancy workflows,

00:00:53.089 --> 00:00:57.256
you use only plain Emacs without any
configuration,

00:00:57.256 --> 00:01:02.556
dired uses ls, grep.el uses grep,

00:01:02.556 --> 00:01:09.356
and info files placed
somewhere in your system.

00:01:09.356 --> 00:01:15.489
Also Emacs can interact with gpg, git,
make, and other stuff.

00:01:15.489 --> 00:01:20.789
When you grow your Emacs Lisp

00:01:20.789 --> 00:01:23.189
init.el file

00:01:23.189 --> 00:01:27.222
or other files in your .emacs.d directory,

00:01:27.222 --> 00:01:29.989
you get much more integration

00:01:29.989 --> 00:01:31.389
with underlying operating system.

00:01:31.389 --> 00:01:36.922
The question is: how to manage such configuration?

00:01:36.922 --> 00:01:40.922
Because you can't just take a bunch of
.el files

00:01:40.922 --> 00:01:43.456
and move to a different machine

00:01:43.456 --> 00:01:47.122
and be sure that everything will work.

00:01:47.122 --> 00:01:50.577
Because you didn't move your
executables.

00:01:50.577 --> 00:01:53.522
You didn't move configuration
of other programs.

00:01:53.522 --> 00:01:55.089
You didn't move your service configurations.

00:01:55.089 --> 00:02:01.889
And you can't even just create dotfiles
for each program

00:02:01.889 --> 00:02:05.022
and move it with your .el files.

00:02:05.022 --> 00:02:08.622
The approach would be a little broader.

00:02:08.622 --> 00:02:13.056
Everything that I am showing today

00:02:13.056 --> 00:02:15.322
is available on Github.

00:02:15.322 --> 00:02:18.022
Any source code, you can find here.

00:02:18.022 --> 00:02:21.589
but my copy of the repository

00:02:21.589 --> 00:02:23.722
is on my local machine.

00:02:23.722 --> 00:02:27.722
As you can see, the font is a little small.

00:02:27.722 --> 00:02:31.122
And also, my terminal font is also a
little small.

00:02:31.122 --> 00:02:36.756
I can do a quick fix and increase the font.

00:02:36.756 --> 00:02:40.789
But imagine how cool it will be

00:02:40.789 --> 00:02:45.756
if you can have a file which contains
the configuration for a system.

00:02:45.756 --> 00:02:52.489
You change some value. Here, for
example, fontSize = 16

00:02:52.489 --> 00:02:54.589
and run some command

00:02:54.589 --> 00:02:58.489
and based on this file

00:02:58.489 --> 00:03:00.322
and some other includes

00:03:00.322 --> 00:03:02.422
your operating system is built

00:03:02.422 --> 00:03:06.389
and all your environment is set up

00:03:06.389 --> 00:03:07.656
and ready for use.

00:03:07.656 --> 00:03:11.622
For example here, we already built
the new operating system,

00:03:11.622 --> 00:03:17.856
and everything is already installed in
my SSD.

00:03:17.856 --> 00:03:21.322
Now I can run the program and you can
see that

00:03:21.322 --> 00:03:29.422
my alacrity terminal has much bigger font

00:03:29.422 --> 00:03:31.789
and also if I restart my Emacs instance

00:03:31.789 --> 00:03:34.089
it by default uses

00:03:34.089 --> 00:03:36.889
a much bigger font for any buffer.

00:03:36.889 --> 00:03:41.089
Practical, and as you can see, it's
already working,

00:03:41.089 --> 00:03:45.889
thanks to Nix and NixOS.

00:03:45.889 --> 00:03:50.722
I will explain a little later how it
works inside,

00:03:50.722 --> 00:03:57.089
but for now, let's specify a little more

00:03:57.089 --> 00:04:00.789
what happened right now.

00:04:00.789 --> 00:04:08.156
I fed my... Oh. It doesn't work. Sorry.
I want...

00:04:08.156 --> 00:04:13.056
I have my whole operating system

00:04:13.056 --> 00:04:15.589
defined in a few Nix files.

00:04:15.589 --> 00:04:18.689
For example, here you saw the file

00:04:18.689 --> 00:04:22.522
which defines some variables for my environment

00:04:22.522 --> 00:04:24.256
and then a few more files

00:04:24.256 --> 00:04:25.722
for different programs.

00:04:25.722 --> 00:04:30.056
There is a folder which contains all
Emacs-related configuration.

00:04:30.056 --> 00:04:36.989
Also, there are package definitions
defined in Nix package repositories

00:04:36.989 --> 00:04:42.522
which is also included for the function
which generates

00:04:42.522 --> 00:04:44.556
the operating system.

00:04:44.556 --> 00:04:47.622
Getting all my configurations
written in Nix language

00:04:47.622 --> 00:04:51.174
and a few firewalls in ?? languages,

00:04:51.174 --> 00:04:54.700
everything is gathered together,

00:04:54.700 --> 00:04:56.722
and from that input

00:04:56.722 --> 00:04:58.322
and only from that input,

00:04:58.322 --> 00:05:00.489
the new operating system is built.

00:05:00.489 --> 00:05:03.856
Emacs now is a part of this operating system.

00:05:03.856 --> 00:05:08.422
I can distribute this Emacs configuration

00:05:08.422 --> 00:05:11.689
with all the environment that I want.

00:05:11.689 --> 00:05:18.389
Practical so far. Let's clarify which
problems does it solve.

00:05:18.389 --> 00:05:21.756
First of all, the integration problem.

00:05:21.756 --> 00:05:27.389
For example, a few minutes ago, you saw
that I changed one variable.

00:05:27.389 --> 00:05:31.348
That was to update... The first one, for
my terminal,

00:05:31.348 --> 00:05:33.889
and the second one, for my Emacs.

00:05:33.889 --> 00:05:40.322
It's pretty good that a few different
programs can share some data.

00:05:40.322 --> 00:05:43.822
For example, you can have one of them
for every application,

00:05:43.822 --> 00:05:45.222
or something like that

00:05:45.222 --> 00:05:48.356
and you change only one value in one place

00:05:48.356 --> 00:05:50.789
and the whole operating system is updated.

00:05:50.789 --> 00:05:56.422
Also, another problem is reproducibility.

00:05:56.422 --> 00:06:00.156


00:06:00.156 --> 00:06:06.600
For example, when you install
your new instance of Emacs

00:06:06.600 --> 00:06:11.089
on your laptop or something like that,

00:06:11.089 --> 00:06:14.289
you can be sure that you will get the
same package versions

00:06:14.289 --> 00:06:17.189
and you can be sure that the
configuration of your work

00:06:17.189 --> 00:06:20.856
results in newly-updated or
newly-installed packages.

00:06:20.856 --> 00:06:25.056
Also, if you update packages

00:06:25.056 --> 00:06:27.656
sometimes it's hard to revert,

00:06:27.656 --> 00:06:36.289
because it's the way your package
manager almost every time works.

00:06:36.289 --> 00:06:38.722
You're just getting the latest
available packages.

00:06:38.722 --> 00:06:43.256
If they are broken, you need to wait for
the maintainer to update them.

00:06:43.256 --> 00:06:50.989
And also, your basic configuration
almost always doesn't contain

00:06:50.989 --> 00:06:56.156
any native dependencies, like
executables or something else.

00:06:56.156 --> 00:07:00.689
Recently, I saw some attempts to make it
possible to

00:07:00.689 --> 00:07:03.089
use use-package for those needs,

00:07:03.089 --> 00:07:06.356
like ensuring native dependencies
or something like that.

00:07:06.356 --> 00:07:11.134
It's obviously... If
your configuration isn't reproducible

00:07:11.134 --> 00:07:15.322
and it doesn't have
your whole environment,

00:07:15.322 --> 00:07:19.522
placed in one repository,

00:07:19.522 --> 00:07:22.322
it's very hard to share such
configuration.

00:07:22.322 --> 00:07:27.089
You can share part of your configuration
and some instruction

00:07:27.089 --> 00:07:32.222
how to get a similar environment,
but it doesn't always work.

00:07:32.222 --> 00:07:39.656
Let's go closer to actually Emacs
configuration itself.

00:07:39.656 --> 00:07:45.306
I had some experience with Spacemacs and
Doom Emacs distributions.

00:07:45.306 --> 00:07:50.414
I also watched a lot of videos and
articles by Protesilaos

00:07:50.414 --> 00:07:56.756
and a lot of other custom configurations
of many different cool people.

00:07:56.756 --> 00:08:03.039
And also I was inspired by use-package

00:08:03.039 --> 00:08:10.839
and decided that I will create a folding
structure for my Emacs configuration.

00:08:10.839 --> 00:08:16.172
I will be using subconfigs. It's almost
the same as layers in Spacemacs,

00:08:16.172 --> 00:08:20.972
or modules in Doom Emacs, which are
self-contained.

00:08:20.972 --> 00:08:26.287
They contain Emacs Lisp code which
configures all packages necessary

00:08:26.287 --> 00:08:28.789
for this part of configuration.

00:08:28.789 --> 00:08:33.493
It contains all Emacs dependencies like
Emacs packages.

00:08:33.493 --> 00:08:36.572
It contains all native dependencies

00:08:36.572 --> 00:08:40.039
like binaries or maybe info pages or
something like that.

00:08:40.039 --> 00:08:45.115
It also contains variables
that can be shared between

00:08:45.115 --> 00:08:47.989
Emacs and other applications,

00:08:47.989 --> 00:08:52.072
and it can contain service
or system definitions

00:08:52.072 --> 00:08:56.072
which configure your systemd service or
something like that

00:08:56.072 --> 00:09:01.306
that you use in your workflow. For
example, for synchronizing your e-mails.

00:09:01.306 --> 00:09:06.239
Let's start from just the example

00:09:06.239 --> 00:09:16.618
that I already am... I have a folding
structure for my configuration.

00:09:16.618 --> 00:09:25.789
I have some files here.
early-init just has this.

00:09:25.789 --> 00:09:33.006
Nothing changes. It will be copied to
that .emacs.d directory later

00:09:33.006 --> 00:09:37.306
with some exceptions that

00:09:37.306 --> 00:09:40.789
it will replace the Nix dir and a
symlink will be created to it.

00:09:40.789 --> 00:09:47.889
I have use-package-init.el.
It's part of configuration

00:09:47.889 --> 00:09:51.522
that will be on top of everything

00:09:51.522 --> 00:09:55.589
to be able to use use-package in my
subconfigurations.

00:09:55.589 --> 00:10:01.156
And actually some Nix code to glue
everything up

00:10:01.156 --> 00:10:06.922
and config dirs which contain all my subconfigs.

00:10:06.922 --> 00:10:11.389
Let's start from faces subconfig.

00:10:11.389 --> 00:10:14.556
Let's start from config.el

00:10:14.556 --> 00:10:20.022
which can be familiar for many people.

00:10:20.022 --> 00:10:23.556
Just use-package definition
for faces package

00:10:23.556 --> 00:10:24.122
and some configuration for it

00:10:24.122 --> 00:10:29.589
which are setting some attributes.

00:10:29.589 --> 00:10:32.156
It reads some variables.

00:10:32.156 --> 00:10:36.922
Those variables are actually defined in
a different place.

00:10:36.922 --> 00:10:44.422
If I open default.nix file, you can see
that it contains

00:10:44.422 --> 00:10:52.689
the definition or subconfig, and it
should contain a definition of variables

00:10:52.689 --> 00:10:55.322
that it uses by... I forgot to move it from

00:10:55.322 --> 00:11:01.889
my original default.nix file somewhere
here.

00:11:01.889 --> 00:11:10.105
You probably can find definition of
those variables just right here.

00:11:10.105 --> 00:11:13.722
I took values from my
Nix expressions.

00:11:13.722 --> 00:11:23.956
Those values will be shared across my
alacrity, Emacs, and other applications.

00:11:23.956 --> 00:11:27.789
Later, they will be placed in generated
Emacs configuration.

00:11:27.789 --> 00:11:32.856
They will be available for faces config.

00:11:32.856 --> 00:11:38.422
Here I will be referencing them just
like Emacs variables.

00:11:38.422 --> 00:11:43.356
Let's take a look at another more
complicated example.

00:11:43.356 --> 00:11:45.922
For example, org-roam package.

00:11:45.922 --> 00:11:49.922
Just a basic use-package configuration

00:11:49.922 --> 00:11:54.607
which uses a variable and
the definition.

00:11:54.607 --> 00:12:00.322
It's a little more complex than the
previous one.

00:12:00.322 --> 00:12:04.222
Elisp configuration in the same file.

00:12:04.222 --> 00:12:08.856
emacsPackages specified here.

00:12:08.856 --> 00:12:12.289
Those two packages: org-roam and company-org-roam.

00:12:12.289 --> 00:12:16.956
systemPackages: it's something that
should be available

00:12:16.956 --> 00:12:18.456
on your host operating system.

00:12:18.456 --> 00:12:24.756
And for emacsPackages, you need sqlite
package,

00:12:24.756 --> 00:12:27.922
and also the definition of the variable

00:12:27.922 --> 00:12:31.556
which will be passed in my Emacs
configuration later.

00:12:31.556 --> 00:12:37.722
It's equal to my workDir, which is
defined in my environment,

00:12:37.722 --> 00:12:39.222
and a subdirectory of it.

00:12:39.222 --> 00:12:43.222
([Amin:] Andrew, you have about five
minutes including questions.)

00:12:43.222 --> 00:12:50.822
Oh, okay. I'm almost finished. It was
last example.

00:12:50.822 --> 00:12:56.556
Let me open my Org file. Okay.

00:12:56.556 --> 00:13:02.222
Right here. I won't give you an
introduction to Nix itself

00:13:02.222 --> 00:13:06.922
and the underlying mechanism,

00:13:06.922 --> 00:13:11.140
but I can say that there's already a
proof of concept framework

00:13:11.140 --> 00:13:14.622
for utilizing Nix and NixOS

00:13:14.622 --> 00:13:18.389
for configuring Emacs
and making a very complex workflow

00:13:18.389 --> 00:13:22.056
reproducible on other machines.

00:13:22.056 --> 00:13:26.222
It gives everything that we
saw right now.

00:13:26.222 --> 00:13:31.522
For the future work, I plan to
reimplement it in Guile,

00:13:31.522 --> 00:13:36.189
which is a Scheme dialect,
which is another Lisp language,

00:13:36.189 --> 00:13:39.199
for the GNU Guix operating system,

00:13:39.199 --> 00:13:41.856
because I like Lisp languages

00:13:41.856 --> 00:13:46.822
a little more than Nix languages and I
want to make

00:13:46.822 --> 00:13:50.089
this project from proof of concept
to some state which

00:13:50.089 --> 00:13:54.189
will be user-friendly and
available for other people.

00:13:54.189 --> 00:13:59.156
If I will have a lot of time,
I will make an operating system

00:13:59.156 --> 00:14:01.356
which will be inspired by Lisp machines

00:14:01.356 --> 00:14:05.389
to make the whole experience very Lispy.

00:14:05.389 --> 00:14:08.556
Thank you for your attention

00:14:08.556 --> 00:14:12.189
and now I will answer questions.

00:14:12.189 --> 00:14:22.622
Oh. There is a lot of... Okay. I see
some questions.

00:14:22.622 --> 00:14:29.222
Did you release some config files such
as Emacs custom.el, some of which have
sensitive data?

00:14:29.222 --> 00:14:39.922
Ideally, in the folding way, I create a
separate directory called local/share/emacs,

00:14:39.922 --> 00:14:45.089
and I place custom el files here. It's
not synchronized in any way,

00:14:45.089 --> 00:14:48.922
and it will be just lost in case you
move to a separate machine.

00:14:48.922 --> 00:14:52.456
I do it for a purpose, because I don't
use custom.el.

00:14:52.456 --> 00:14:59.456
It's hard to make it reproducible if
you're using such mechanism as custom.el.

00:14:59.456 --> 00:15:06.656
How do you learn the Nix language
basics? Just from the manual?

00:15:06.656 --> 00:15:10.141
I read a lot of documentation.

00:15:10.141 --> 00:15:15.989
Also, I saw the course like Learn Nix in
15 minutes.

00:15:15.989 --> 00:15:19.289
And also there was another resource.

00:15:19.289 --> 00:15:25.689
Better to ask this question in Nix or
NixOS channel in IRC,

00:15:25.689 --> 00:15:32.989
which will be treated in more details.

00:15:32.989 --> 00:15:36.656
What are the main advantages besides
switching computers,

00:15:36.656 --> 00:15:38.909
which most people rarely do?

00:15:38.909 --> 00:15:44.422
For example, the original idea was to
make part of configurations

00:15:44.422 --> 00:15:46.422
available for projects.

00:15:46.422 --> 00:15:48.156
For example, you have some project,

00:15:48.156 --> 00:15:51.914
you made the setup,
and want other developers

00:15:51.914 --> 00:15:55.256
to use the same setup
on their machine,

00:15:55.256 --> 00:15:58.122
but you implement only the part of
stuff,

00:15:58.122 --> 00:16:01.156
like one subconfig especially for
this language

00:16:01.156 --> 00:16:05.389
for this project. With such approach,
you can easily

00:16:05.389 --> 00:16:10.556
share such subconfig with other people.

00:16:10.556 --> 00:16:15.239
Have you tried Guix in place of Nix?

00:16:15.239 --> 00:16:22.272
Yes, I tried it, and currently I am in
the state of switching from Nix to Guix.

00:16:22.272 --> 00:16:26.739
You can follow my Youtube channel, I think,

00:16:26.739 --> 00:16:32.522
I do streams twice in a month

00:16:32.522 --> 00:16:35.922
talking about reproducibility and
related stuff.

00:16:35.922 --> 00:16:39.306
Probably soon I will be talking
about installation of Guix

00:16:39.306 --> 00:16:41.239
and configuration of it.

00:16:41.239 --> 00:16:44.956
In case you're watching this video later,

00:16:44.956 --> 00:16:47.972
you can find me somewhere on the network
using those contacts.

00:16:47.972 --> 00:16:50.406
It's my nickname and my e-mail address.

00:16:54.072 --> 00:16:56.556
([Amin:] Awesome. I think we're wrapping
up just on time.

00:16:56.556 --> 00:17:00.889
Thank you so much, Andrew, for your
great talk,

00:17:00.889 --> 00:17:04.622
and for hanging out to answer the
questions live.)

00:17:04.622 --> 00:17:08.022
[Andrew:] Thank you for organizing the
conference

00:17:08.022 --> 00:17:11.572
and thank you all participants for
questions and participation.

00:17:11.572 --> 00:17:18.000
See you soon!