path: root/2024/captions/emacsconf-2024-secrets--committing-secrets-with-git-using-sopsmode--jonathan-otsuka--main.vtt

WEBVTT

NOTE Introduction

00:00:00.000 --> 00:00:01.349
Yeah, my talk is committing

00:00:01.350 --> 00:00:10.189
secrets with Git via SOPS mode. So what is SOPS? SOPS

00:00:10.190 --> 00:00:15.069
came originally from Mozilla, and their acronym was Secret

00:00:15.070 --> 00:00:19.669
Operations, so S-O-P-S. It's a utility that allows you to

00:00:19.670 --> 00:00:24.269
encrypt pretty much any file you want and then have the

00:00:24.270 --> 00:00:27.869
ability to commit it or just share it with somebody that has

00:00:27.870 --> 00:00:32.709
the ability to decrypt it. I've mostly used it with AWS KMS,

00:00:32.710 --> 00:00:36.829
but there's a number of encryptions, ways you can encrypt

00:00:36.830 --> 00:00:41.909
it. Um, so yeah, that's what SOPS mode is. Most of the

00:00:41.910 --> 00:00:45.709
time I've used it is with application or deployment

00:00:45.710 --> 00:00:48.269
secrets--decrypt them on the fly during a

00:00:48.270 --> 00:00:52.109
pipeline run and then use them. We've also

00:00:52.110 --> 00:00:56.549
been using it for kind of a self-service for engineers

00:00:56.550 --> 00:00:59.629
to be able to say, if there's an API token that they

00:00:59.630 --> 00:01:03.429
need either in the container or that gets put

00:01:03.430 --> 00:01:09.109
somewhere else, that's another way to use SOPS.

00:01:09.110 --> 00:01:13.949
Just sharing secrets. The tooling, there's

00:01:13.950 --> 00:01:16.814
quite a bit of tooling for Terraform.

00:01:16.815 --> 00:01:19.090
You can just decrypt it and then use it

00:01:19.091 --> 00:01:21.309
however you want. Ansible, it's another

00:01:21.310 --> 00:01:23.648
place, and then Kubernetes...

00:01:23.649 --> 00:01:26.124
There'll be links at the very end.

00:01:26.125 --> 00:01:28.982
There's actually a Nix SOPS too.

00:01:28.983 --> 00:01:30.237
I think there's a link in the end.

00:01:30.238 --> 00:01:32.830
So yeah, I'll just show a quick demo.

00:01:32.831 --> 00:01:35.286
I'll actually show it in Emacs too,

00:01:35.287 --> 00:01:36.983
but this is the idea.

00:01:36.984 --> 00:01:39.450
I'm just going to create a file

00:01:39.451 --> 00:01:42.051
and then I'm going to encrypt it with age.

00:01:42.052 --> 00:01:46.874
Then we should see the encrypted file

00:01:46.875 --> 00:01:53.189
be output here. The idea is you can decrypt it

00:01:53.190 --> 00:01:59.349
there. So my talk was... the reason how my

00:01:59.350 --> 00:02:04.429
talk came about was there was no mode like this yet.

00:02:04.430 --> 00:02:08.829
So I didn't want to have to...

00:02:08.830 --> 00:02:10.269
What you can do is you can pass

00:02:10.270 --> 00:02:13.029
in the editor variable, set your Emacs,

00:02:13.030 --> 00:02:16.441
then call the command, but that opens

00:02:16.442 --> 00:02:20.589
a whole new window. I wanted to live in my

00:02:20.590 --> 00:02:22.994
current Emacs. So this is that

00:02:22.995 --> 00:02:25.555
same encrypted file that we just created.

00:02:25.556 --> 00:02:28.566
I'm going to quickly do C-c C-d.

00:02:28.567 --> 00:02:32.309
So now we're in the SOPS decrypted mode of the

00:02:32.310 --> 00:02:38.057
file. I can save this, or make changes and save it.

00:02:38.058 --> 00:02:39.963
And then it resaves it.

00:02:39.964 --> 00:02:42.918
I'll just show you that decrypting it

00:02:42.919 --> 00:02:44.629
shows what we changed.

00:02:44.630 --> 00:02:52.831
I think that's most of my talk.

00:02:52.832 --> 00:02:55.882
There's future stuff that I would like to do

00:02:55.883 --> 00:03:00.447
with this. There's no way to create SOPS files

00:03:00.448 --> 00:03:03.191
from scratch. And then just putting more

00:03:03.192 --> 00:03:06.069
documentation around the other ways you can

00:03:06.070 --> 00:03:14.029
set up your editor to decrypt. But yeah,

00:03:14.030 --> 00:03:19.109
here's all the links. I haven't uploaded

00:03:19.110 --> 00:03:23.309
this yet, but yeah, that is my talk.

NOTE Q&A technical issues

00:03:23.310 --> 00:03:27.770
[Leo] Okay. Thank you, Jonathan.

00:03:27.771 --> 00:03:32.692
Let me just make sure. So everything you've mentioned

00:03:32.693 --> 00:03:34.887
about putting stuff available to everyone,

00:03:34.888 --> 00:03:36.469
we'll make sure that everything

00:03:36.470 --> 00:03:38.513
ends up on the pad and on the website,

00:03:38.514 --> 00:03:40.850
so don't worry. Let me see if we can get up

00:03:40.851 --> 00:03:41.753
the pad for you.

00:03:41.754 --> 00:03:43.284
Do you have any preference with regards

00:03:43.285 --> 00:03:45.467
to the questions? Do you want to read them yourself

00:03:45.468 --> 00:03:50.121
or do you want one of us to read them for you?

00:03:50.122 --> 00:03:53.389
[Jonathan]: I'm okay with talking first,

00:03:53.390 --> 00:03:57.461
saying it out loud if there are some.

00:03:57.462 --> 00:04:00.749
[Leo]: Sure. Let me just find you the pads.

00:04:00.750 --> 00:04:02.757
Where is it? There you go.

00:04:02.758 --> 00:04:05.409
Do you have access to the pad on your end?

00:04:05.410 --> 00:04:06.153
Yep.

00:04:06.154 --> 00:04:09.549
Okay. Well, if you, since you're already showing

00:04:09.550 --> 00:04:12.389
your screen, if you can maybe switch the window to the one

00:04:12.390 --> 00:04:13.435
that is hosting the pad

00:04:13.436 --> 00:04:15.814
and feel free to start answering questions.

00:04:15.815 --> 00:04:16.262
Yep.

00:04:16.263 --> 00:04:20.109
It didn't look like we have any yet, but...

00:04:20.110 --> 00:04:21.942
Well, there's still coming, don't worry.

00:04:21.943 --> 00:04:29.149
We're just waiting for people to catch up.

00:04:29.150 --> 00:04:31.533
I probably need to make it bigger.

00:04:31.534 --> 00:04:34.460
Is it big enough or do I need to make it bigger?

00:04:34.461 --> 00:04:40.247
Right now, it's just a black screen on my end, so...

00:04:40.248 --> 00:04:45.269
Oh, wow. Weird. I can see it on mine, weirdly.

00:04:45.270 --> 00:04:47.536
Maybe it's just me. Let me check here.

00:04:47.537 --> 00:04:48.989
No, it seems to be just a

00:04:48.990 --> 00:04:50.069
black square, even on the stream.

00:04:50.070 --> 00:05:00.927
Try it again. That change at all? No, it's still black.

00:05:00.928 --> 00:05:02.743
Can you maybe start switching window

00:05:02.744 --> 00:05:04.069
and coming back to the one?

00:05:04.070 --> 00:05:08.869
Otherwise, I'll just stream it on my end.

00:05:08.870 --> 00:05:13.629
Yeah. All right, I'll do it. I'll take presenter in just a

00:05:13.630 --> 00:05:22.229
second. Yeah, sorry about that. Thank you.

00:05:22.230 --> 00:05:27.069
If I can take presenter, and I will share the screen.

00:05:27.070 --> 00:05:36.749
Sorry, I'm just trying to find a chat. There we go.

00:05:36.750 --> 00:05:39.509
Normally, I'm not supposed to be on the dev track, which is

00:05:39.510 --> 00:05:42.309
why I'm confusing all my windows. Give me just a second.

00:05:42.310 --> 00:05:53.709
Shell, casual. So we are on the dev track, and it is this

00:05:53.710 --> 00:05:54.189
one.

00:05:54.190 --> 00:06:08.229
There we go. No, that's not a guide, damn it. Secrets.

00:06:08.230 --> 00:06:10.109
And...

00:06:10.110 --> 00:06:15.509
There we go, finally.

00:06:15.510 --> 00:06:19.109
Ah. Probably just for the delay, do some jazz hands in the

00:06:19.110 --> 00:06:20.889
background as we did in the start.

00:06:20.890 --> 00:06:23.600
It feels like Yordle[??] Castle this year,

00:06:23.601 --> 00:06:25.462
where nothing works properly.

00:06:25.463 --> 00:06:26.269
That's right.

00:06:26.270 --> 00:06:39.149
All right. There we go. It's loading up. Obviously.

00:06:39.150 --> 00:06:44.189
There we go.

00:06:44.190 --> 00:06:49.189
All right. You should be able to see my screen now. Yep. All

00:06:49.190 --> 00:06:53.789
right. So, well, we've gone so far. Oh, it did stop. Damn it.

00:06:53.790 --> 00:07:02.989
Sorry, now it's BBB not behaving properly. That's right.

00:07:02.990 --> 00:07:10.309
Okay, let me just join, leave and join again. Okay. I just did

00:07:10.310 --> 00:07:11.909
exactly that for what it's worth.

00:07:11.910 --> 00:07:26.189
Nothing. All right.

00:07:26.190 --> 00:07:29.029
All right, I seem to be back. Let me show. And there we go.

00:07:29.030 --> 00:07:36.909
All right, everything is working. I'm not touching

00:07:36.910 --> 00:07:39.187
anything. So. Cool.

NOTE Q: Can you describe some potential interactive uses for this within Emacs?

00:07:39.188 --> 00:07:43.629
Yeah, I'll just start with the top. Can

00:07:43.630 --> 00:07:47.349
you describe some potential interactive uses for this with

00:07:47.350 --> 00:07:52.789
an Emacs? Um, I'm, I'm not actually sure what this means.

00:07:52.790 --> 00:08:01.029
Could we, could you add some more context maybe? Or, um,

00:08:01.030 --> 00:08:03.549
I think we'll maybe come back to that one. I'm not sure what,

00:08:03.550 --> 00:08:08.531
uh, potential interactive uses mean, but.

NOTE Q: Is this saved in the repo or file as \"run sops here\" or is the encrypted blob in the git repo?

00:08:08.532 --> 00:08:10.429
Yep. Uh, is this

00:08:10.430 --> 00:08:18.749
saved in the repo or file as run SOPs here? Oh, encrypted.

00:08:18.750 --> 00:08:24.829
They're saved as just text files so that you can do

00:08:24.830 --> 00:08:28.103
SOPs and encrypt like a binary. I think in the end,

00:08:28.104 --> 00:08:30.819
no matter what, they become just a text file,

00:08:30.820 --> 00:08:34.520
and then it does the encoding and decoding on the fly

00:08:34.521 --> 00:08:36.753
when you encrypt or decrypt. So no matter

00:08:36.754 --> 00:08:41.984
what it's going to be, I think it might just be

00:08:41.985 --> 00:08:44.989
a JSON in the end. Uh, so yeah.

00:08:44.990 --> 00:08:56.309
I'll try to, well, I can type out that answer, but all

00:08:56.310 --> 00:08:56.855
right.

00:08:56.856 --> 00:08:59.429
Don't worry about typing it out.

00:08:59.430 --> 00:09:00.989
We are gathering the

00:09:00.990 --> 00:09:04.069
recordings at the end, you know, even answers that are not

00:09:04.070 --> 00:09:05.782
provided, we'll type them out eventually.

00:09:05.783 --> 00:09:09.029
So don't stress too much about the actual answers being written.

00:09:09.030 --> 00:09:12.066
Okay. All right. So I'll go to the third one.

NOTE Q: How do you decide whether to use SOPS or other solutions such as pass-cli?

00:09:12.067 --> 00:09:13.189
How do you decide

00:09:13.190 --> 00:09:18.949
whether to use SOPS or other solutions such as pass-cli?

00:09:18.950 --> 00:09:24.469
The biggest use case that I've been using it recently is,

00:09:24.470 --> 00:09:29.109
Bitbucket has a way to... In a repository,

00:09:29.110 --> 00:09:35.829
you can store non-secrets and secrets. So

00:09:35.830 --> 00:09:39.549
we're trying to move the secrets into the repository

00:09:39.550 --> 00:09:43.109
and then allow the engineers to have

00:09:43.110 --> 00:09:48.789
access to that.

00:09:48.790 --> 00:09:52.389
Bitbucket variables is a black box. Since the devs can

00:09:52.390 --> 00:09:56.841
access it, it's manual work for everybody

00:09:56.842 --> 00:10:00.869
that has to deal with it. Since we're moving

00:10:00.870 --> 00:10:04.339
SOPS-encrypted files into the repo,

00:10:04.340 --> 00:10:06.830
now there's that trackability

00:10:06.831 --> 00:10:10.942
from who made the change and what it changed from,

00:10:10.943 --> 00:10:16.589
what did it go to, and just things like that.

00:10:16.590 --> 00:10:23.629
You can use it anytime you'd want to commit them.

NOTE Q: One limitation with guix (similar package manager to nix) is there is no great way of storing secrets in the store, would SOPS be useful for this?

00:10:23.630 --> 00:10:32.029
One limitation with GUIX is there's no great way to store

00:10:32.030 --> 00:10:36.869
secrets in the store. Yeah, I think, sorry... Let me. One

00:10:36.870 --> 00:10:40.189
limitation of GUIX is there's no way to store secrets in the

00:10:40.190 --> 00:10:42.108
store. Would SOPS be useful for this?

00:10:42.109 --> 00:10:44.829
I think so, but I don't know how

00:10:44.830 --> 00:10:48.869
that package manager works, if it's just like

00:10:48.870 --> 00:10:52.989
some sort of "you decrypt and then you run the package

00:10:52.990 --> 00:10:56.109
manager," then yeah, that's a lot of our workflows.

00:10:56.110 --> 00:10:58.989
If we're doing a deployment and the container

00:10:58.990 --> 00:11:01.629
needs it, we'll decrypt, put that in

00:11:01.630 --> 00:11:03.829
whatever place, or source it if it's an

00:11:03.830 --> 00:11:06.629
environment file for the container, and then

00:11:06.630 --> 00:11:11.982
pass it in. I think it'd be a great choice there.

NOTE Q: Wacky question: what happens in sops-mode if you encrypt the already encrypted file as if it was plaintext?

00:11:11.983 --> 00:11:17.069
A wacky question. What happens in sops mode if you

00:11:17.070 --> 00:11:21.709
encrypt an already encrypted file as if it was plain text?

00:11:21.710 --> 00:11:24.949
You know, I might have actually accidentally did that

00:11:24.950 --> 00:11:29.709
today. I didn't actually see the resulting file. But that's

00:11:29.710 --> 00:11:31.709
a great question.

00:11:31.710 --> 00:11:38.189
Well, it's technically still binary, isn't it, at the end?

00:11:38.190 --> 00:11:40.389
You've got binary stuff that is being encrypted

00:11:40.390 --> 00:11:42.949
again. It's just double encryption.

00:11:42.950 --> 00:11:44.842
I'm pretty sure it works.

00:11:44.843 --> 00:11:48.869
Yeah, probably. I'm going to go back up to the

00:11:48.870 --> 00:11:49.438
top one.

NOTE Q: can you describe some potential interactive uses for this within Emacs

00:11:49.439 --> 00:11:52.469
Can you describe some potential interactive uses

00:11:52.470 --> 00:11:57.349
for this within Emacs? Is there some other activity that

00:11:57.350 --> 00:12:01.909
would enable or it would be enabled with SOPS decryption

00:12:01.910 --> 00:12:12.529
first, like an IT configuration task.

00:12:12.530 --> 00:12:18.509
So in the README right now,

00:12:18.510 --> 00:12:22.629
there is a block and it's called SOPS setup

00:12:22.630 --> 00:12:27.687
environment. I think it's a hook. Don't quote me.

00:12:27.688 --> 00:12:29.596
I haven't touched it in a while.

00:12:29.597 --> 00:12:32.051
I think that hook runs prior to

00:12:32.052 --> 00:12:35.349
doing any sort of decryption or encryption.

00:12:35.350 --> 00:12:40.654
So there's an example in the README for ways

00:12:40.655 --> 00:12:44.669
that you can set up your SOPS mode for AWS.

00:12:44.670 --> 00:12:51.136
You can set the profile. It was actually

00:12:51.137 --> 00:12:58.829
a pretty fun thing to add because with that bit of code,

00:12:58.830 --> 00:13:01.199
I can pretty much go to any one of our repos

00:13:01.200 --> 00:13:04.085
and decrypt and encrypt on the fly and

00:13:04.086 --> 00:13:06.749
not have to do much fanfare of like,

00:13:06.750 --> 00:13:09.269
well, what account or what profile

00:13:09.270 --> 00:13:12.324
do I need to switch to? I haven't looked at

00:13:12.325 --> 00:13:15.309
GCP yet or Azure, and that's kind of one of

00:13:15.310 --> 00:13:19.079
my future things. I need to maybe look into those

00:13:19.080 --> 00:13:21.055
to see what they look like

00:13:21.056 --> 00:13:23.909
and give example configs to help users.

00:13:23.910 --> 00:13:28.993
Hopefully that answered your question.

00:13:28.994 --> 00:13:30.949
I think so.

00:13:30.950 --> 00:13:34.849
Continuing the theme of this, both of you being cursed,

00:13:34.850 --> 00:13:36.947
my X11 decided to crash.

00:13:36.948 --> 00:13:40.201
Nothing is going well with this one.

00:13:40.202 --> 00:13:44.509
Have you answered all the questions? I think so.

00:13:44.510 --> 00:13:46.438
Well, do you have anything else to add, perhaps?

00:13:46.439 --> 00:13:48.327
Maybe something that wasn't enough

00:13:48.328 --> 00:13:50.109
to fit in your live presentation?

00:13:50.110 --> 00:13:56.669
No, I'm excited to see the other talks and I hope everybody

00:13:56.670 --> 00:13:57.811
has fun too.

00:13:57.812 --> 00:14:03.303
Yeah, if you have any other questions, just email me.

00:14:03.304 --> 00:14:05.210
That's all.

00:14:05.211 --> 00:14:07.594
I got nothing.

00:14:07.595 --> 00:14:08.222
Okay, cool.

00:14:08.223 --> 00:14:10.469
[Leo]: Well, thank you so much, Jonathan, for your

00:14:10.470 --> 00:14:12.789
presentation. It was, sorry for all the technical

00:14:12.790 --> 00:14:14.162
problems, we tried our best,

00:14:14.163 --> 00:14:15.532
but I think we still managed to have

00:14:15.533 --> 00:14:17.309
a live presentation, and we managed to have some

00:14:17.310 --> 00:14:20.137
questions from the crowd. So, as far as I'm concerned,

00:14:20.138 --> 00:14:21.837
I think we did a good job.

00:14:21.838 --> 00:14:24.894
[Jonathan]: Yeah, you stomped it in this whole dev track,

00:14:24.895 --> 00:14:28.349
I just have to say. It's been a privilege to jump in

00:14:28.350 --> 00:14:31.069
with it here and there and to just listen to the great

00:14:31.070 --> 00:14:33.180
conversations.

00:14:33.181 --> 00:14:38.949
[Leo]: I think next up we have Emacs and McCLIM,

00:14:38.950 --> 00:14:41.904
which is going to be a similar format to this talk.

00:14:41.905 --> 00:14:44.709
We'll probably jump right into that in just about two minutes.

00:14:44.710 --> 00:14:47.821
We'll give you another countdown here. One second.

00:14:47.822 --> 00:14:49.916
Well, we arranged that and meanwhile,

00:14:49.917 --> 00:14:51.349
I just want to take my own

00:14:51.350 --> 00:14:55.309
little humble opportunity to thank you Jonathan, and I

00:14:55.310 --> 00:14:57.085
guess everybody else.