path: root/2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer.vtt

WEBVTT

00:00:04.080 --> 00:00:04.960
Hi.

00:00:04.960 --> 00:00:08.559
I'm Zachary Kanfer, and this is waving

00:00:08.559 --> 00:00:11.679
at repetitive repetitive repetitive

00:00:11.679 --> 00:00:15.759
music. Over quarantine, I've been bored,

00:00:15.759 --> 00:00:18.400
and I found this Android app that has a

00:00:18.400 --> 00:00:19.680
bunch of mini-games,

00:00:19.680 --> 00:00:23.519
one of which lets you compose music.

00:00:23.519 --> 00:00:26.640
And it works, but I

00:00:26.640 --> 00:00:28.160
want a little bit more functionality

00:00:28.160 --> 00:00:30.720
than it offers, it's not very flexible.

00:00:30.720 --> 00:00:33.760
So, I thought what if I made this,

00:00:33.760 --> 00:00:35.360
and what programs could I make this

00:00:35.360 --> 00:00:37.760
in that are really flexible, are really

00:00:37.760 --> 00:00:40.320
customizable.

00:00:40.320 --> 00:00:44.239
Emacs. So, I looked into it, and

00:00:44.239 --> 00:00:46.320
Emacs can play sounds, right? If you hit

00:00:46.320 --> 00:00:47.840
control g a couple of times, you'll hear

00:00:47.840 --> 00:00:49.760
like an error tone,

00:00:49.760 --> 00:00:51.360
and it turns out that, that is actually

00:00:51.360 --> 00:00:54.559
playing a WAVE file,

00:00:54.559 --> 00:00:58.960
but what's a WAVE file?

00:00:58.960 --> 00:01:01.920
Well, it turns out that WAVE is a musical

00:01:01.920 --> 00:01:03.440
file format,

00:01:03.440 --> 00:01:07.840
or really an annoying file format.

00:01:07.840 --> 00:01:12.080
So, data in it can be an unsigned integer

00:01:12.080 --> 00:01:15.360
or a signed integer, it's not consistent,

00:01:15.360 --> 00:01:17.920
and it's little-endian by default which

00:01:17.920 --> 00:01:20.320
is not the way I like to think about it.

00:01:20.320 --> 00:01:21.600
Now, you can

00:01:21.600 --> 00:01:23.733
set a WAVE file to be big-endian,

00:01:23.733 --> 00:01:25.200
but if you do that,

00:01:25.200 --> 00:01:26.600
Emacs can't play it.

00:01:26.600 --> 00:01:30.479
So, little-endian it is.

00:01:30.479 --> 00:01:33.520
There's also duplicate data fields.

00:01:33.520 --> 00:01:35.360
Here are some fields that are fine, but

00:01:35.360 --> 00:01:36.960
then there's a fourth field that's

00:01:36.960 --> 00:01:38.960
calculated based on multiplying two of

00:01:38.960 --> 00:01:40.799
the other ones together,

00:01:40.799 --> 00:01:42.399
and then there's another data field

00:01:42.399 --> 00:01:44.880
that's… you multiply those three ones

00:01:44.880 --> 00:01:48.479
together. So, it's just repetitive and

00:01:48.479 --> 00:01:50.479
unnecessary, but you have to do it, or

00:01:50.479 --> 00:01:54.159
it's not a valid WAVE file.

00:01:54.159 --> 00:01:56.479
Also, the last part of the file is

00:01:56.479 --> 00:01:58.159
described as data

00:01:58.159 --> 00:02:01.759
or as one website I found said,

00:02:01.759 --> 00:02:05.200
the actual sound data. Now,

00:02:05.200 --> 00:02:06.960
I don't know about you but when I see

00:02:06.960 --> 00:02:09.039
that, I think,

00:02:09.039 --> 00:02:12.080
what is data? It turns out

00:02:12.080 --> 00:02:15.120
that sound is just a wave, and

00:02:15.120 --> 00:02:17.760
the data is just a bunch of measurements

00:02:17.760 --> 00:02:19.120
of the height of that wave forming each

00:02:19.120 --> 00:02:20.400
sample.

00:02:20.400 --> 00:02:24.480
So, this wave starts at 8 goes 9, 11, 13,

00:02:24.480 --> 00:02:27.840
14, 15, and then back down. If you just

00:02:27.840 --> 00:02:29.520
take those measurements, those numbers,

00:02:29.520 --> 00:02:30.700
put them in a file,

00:02:30.700 --> 00:02:35.200
that's all your data is.

00:02:35.200 --> 00:02:36.560
All right. Let's go to a demo of my

00:02:36.560 --> 00:02:38.959
program.

00:02:38.959 --> 00:02:41.920
So, this is what zmusic looks like. The

00:02:41.920 --> 00:02:44.080
blue highlighted row is a single beat,

00:02:44.080 --> 00:02:48.319
there's 16 of them in this zmusic file,

00:02:48.319 --> 00:02:52.000
and each dash in the row

00:02:52.000 --> 00:02:55.040
is a single note increasing…, starting

00:02:55.040 --> 00:02:57.920
really low all the way on the left and

00:02:57.920 --> 00:02:58.959
going up

00:02:58.959 --> 00:03:01.680
as we go to the right. So, if we started

00:03:01.680 --> 00:03:02.400
playing,

00:03:02.400 --> 00:03:05.040
we won't hear anything, but we'll see the

00:03:05.040 --> 00:03:06.400
highlighted beat is the currently

00:03:06.400 --> 00:03:07.680
playing one,

00:03:07.680 --> 00:03:09.833
and we see that it loops.

00:03:09.833 --> 00:03:12.720
So, we can stop it,

00:03:12.720 --> 00:03:20.633
and now we can click to add some notes.

00:03:20.633 --> 00:03:30.879
[Music]

00:03:30.879 --> 00:03:33.120
Even more than one note at the same time

00:03:33.120 --> 00:03:37.333
works.

00:03:37.333 --> 00:03:43.920
[Music]

00:03:43.920 --> 00:03:45.567
And we can even add notes

00:03:45.567 --> 00:03:56.160
while it's playing.

00:03:56.160 --> 00:04:08.239
[Music]

00:04:08.239 --> 00:04:10.480
Okay, here are some other features that I

00:04:10.480 --> 00:04:12.959
didn't have time to demo.

00:04:12.959 --> 00:04:16.000
So, you can save the music to a file, and

00:04:16.000 --> 00:04:18.239
this is interesting because normally

00:04:18.239 --> 00:04:20.880
if there's no note in a beat, we just

00:04:20.880 --> 00:04:22.800
don't play that beat,

00:04:22.800 --> 00:04:24.639
but if you're writing to a file you have

00:04:24.639 --> 00:04:25.919
to put something in,

00:04:25.919 --> 00:04:27.840
so when it's playing it knows to not

00:04:27.840 --> 00:04:31.360
make a sound there.

00:04:31.360 --> 00:04:33.800
We can also use different scales.

00:04:33.800 --> 00:04:36.240
We're using the minor pentatonic

00:04:36.240 --> 00:04:38.080
in the demo, but you can use the major

00:04:38.080 --> 00:04:39.680
scale the minor scale

00:04:39.680 --> 00:04:40.967
or anything else.

00:04:40.967 --> 00:04:43.367
And there's also keyboard support,

00:04:43.367 --> 00:04:45.199
but it sounds really bad,

00:04:45.199 --> 00:04:48.240
and I'll explain why later.

00:04:48.240 --> 00:04:50.800
Here are some things I learned while

00:04:50.800 --> 00:04:53.520
writing zmusic.

00:04:53.520 --> 00:04:56.479
Emacs has buttons which are great as

00:04:56.479 --> 00:04:58.240
long

00:04:58.240 --> 00:04:59.919
as you don't put two of them right next

00:04:59.919 --> 00:05:02.160
to each other. So, if you do that,

00:05:02.160 --> 00:05:04.400
mousing over one of them highlights both

00:05:04.400 --> 00:05:06.160
of them.

00:05:06.160 --> 00:05:07.919
Now, that's because a button is really

00:05:07.919 --> 00:05:10.080
just a series of characters with a text

00:05:10.080 --> 00:05:11.680
property to highlight them.

00:05:11.680 --> 00:05:15.120
So, the fix is, you put another character

00:05:15.120 --> 00:05:16.479
between the two buttons,

00:05:16.479 --> 00:05:18.080
then mousing over one of them only

00:05:18.080 --> 00:05:19.759
highlights the one you want,

00:05:19.759 --> 00:05:21.600
but even this doesn't work really great

00:05:21.600 --> 00:05:22.800
for zmusic,

00:05:22.800 --> 00:05:25.440
because zmusic has a lot of very

00:05:25.440 --> 00:05:27.120
small buttons in a row.

00:05:27.120 --> 00:05:29.840
So, it's really easy to accidentally put

00:05:29.840 --> 00:05:31.600
your cursor over the space

00:05:31.600 --> 00:05:34.639
and click on that instead of the button.

00:05:34.639 --> 00:05:37.120
So, I looked into unicode, and I found

00:05:37.120 --> 00:05:38.160
this character called

00:05:38.160 --> 00:05:40.800
a zero-width space. So, we should be able

00:05:40.800 --> 00:05:42.560
to put that between buttons and not be

00:05:42.560 --> 00:05:45.120
able to accidentally click on it.

00:05:45.120 --> 00:05:46.960
Unfortunately, a zero-width space isn't

00:05:46.960 --> 00:05:50.080
actually zero width.

00:05:50.080 --> 00:05:51.680
If we put a hundred of them between two

00:05:51.680 --> 00:05:53.759
other characters, you can see there's

00:05:53.759 --> 00:05:54.880
space there,

00:05:54.880 --> 00:05:56.240
and I think what's happening is, the

00:05:56.240 --> 00:05:58.160
space is zero width

00:05:58.160 --> 00:06:01.360
but then Emacs `put` uses one pixel between

00:06:01.360 --> 00:06:02.800
each pair of characters

00:06:02.800 --> 00:06:08.080
for the cursor, so it's almost zero width.

00:06:08.080 --> 00:06:09.919
Some ways to play sound that don't quite

00:06:09.919 --> 00:06:12.960
work! `play-sound` plays music,

00:06:12.960 --> 00:06:14.800
but it blocks, you can't do things like,

00:06:14.800 --> 00:06:17.280
set other notes or even pause the music.

00:06:17.280 --> 00:06:19.600
And if you throw it into async.el, it's

00:06:19.600 --> 00:06:21.600
silent, and I don't know why.

00:06:21.600 --> 00:06:24.479
So, the solution I went with is taking

00:06:24.479 --> 00:06:25.440
that WAVE file,

00:06:25.440 --> 00:06:27.039
ran into the file system, and then

00:06:27.039 --> 00:06:29.199
shelling out to a native executable to

00:06:29.199 --> 00:06:30.639
play the sound.

00:06:30.639 --> 00:06:32.880
And that works fine as long as you only

00:06:32.880 --> 00:06:34.240
do it once,

00:06:34.240 --> 00:06:36.080
because if you do it a couple of times at

00:06:36.080 --> 00:06:37.600
the same time like if you have a chord,

00:06:37.600 --> 00:06:38.639
and you want to play three notes

00:06:38.639 --> 00:06:40.000
simultaneously,

00:06:40.000 --> 00:06:42.319
you get this weird interference, and

00:06:42.319 --> 00:06:43.680
that's actually why the keyboard from

00:06:43.680 --> 00:06:46.319
before didn't work.

00:06:46.319 --> 00:06:49.199
Also, side effects have this unexpected

00:06:49.199 --> 00:06:50.240
impact,

00:06:50.240 --> 00:06:53.039
when you saw the demo it was running

00:06:53.039 --> 00:06:55.120
pretty smoothly,

00:06:55.120 --> 00:06:57.680
but if I just add one message statement

00:06:57.680 --> 00:06:58.479
every beat for

00:06:58.479 --> 00:07:00.960
debugging purposes, I was getting lag and

00:07:00.960 --> 00:07:03.759
jitter.

00:07:03.759 --> 00:07:05.440
Here's the one thing I learned about

00:07:05.440 --> 00:07:08.319
music theory, music theory

00:07:08.319 --> 00:07:11.599
is not easy to program.

00:07:11.599 --> 00:07:12.880
I was looking around to see what

00:07:12.880 --> 00:07:15.440
concepts we can use to code the scales,

00:07:15.440 --> 00:07:16.933
to code the notes,

00:07:16.933 --> 00:07:18.160
the first thing that I saw is

00:07:18.160 --> 00:07:20.240
scale degrees,

00:07:20.240 --> 00:07:23.360
and this when I looked into it, you don't

00:07:23.360 --> 00:07:26.319
want to program in scale degrees.

00:07:26.319 --> 00:07:28.240
So, you see we have the first, second,

00:07:28.240 --> 00:07:29.759
third, fourth, fifth, sixth, seventh but

00:07:29.759 --> 00:07:32.080
then it wraps around. That octave up

00:07:32.080 --> 00:07:33.919
is also a first, and that's because both

00:07:33.919 --> 00:07:36.000
of those notes are C,

00:07:36.000 --> 00:07:38.479
so that didn't work, and also you

00:07:38.479 --> 00:07:39.759
couldn't really easily

00:07:39.759 --> 00:07:43.599
specify a flat or sharp. You could say

00:07:43.599 --> 00:07:47.680
a sharp third or

00:07:47.680 --> 00:07:50.879
you know, a flat seventh or whatever, but

00:07:50.879 --> 00:07:52.319
then you kind of have these two pieces

00:07:52.319 --> 00:07:53.759
of data that indicate the note, and I

00:07:53.759 --> 00:07:55.840
didn't love that.

00:07:55.840 --> 00:07:57.440
So, I looked again, and I

00:07:57.440 --> 00:07:59.280
found intervals,

00:07:59.280 --> 00:08:01.120
and then I thought about it, and you

00:08:01.120 --> 00:08:02.080
don't really want to program in

00:08:02.080 --> 00:08:03.440
intervals either.

00:08:03.440 --> 00:08:05.520
It fixes some of the problems with scale

00:08:05.520 --> 00:08:07.039
degrees, you see,

00:08:07.039 --> 00:08:08.319
all the way on the right you have an

00:08:08.319 --> 00:08:10.319
octave, so you wrap to 8, and you

00:08:10.319 --> 00:08:13.039
go 9, 10 and that works. But you solve the

00:08:13.039 --> 00:08:14.319
same problem, you see you have a major

00:08:14.319 --> 00:08:16.400
third but below we also have a minor

00:08:16.400 --> 00:08:18.080
third, so you saw that problem of having

00:08:18.080 --> 00:08:20.160
two pieces of information.

00:08:20.160 --> 00:08:23.360
So, I thought about it. Music is really

00:08:23.360 --> 00:08:27.120
frequencies. Like an A is 440 hertz.

00:08:27.120 --> 00:08:28.400
So, at a low level that's what we're

00:08:28.400 --> 00:08:29.360
going to do, we're just going to use

00:08:29.360 --> 00:08:30.560
frequencies.

00:08:30.560 --> 00:08:32.800
And then at the one level above that,

00:08:32.800 --> 00:08:34.159
that's a little bit easier for humans to

00:08:34.159 --> 00:08:35.360
think about,

00:08:35.360 --> 00:08:37.039
we're going to use semitones up from the

00:08:37.039 --> 00:08:39.519
root, which is kind of like scale degrees,

00:08:39.519 --> 00:08:40.959
but instead of just counting each note

00:08:40.959 --> 00:08:42.880
as one more, we're going to say how many

00:08:42.880 --> 00:08:44.560
semitones up it is.

00:08:44.560 --> 00:08:47.200
So, if there's a sharp between two notes,

00:08:47.200 --> 00:08:48.480
that's going to be two steps up instead

00:08:48.480 --> 00:08:50.000
of just one.

00:08:50.000 --> 00:08:51.279
And then we translate those two

00:08:51.279 --> 00:08:54.160
frequencies, so your A is 440 hertz,

00:08:54.160 --> 00:08:56.920
another note might be

00:08:56.920 --> 00:08:58.480
613.5,

00:08:58.480 --> 00:09:00.720
or whatever, and that's we use the low

00:09:00.720 --> 00:09:02.959
level to play.

00:09:02.959 --> 00:09:05.200
Some future work I have, I want to add

00:09:05.200 --> 00:09:06.480
some drums.

00:09:06.480 --> 00:09:08.240
I want to make that keyboard actually

00:09:08.240 --> 00:09:09.680
work,

00:09:09.680 --> 00:09:12.399
and computers and synthesizers are

00:09:12.399 --> 00:09:14.320
the only place you hear a pure sine wave

00:09:14.320 --> 00:09:16.000
like the one we have here,

00:09:16.000 --> 00:09:17.680
so I want to add overtones or other

00:09:17.680 --> 00:09:19.440
octaves above it just to make it

00:09:19.440 --> 00:09:23.120
sound a little bit more realistic.

00:09:23.120 --> 00:09:25.360
I've put notes references and the source

00:09:25.360 --> 00:09:26.000
code up

00:09:26.000 --> 00:09:30.067
at zck.org/emacsconf2020 [updated].

00:09:30.067 --> 00:09:33.040
I'm one of the organizers of EmacsNYC

00:09:33.040 --> 00:09:36.080
check that out. And if you take a look

00:09:36.080 --> 00:09:38.240
or have any thoughts, I'd love to hear

00:09:38.240 --> 00:09:39.839
them, and

00:09:39.839 --> 00:09:46.160
thanks so much for coming to my talk.