summaryrefslogtreecommitdiffstats
path: root/2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer.vtt
blob: 57837210efdc459ed4d836f8a30788b741f4df25 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
WEBVTT

00:00:04.080 --> 00:00:04.960
Hi.

00:00:04.960 --> 00:00:08.559
I'm Zachary Kanfer, and this is waving

00:00:08.559 --> 00:00:11.679
at repetitive repetitive repetitive

00:00:11.679 --> 00:00:15.759
music. Over quarantine, I've been bored,

00:00:15.759 --> 00:00:18.400
and I found this Android app that has a

00:00:18.400 --> 00:00:19.680
bunch of mini-games,

00:00:19.680 --> 00:00:23.519
one of which lets you compose music.

00:00:23.519 --> 00:00:26.640
And it works, but I

00:00:26.640 --> 00:00:28.160
want a little bit more functionality

00:00:28.160 --> 00:00:30.720
than it offers, it's not very flexible.

00:00:30.720 --> 00:00:33.760
So, I thought what if I made this,

00:00:33.760 --> 00:00:35.360
and what programs could I make this

00:00:35.360 --> 00:00:37.760
in that are really flexible, are really

00:00:37.760 --> 00:00:40.320
customizable.

00:00:40.320 --> 00:00:44.239
Emacs. So, I looked into it, and

00:00:44.239 --> 00:00:46.320
Emacs can play sounds, right? If you hit

00:00:46.320 --> 00:00:47.840
control g a couple of times, you'll hear

00:00:47.840 --> 00:00:49.760
like an error tone,

00:00:49.760 --> 00:00:51.360
and it turns out that, that is actually

00:00:51.360 --> 00:00:54.559
playing a WAVE file,

00:00:54.559 --> 00:00:58.960
but what's a WAVE file?

00:00:58.960 --> 00:01:01.920
Well, it turns out that WAVE is a musical

00:01:01.920 --> 00:01:03.440
file format,

00:01:03.440 --> 00:01:07.840
or really an annoying file format.

00:01:07.840 --> 00:01:12.080
So, data in it can be an unsigned integer

00:01:12.080 --> 00:01:15.360
or a signed integer, it's not consistent,

00:01:15.360 --> 00:01:17.920
and it's little-endian by default which

00:01:17.920 --> 00:01:20.320
is not the way I like to think about it.

00:01:20.320 --> 00:01:21.600
Now, you can

00:01:21.600 --> 00:01:23.733
set a WAVE file to be big-endian,

00:01:23.733 --> 00:01:25.200
but if you do that,

00:01:25.200 --> 00:01:26.600
Emacs can't play it.

00:01:26.600 --> 00:01:30.479
So, little-endian it is.

00:01:30.479 --> 00:01:33.520
There's also duplicate data fields.

00:01:33.520 --> 00:01:35.360
Here are some fields that are fine, but

00:01:35.360 --> 00:01:36.960
then there's a fourth field that's

00:01:36.960 --> 00:01:38.960
calculated based on multiplying two of

00:01:38.960 --> 00:01:40.799
the other ones together,

00:01:40.799 --> 00:01:42.399
and then there's another data field

00:01:42.399 --> 00:01:44.880
that's… you multiply those three ones

00:01:44.880 --> 00:01:48.479
together. So, it's just repetitive and

00:01:48.479 --> 00:01:50.479
unnecessary, but you have to do it, or

00:01:50.479 --> 00:01:54.159
it's not a valid WAVE file.

00:01:54.159 --> 00:01:56.479
Also, the last part of the file is

00:01:56.479 --> 00:01:58.159
described as data

00:01:58.159 --> 00:02:01.759
or as one website I found said,

00:02:01.759 --> 00:02:05.200
the actual sound data. Now,

00:02:05.200 --> 00:02:06.960
I don't know about you but when I see

00:02:06.960 --> 00:02:09.039
that, I think,

00:02:09.039 --> 00:02:12.080
what is data? It turns out

00:02:12.080 --> 00:02:15.120
that sound is just a wave, and

00:02:15.120 --> 00:02:17.760
the data is just a bunch of measurements

00:02:17.760 --> 00:02:19.120
of the height of that wave forming each

00:02:19.120 --> 00:02:20.400
sample.

00:02:20.400 --> 00:02:24.480
So, this wave starts at 8 goes 9, 11, 13,

00:02:24.480 --> 00:02:27.840
14, 15, and then back down. If you just

00:02:27.840 --> 00:02:29.520
take those measurements, those numbers,

00:02:29.520 --> 00:02:30.700
put them in a file,

00:02:30.700 --> 00:02:35.200
that's all your data is.

00:02:35.200 --> 00:02:36.560
All right. Let's go to a demo of my

00:02:36.560 --> 00:02:38.959
program.

00:02:38.959 --> 00:02:41.920
So, this is what zmusic looks like. The

00:02:41.920 --> 00:02:44.080
blue highlighted row is a single beat,

00:02:44.080 --> 00:02:48.319
there's 16 of them in this zmusic file,

00:02:48.319 --> 00:02:52.000
and each dash in the row

00:02:52.000 --> 00:02:55.040
is a single note increasing…, starting

00:02:55.040 --> 00:02:57.920
really low all the way on the left and

00:02:57.920 --> 00:02:58.959
going up

00:02:58.959 --> 00:03:01.680
as we go to the right. So, if we started

00:03:01.680 --> 00:03:02.400
playing,

00:03:02.400 --> 00:03:05.040
we won't hear anything, but we'll see the

00:03:05.040 --> 00:03:06.400
highlighted beat is the currently

00:03:06.400 --> 00:03:07.680
playing one,

00:03:07.680 --> 00:03:09.833
and we see that it loops.

00:03:09.833 --> 00:03:12.720
So, we can stop it,

00:03:12.720 --> 00:03:20.633
and now we can click to add some notes.

00:03:20.633 --> 00:03:30.879
[Music]

00:03:30.879 --> 00:03:33.120
Even more than one note at the same time

00:03:33.120 --> 00:03:37.333
works.

00:03:37.333 --> 00:03:43.920
[Music]

00:03:43.920 --> 00:03:45.567
And we can even add notes

00:03:45.567 --> 00:03:56.160
while it's playing.

00:03:56.160 --> 00:04:08.239
[Music]

00:04:08.239 --> 00:04:10.480
Okay, here are some other features that I

00:04:10.480 --> 00:04:12.959
didn't have time to demo.

00:04:12.959 --> 00:04:16.000
So, you can save the music to a file, and

00:04:16.000 --> 00:04:18.239
this is interesting because normally

00:04:18.239 --> 00:04:20.880
if there's no note in a beat, we just

00:04:20.880 --> 00:04:22.800
don't play that beat,

00:04:22.800 --> 00:04:24.639
but if you're writing to a file you have

00:04:24.639 --> 00:04:25.919
to put something in,

00:04:25.919 --> 00:04:27.840
so when it's playing it knows to not

00:04:27.840 --> 00:04:31.360
make a sound there.

00:04:31.360 --> 00:04:33.800
We can also use different scales.

00:04:33.800 --> 00:04:36.240
We're using the minor pentatonic

00:04:36.240 --> 00:04:38.080
in the demo, but you can use the major

00:04:38.080 --> 00:04:39.680
scale the minor scale

00:04:39.680 --> 00:04:40.967
or anything else.

00:04:40.967 --> 00:04:43.367
And there's also keyboard support,

00:04:43.367 --> 00:04:45.199
but it sounds really bad,

00:04:45.199 --> 00:04:48.240
and I'll explain why later.

00:04:48.240 --> 00:04:50.800
Here are some things I learned while

00:04:50.800 --> 00:04:53.520
writing zmusic.

00:04:53.520 --> 00:04:56.479
Emacs has buttons which are great as

00:04:56.479 --> 00:04:58.240
long

00:04:58.240 --> 00:04:59.919
as you don't put two of them right next

00:04:59.919 --> 00:05:02.160
to each other. So, if you do that,

00:05:02.160 --> 00:05:04.400
mousing over one of them highlights both

00:05:04.400 --> 00:05:06.160
of them.

00:05:06.160 --> 00:05:07.919
Now, that's because a button is really

00:05:07.919 --> 00:05:10.080
just a series of characters with a text

00:05:10.080 --> 00:05:11.680
property to highlight them.

00:05:11.680 --> 00:05:15.120
So, the fix is, you put another character

00:05:15.120 --> 00:05:16.479
between the two buttons,

00:05:16.479 --> 00:05:18.080
then mousing over one of them only

00:05:18.080 --> 00:05:19.759
highlights the one you want,

00:05:19.759 --> 00:05:21.600
but even this doesn't work really great

00:05:21.600 --> 00:05:22.800
for zmusic,

00:05:22.800 --> 00:05:25.440
because zmusic has a lot of very

00:05:25.440 --> 00:05:27.120
small buttons in a row.

00:05:27.120 --> 00:05:29.840
So, it's really easy to accidentally put

00:05:29.840 --> 00:05:31.600
your cursor over the space

00:05:31.600 --> 00:05:34.639
and click on that instead of the button.

00:05:34.639 --> 00:05:37.120
So, I looked into unicode, and I found

00:05:37.120 --> 00:05:38.160
this character called

00:05:38.160 --> 00:05:40.800
a zero-width space. So, we should be able

00:05:40.800 --> 00:05:42.560
to put that between buttons and not be

00:05:42.560 --> 00:05:45.120
able to accidentally click on it.

00:05:45.120 --> 00:05:46.960
Unfortunately, a zero-width space isn't

00:05:46.960 --> 00:05:50.080
actually zero width.

00:05:50.080 --> 00:05:51.680
If we put a hundred of them between two

00:05:51.680 --> 00:05:53.759
other characters, you can see there's

00:05:53.759 --> 00:05:54.880
space there,

00:05:54.880 --> 00:05:56.240
and I think what's happening is, the

00:05:56.240 --> 00:05:58.160
space is zero width

00:05:58.160 --> 00:06:01.360
but then Emacs `put` uses one pixel between

00:06:01.360 --> 00:06:02.800
each pair of characters

00:06:02.800 --> 00:06:08.080
for the cursor, so it's almost zero width.

00:06:08.080 --> 00:06:09.919
Some ways to play sound that don't quite

00:06:09.919 --> 00:06:12.960
work! `play-sound` plays music,

00:06:12.960 --> 00:06:14.800
but it blocks, you can't do things like,

00:06:14.800 --> 00:06:17.280
set other notes or even pause the music.

00:06:17.280 --> 00:06:19.600
And if you throw it into async.el, it's

00:06:19.600 --> 00:06:21.600
silent, and I don't know why.

00:06:21.600 --> 00:06:24.479
So, the solution I went with is taking

00:06:24.479 --> 00:06:25.440
that WAVE file,

00:06:25.440 --> 00:06:27.039
ran into the file system, and then

00:06:27.039 --> 00:06:29.199
shelling out to a native executable to

00:06:29.199 --> 00:06:30.639
play the sound.

00:06:30.639 --> 00:06:32.880
And that works fine as long as you only

00:06:32.880 --> 00:06:34.240
do it once,

00:06:34.240 --> 00:06:36.080
because if you do it a couple of times at

00:06:36.080 --> 00:06:37.600
the same time like if you have a chord,

00:06:37.600 --> 00:06:38.639
and you want to play three notes

00:06:38.639 --> 00:06:40.000
simultaneously,

00:06:40.000 --> 00:06:42.319
you get this weird interference, and

00:06:42.319 --> 00:06:43.680
that's actually why the keyboard from

00:06:43.680 --> 00:06:46.319
before didn't work.

00:06:46.319 --> 00:06:49.199
Also, side effects have this unexpected

00:06:49.199 --> 00:06:50.240
impact,

00:06:50.240 --> 00:06:53.039
when you saw the demo it was running

00:06:53.039 --> 00:06:55.120
pretty smoothly,

00:06:55.120 --> 00:06:57.680
but if I just add one message statement

00:06:57.680 --> 00:06:58.479
every beat for

00:06:58.479 --> 00:07:00.960
debugging purposes, I was getting lag and

00:07:00.960 --> 00:07:03.759
jitter.

00:07:03.759 --> 00:07:05.440
Here's the one thing I learned about

00:07:05.440 --> 00:07:08.319
music theory, music theory

00:07:08.319 --> 00:07:11.599
is not easy to program.

00:07:11.599 --> 00:07:12.880
I was looking around to see what

00:07:12.880 --> 00:07:15.440
concepts we can use to code the scales,

00:07:15.440 --> 00:07:16.933
to code the notes,

00:07:16.933 --> 00:07:18.160
the first thing that I saw is

00:07:18.160 --> 00:07:20.240
scale degrees,

00:07:20.240 --> 00:07:23.360
and this when I looked into it, you don't

00:07:23.360 --> 00:07:26.319
want to program in scale degrees.

00:07:26.319 --> 00:07:28.240
So, you see we have the first, second,

00:07:28.240 --> 00:07:29.759
third, fourth, fifth, sixth, seventh but

00:07:29.759 --> 00:07:32.080
then it wraps around. That octave up

00:07:32.080 --> 00:07:33.919
is also a first, and that's because both

00:07:33.919 --> 00:07:36.000
of those notes are C,

00:07:36.000 --> 00:07:38.479
so that didn't work, and also you

00:07:38.479 --> 00:07:39.759
couldn't really easily

00:07:39.759 --> 00:07:43.599
specify a flat or sharp. You could say

00:07:43.599 --> 00:07:47.680
a sharp third or

00:07:47.680 --> 00:07:50.879
you know, a flat seventh or whatever, but

00:07:50.879 --> 00:07:52.319
then you kind of have these two pieces

00:07:52.319 --> 00:07:53.759
of data that indicate the note, and I

00:07:53.759 --> 00:07:55.840
didn't love that.

00:07:55.840 --> 00:07:57.440
So, I looked again, and I

00:07:57.440 --> 00:07:59.280
found intervals,

00:07:59.280 --> 00:08:01.120
and then I thought about it, and you

00:08:01.120 --> 00:08:02.080
don't really want to program in

00:08:02.080 --> 00:08:03.440
intervals either.

00:08:03.440 --> 00:08:05.520
It fixes some of the problems with scale

00:08:05.520 --> 00:08:07.039
degrees, you see,

00:08:07.039 --> 00:08:08.319
all the way on the right you have an

00:08:08.319 --> 00:08:10.319
octave, so you wrap to 8, and you

00:08:10.319 --> 00:08:13.039
go 9, 10 and that works. But you solve the

00:08:13.039 --> 00:08:14.319
same problem, you see you have a major

00:08:14.319 --> 00:08:16.400
third but below we also have a minor

00:08:16.400 --> 00:08:18.080
third, so you saw that problem of having

00:08:18.080 --> 00:08:20.160
two pieces of information.

00:08:20.160 --> 00:08:23.360
So, I thought about it. Music is really

00:08:23.360 --> 00:08:27.120
frequencies. Like an A is 440 hertz.

00:08:27.120 --> 00:08:28.400
So, at a low level that's what we're

00:08:28.400 --> 00:08:29.360
going to do, we're just going to use

00:08:29.360 --> 00:08:30.560
frequencies.

00:08:30.560 --> 00:08:32.800
And then at the one level above that,

00:08:32.800 --> 00:08:34.159
that's a little bit easier for humans to

00:08:34.159 --> 00:08:35.360
think about,

00:08:35.360 --> 00:08:37.039
we're going to use semitones up from the

00:08:37.039 --> 00:08:39.519
root, which is kind of like scale degrees,

00:08:39.519 --> 00:08:40.959
but instead of just counting each note

00:08:40.959 --> 00:08:42.880
as one more, we're going to say how many

00:08:42.880 --> 00:08:44.560
semitones up it is.

00:08:44.560 --> 00:08:47.200
So, if there's a sharp between two notes,

00:08:47.200 --> 00:08:48.480
that's going to be two steps up instead

00:08:48.480 --> 00:08:50.000
of just one.

00:08:50.000 --> 00:08:51.279
And then we translate those two

00:08:51.279 --> 00:08:54.160
frequencies, so your A is 440 hertz,

00:08:54.160 --> 00:08:56.920
another note might be

00:08:56.920 --> 00:08:58.480
613.5,

00:08:58.480 --> 00:09:00.720
or whatever, and that's we use the low

00:09:00.720 --> 00:09:02.959
level to play.

00:09:02.959 --> 00:09:05.200
Some future work I have, I want to add

00:09:05.200 --> 00:09:06.480
some drums.

00:09:06.480 --> 00:09:08.240
I want to make that keyboard actually

00:09:08.240 --> 00:09:09.680
work,

00:09:09.680 --> 00:09:12.399
and computers and synthesizers are

00:09:12.399 --> 00:09:14.320
the only place you hear a pure sine wave

00:09:14.320 --> 00:09:16.000
like the one we have here,

00:09:16.000 --> 00:09:17.680
so I want to add overtones or other

00:09:17.680 --> 00:09:19.440
octaves above it just to make it

00:09:19.440 --> 00:09:23.120
sound a little bit more realistic.

00:09:23.120 --> 00:09:25.360
I've put notes references and the source

00:09:25.360 --> 00:09:26.000
code up

00:09:26.000 --> 00:09:30.067
at zck.org/emacsconf2020 [updated].

00:09:30.067 --> 00:09:33.040
I'm one of the organizers of EmacsNYC

00:09:33.040 --> 00:09:36.080
check that out. And if you take a look

00:09:36.080 --> 00:09:38.240
or have any thoughts, I'd love to hear

00:09:38.240 --> 00:09:39.839
them, and

00:09:39.839 --> 00:09:46.160
thanks so much for coming to my talk.