path: root/2020/info/35.md

# WAVEing at Repetitive Repetitive Repetitive Music
Zachary Kanfer

[[!template id=vid src="https://mirror.csclub.uwaterloo.ca/emacsconf/2020/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer.webm" size="122M" subtitles="/2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer.vtt" duration="9:44"]] 
[Download compressed .webm video (12.7M)](https://mirror.csclub.uwaterloo.ca/emacsconf/2020/smaller/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer--vp9-q56-video-original-audio.webm)
[View transcript](#transcript)

[[!template id=vid src="https://mirror.csclub.uwaterloo.ca/emacsconf/2020/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--questions--zachary-kanfer.webm" download="Download Q&A video" size="89M" subtitles="/2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--questions--zachary-kanfer.vtt" duration="6:18"]]  
[Download compressed Q&A .webm video (6.9M)](https://mirror.csclub.uwaterloo.ca/emacsconf/2020/smaller/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--questions--zachary-kanfer--vp9-q56-video-original-audio.webm)
[View transcript for Q&A](#transcript-questions)

During quarantine, I found myself spending time with an Android app.
One of the features this app has is composing music that loops
endlessly.  As with many things, I wondered how much better this tool
would be, if only it was inside Emacs.

This talk will explain how I made this tool inside Emacs, with detours
through Emacs text properties, font rendering, the .WAVE file format,
and music theory.  And hopefully at the end, we'll have something
worth listening to.

There are extended notes, references, and links at
<https://zck.org/emacsconf2020>.

The source can be found at <https://hg.sr.ht/~zck/zmusic/>.

<!-- from the pad --->

- Actual start and end time (EST): Start 2020-11-29T16:29; Stop 2020-11-29T16:46

# Questions

## Q9: What were some of the challenges with writing a special-mode for Emacs? I'm interested in getting into this in the future, but I'm not really sure where to start.
I used define-derived-mode
(<https://www.gnu.org/software/emacs/manual/html_node/elisp/Derived-Modes.html>)
to make this mode. It's really useful! For more information, I
recorded a talk about making major modes
(<https://www.youtube.com/watch?v=gk39mp8Vy4M>) a few years ago, at an
EmacsNYC (<https://emacsnyc.org/>) meeting.

- That'd be awesome, thanks! Will do.

## Q8: Any MIDI mapping possibilities?
Should be! Would just need to change the low-level.

## Q7: Do you think would be possible to add a set of recorded sounds in order to use those?
Yes! Part of zmusic is tooling to make wave files, so it should be
possible to slice-and-dice input data, and output valid wave files.

## Q6: Have you written any actual songs (in RRRM/WAVEing)? Can you play one?
Nothing super or well put together as of now.

## Q5: Are there any open source musical instrument sample libraries that could be used? E.g. "play A 440 on Piano sample 1" to provide better quality notes than built-in tones
Experimentation would be fun. However, the nice part about Emacs is
that it doesn't have any external dependencies, you only need a way to
play WAVes.

## Q4: What is your musical background? Do you play any instruments?
Random instruments, started with recorder, played cello for a long
time, now playing guitar.

## Q3: Any chance for an Emacs tracker/mod player? (plays several samples arranged in the same top-down fashion with effects applied to them for chiptune and keygen music)
I don't really know what a tracker/mod player is.

## Q2: Will you play us another song? (RIP ears — who needs 'em, this is awesome! it is!)
UPDATE: can confirm, it was easy to play a song myself :-)  Very nice!

However git clone <https://hg.sr.ht/~zck/zmusic> didn't work, I wonder if I'm doing it wrong

- Had to browse to <https://hg.sr.ht/~zck/zmusic/browse/zmusic.el> and copy/paste.
- It's mercurial! (I have Opinions about version control systems). Try
  `hg clone` instead, or copy/paste from the link directly.
- BAM! hg clone works fine.
  - Hooray!

## Q1: Why do you go top-to-bottom for time progression and left-to-right for low-to-high in stead of doing it pivoted? (e.g. higher is higher tone, left-to-right is time progression). This is awesome by the way!
The initial app (the inspiration) worked this way. It is definitely
something worth looking into.

# Notes
Notes, references, and links at <https://zck.org/emacsconf2020>


<!-- transcript: 2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--zachary-kanfer.vtt -->

<a name="transcript"></a>
# Transcript

(00:04) Hi. I'm Zachary Kanfer, and this is waving at repetitive
repetitive repetitive music. Over quarantine, I've been bored, and I
found this Android app that has a bunch of mini-games, one of which
lets you compose music. And it works, but I want a little bit more
functionality than it offers, it's not very flexible. So, I thought
what if I made this, and what programs could I make this in that are
really flexible, are really customizable. Emacs. So, I looked into
it, and Emacs can play sounds, right? If you hit control g a couple of
times, you'll hear like an error tone, and it turns out that, that is
actually playing a WAVE file, but what's a WAVE file?

(00:58) Well, it turns out that WAVE is a musical file format, or
really an annoying file format. So, data in it can be an unsigned
integer or a signed integer, it's not consistent, and it's
little-endian by default which is not the way I like to think about
it. Now, you can set a WAVE file to be big-endian, but if you do
that, Emacs can't play it. So, little-endian it is. There's also
duplicate data fields. Here are some fields that are fine, but then
there's a fourth field that's calculated based on multiplying two of
the other ones together, and then there's another data field that's…
you multiply those three ones together. So, it's just repetitive and
unnecessary, but you have to do it, or it's not a valid WAVE file.
Also, the last part of the file is described as data or as one website
I found said, the actual sound data.

(02:04) Now, I don't know about you but when I see that, I think, what
is data? It turns out that sound is just a wave, and the data is just
a bunch of measurements of the height of that wave forming each
sample. So, this wave starts at 8 goes 9, 11, 13, 14, 15, and then
back down. If you just take those measurements, those numbers, put
them in a file, that's all your data is.

(02:35) All right. Let's go to a demo of my program. So, this is what
zmusic looks like. The blue highlighted row is a single beat, there's
16 of them in this zmusic file, and each dash in the row is a single
note increasing…, starting really low all the way on the left and
going up as we go to the right. So, if we started playing, we won't
hear anything, but we'll see the highlighted beat is the currently
playing one, and we see that it loops. So, we can stop it, and now we
can click to add some notes. [Music] Even more than one note at the
same time works. [Music] And we can even add notes while it's
playing. [Music]

(04:08) Okay, here are some other features that I didn't have time to
demo. So, you can save the music to a file, and this is interesting
because normally if there's no note in a beat, we just don't play that
beat, but if you're writing to a file you have to put something in, so
when it's playing it knows to not make a sound there. We can also use
different scales. We're using the minor pentatonic in the demo, but
you can use the major scale the minor scale or anything else. And
there's also keyboard support, but it sounds really bad, and I'll
explain why later.

(04:48) Here are some things I learned while writing zmusic. Emacs
has buttons which are great as long as you don't put two of them right
next to each other. So, if you do that, mousing over one of them
highlights both of them. Now, that's because a button is really just
a series of characters with a text property to highlight them. So,
the fix is, you put another character between the two buttons, then
mousing over one of them only highlights the one you want, but even
this doesn't work really great for zmusic, because zmusic has a lot of
very small buttons in a row. So, it's really easy to accidentally put
your cursor over the space and click on that instead of the button.
So, I looked into unicode, and I found this character called a
zero-width space. So, we should be able to put that between buttons
and not be able to accidentally click on it. Unfortunately, a
zero-width space isn't actually zero width. If we put a hundred of
them between two other characters, you can see there's space there,
and I think what's happening is, the space is zero width but then
Emacs `put` uses one pixel between each pair of characters for the
cursor, so it's almost zero width. Some ways to play sound that don't
quite work! `play-sound` plays music, but it blocks, you can't do
things like, set other notes or even pause the music. And if you
throw it into async.el, it's silent, and I don't know why. So, the
solution I went with is taking that WAVE file, ran into the file
system, and then shelling out to a native executable to play the
sound. And that works fine as long as you only do it once, because if
you do it a couple of times at the same time like if you have a chord,
and you want to play three notes simultaneously, you get this weird
interference, and that's actually why the keyboard from before didn't
work. Also, side effects have this unexpected impact, when you saw
the demo it was running pretty smoothly, but if I just add one message
statement every beat for debugging purposes, I was getting lag and
jitter.

(07:03) Here's the one thing I learned about music theory, music
theory is not easy to program. I was looking around to see what
concepts we can use to code the scales, to code the notes, the first
thing that I saw is scale degrees, and this when I looked into it, you
don't want to program in scale degrees. So, you see we have the
first, second, third, fourth, fifth, sixth, seventh but then it wraps
around. That octave up is also a first, and that's because both of
those notes are C, so that didn't work, and also you couldn't really
easily specify a flat or sharp. You could say a sharp third or you
know, a flat seventh or whatever, but then you kind of have these two
pieces of data that indicate the note, and I didn't love that.

(07:55) So, I looked again, and I found intervals, and then I thought
about it, and you don't really want to program in intervals either.
It fixes some of the problems with scale degrees, you see, all the way
on the right you have an octave, so you wrap to 8, and you go 9, 10
and that works. But you solve the same problem, you see you have a
major third but below we also have a minor third, so you saw that
problem of having two pieces of information.

(08:20) So, I thought about it. Music is really frequencies. Like an A
is 440 hertz. So, at a low level that's what we're going to do, we're
just going to use frequencies. And then at the one level above that,
that's a little bit easier for humans to think about, we're going to
use semitones up from the root, which is kind of like scale degrees,
but instead of just counting each note as one more, we're going to say
how many semitones up it is. So, if there's a sharp between two
notes, that's going to be two steps up instead of just one. And then
we translate those two frequencies, so your A is 440 hertz, another
note might be 613.5, or whatever, and that's we use the low level to
play.

(09:02) Some future work I have, I want to add some drums. I want to
make that keyboard actually work, and computers and synthesizers are
the only place you hear a pure sine wave like the one we have here, so
I want to add overtones or other octaves above it just to make it
sound a little bit more realistic. I've put notes references and the
source code up at <https://zck.org/emacsconf2020> [updated]. I'm one
of the organizers of EmacsNYC check that out. And if you take a look
or have any thoughts, I'd love to hear them, and thanks so much for
coming to my talk.

<!-- /transcript -->


<!-- transcript: 2020/subtitles/emacsconf-2020--35-waveing-at-repetitive-repetitive-repetitive-music-zmusic--questions--zachary-kanfer.vtt -->

<a name="transcript-questions"></a>
# Transcript: Q&A

(00:09) Okay, yeah, so I'm Zachary Kanfer, let's go to the
questions. The first question, "Why do we go top to bottom for time
progression, and left to right for low to high?" Interesting, I
think…, so the initial thing I was copying, that initial app work this
way, and yeah, I mean, certainly traditional music, you know, on a
staff does go left to right like this. I mean, going top to bottom
does make it easier to add more beats without having to wrap, but
certainly that could be managed. Yeah, I had not really thought about
it, but it is definitely something worth looking into.

(01:21) Two, "Will you play us another song?" Not now, I can make some
recordings of it, or certainly you can try it. I couldn't quite get
the microphone and the webcam and everything to work with the sound
playing now. So, I can record some. Also, I have put a link in the
Etherpad, <https://zck.org/emacsconf2020> [updated], where you can go
and get the source, and you can try it yourself. There's no
dependencies needed, so it's just all in Emacs. So, please, you know,
try it yourself.

(01:57) "Any chance for an Emacs tracker or mod player?" I don't
really know what a mod player or tracker are, but I mean, I'm sure
that would be cool, maybe there's one on now, but I don't know.

(02:11) My musical background. So, I've played various instruments
since about the third grade. Started recorder, play cello, I play
guitar now. But yeah, so just kind of random instruments, and I guess
kind of some of those things influence how I think about music.

(02:33) "Are there any open source musical management sample libraries
that could be used?" Good question, I'm sure there are, I don't know
any of that integrate really well with Emacs. One of the cool things
that I liked about this is that there are no dependencies, you know,
you don't need any external program to generate the music. I mean, it
does shell out to to play, but that should be able to be done on any
operating system, as always, you have something that can play WAVE
files, but yeah, it is interesting to kind of try the different sounds
and different tones that you could get with different instruments.

(03:14) Have I written any actual songs? Nothing super well put
together, I kind of just been playing around with this. It's kind of…
making this was one of those things where once I made it, I was like,
okay, now I can play with it, and I did a little bit, and was like, I
don't know if I feel like it right now. You know, which I've found
that to be the case with some things that I've implemented in Emacs
where it's… I make it, and then it's the kind of some of the desire to
use it all the time goes away, but I'm sure I'll circle back around at
some point especially kind of maybe once I add in different tones or
something.

(03:54) I guess a similar question for pre-recorded sounds. Yeah, I
mean, if it's… part of what I did, what I wrote was a WAVE generation
library, so, if you kind of have the data, you could use those and
chop them up and take certain lengths of them and make a WAVE file, so
it's not plug and play right now, but you could certainly add those
notes to do it.

(04:19) "Any MIDI mapping possibilities?" I haven't looked into it,
but I'm sure you definitely could output to MIDI which is another
benefit of having that multiple layers with the top layer is just, you
know, if the root note is this, we're just two semitones up or seven
semitones up or whatever it is. It should be relatively simple to
kind of switch out that layer underneath from WAVE to MIDI or other
things.

(04:44) "What were some of the challenges with writing a special mode
for Emacs?" Interested in getting into this, not sure where to start.
This isn't the first mode I've written, so that's right…, certainly
that helps. I actually… I have a video that we recorded it as part of
EmacsNYC on making a major mode. That's basically starts from
nothing, and kind of builds up to an implementation of tic-tac-toe,
but so it kind of goes into printing things out and buttons and making
the mode. I mean, one of the best parts about Emacs is, because it's
so configurable and so introspectible, you can start pretty simply,
and just kind of ask Emacs about things, and then make one little
change. It's really… it's not that bad, so, I'll try to throw a link
up on that page I put up, or please email me for whoever asked this
question to get a link to that video, or just look at the source code
of this or any other major mode. Emacs makes it pretty easy to extend
major modes.

(05:54) And I think that's the last question in the Etherpad, so,
thanks so much everybody for coming. (Amin: Thank you so much to
Zachary for your awesome talk, and for doing live questions. Thank
you.) Thank you. (Amin: Cheers.)

<!-- /transcript -->