path: root/2022/info/async-after.md

<!-- Automatically generated by emacsconf-publish-after-page -->


<a name="async-mainVideo-transcript"></a>
# Transcript

[[!template new="1" text="""Hey everyone. I'm Michael,""" start="00:00:00.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""and I'm going to be talking to you today""" start="00:00:02.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""about asynchronous programming in Emacs Lisp.""" start="00:00:04.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""I'm located in the San Francisco Bay Area,""" start="00:00:07.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""where I'm a developer as well as""" start="00:00:10.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""a long time Emacs user.""" start="00:00:11.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""You may have heard of async""" start="00:00:14.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""or asynchronous programming.""" start="00:00:15.981" video="mainVideo-async" id="subtitle"]]
[[!template text="""The idea has been around for decades,""" start="00:00:18.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""but it first gained widespread attention""" start="00:00:20.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""in JavaScript back in the aughts.""" start="00:00:23.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""And in the teens it gained tremendous popularity""" start="00:00:26.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""in the DevOps world with Go lang.""" start="00:00:29.521" video="mainVideo-async" id="subtitle"]]
[[!template text="""And just in the last few years,""" start="00:00:31.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""support for async programming has landed in Rust.""" start="00:00:33.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""Well, it can be done in Emacs as well,""" start="00:00:37.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""and this talk will demonstrate that""" start="00:00:40.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""by walking you through a little problem""" start="00:00:42.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""that I actually solved for myself.""" start="00:00:44.600" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""Like a lot of these stories,""" start="00:00:47.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""it begins with scratching a personal itch.""" start="00:00:49.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""In my case, automating my music server.""" start="00:00:51.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""I use something called the Music Player Daemon""" start="00:00:55.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""locally, and as the name suggests,""" start="00:00:57.537" video="mainVideo-async" id="subtitle"]]
[[!template text="""it just kind of hangs out in the background.""" start="00:01:00.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""Reads music files,""" start="00:01:03.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""and talks to assorted sound drivers.""" start="00:01:04.771" video="mainVideo-async" id="subtitle"]]
[[!template text="""In fact, it is so focused on""" start="00:01:08.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""that mission that it doesn't even offer""" start="00:01:09.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""a user interface.""" start="00:01:11.354" video="mainVideo-async" id="subtitle"]]
[[!template text="""Instead, it serves an API""" start="00:01:12.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""and invites application developers""" start="00:01:14.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""to build clients on top of that API.""" start="00:01:16.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, so let's hop into a vterm,""" start="00:01:19.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""and I'd like to show you the MPD client I use""" start="00:01:22.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""for my daily driver, something called NCMP CPP.""" start="00:01:25.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""Doesn't exactly roll off the tongue,""" start="00:01:29.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""but I've got a playlist,""" start="00:01:31.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""I can browse the file system,""" start="00:01:33.461" video="mainVideo-async" id="subtitle"]]
[[!template text="""looks like I can search my music library,""" start="00:01:36.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""yada yada yada.""" start="00:01:39.344" video="mainVideo-async" id="subtitle"]]
[[!template text="""It's got all the basic features.""" start="00:01:40.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""The point that I want to make is that""" start="00:01:42.560" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""NCMP CPP is a completely independent project""" start="00:01:44.854" video="mainVideo-async" id="subtitle"]]
[[!template text="""of MPD, separate and distinct.""" start="00:01:50.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""It does all of its work by simply communicating""" start="00:01:53.680" video="mainVideo-async" id="subtitle"]]
[[!template text="""with the Music Player Demon over the API.""" start="00:01:57.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""Well, I wanted to program to that API""" start="00:02:01.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""only from within Emacs.""" start="00:02:03.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""Now, there are already Emacs MPD clients out there,""" start="00:02:05.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""but I didn't really want a full-blown client.""" start="00:02:09.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""I just wanted a few small tweaks""" start="00:02:11.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""over my current configuration.""" start="00:02:14.137" video="mainVideo-async" id="subtitle"]]
[[!template text="""A command to skip to the next song.""" start="00:02:16.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""Maybe shove the current track into the mode line.""" start="00:02:19.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""Things like this.""" start="00:02:22.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""I needed an Elisp API that would let me do this.""" start="00:02:24.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, well, let's get out of ncmpcpp,""" start="00:02:28.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""and let's get into a netcat session""" start="00:02:32.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""with my local MPD server.""" start="00:02:37.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""As you can see, we get a welcome string.""" start="00:02:39.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, it is a server goes first protocol.""" start="00:02:43.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""But after that it's a very familiar""" start="00:02:46.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""text based request response oriented protocol.""" start="00:02:49.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""I can ask for the volume.""" start="00:02:53.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""I can ask for the status.""" start="00:02:56.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""But in particular I wanted an asynchronous API.""" start="00:02:58.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""If I issue a command like""" start="00:03:06.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""find every track in my library,""" start="00:03:07.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""that's going to produce a lot of data.""" start="00:03:11.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""That's a human perceptible pause""" start="00:03:15.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""as Emacs processes all the input.""" start="00:03:18.920" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""What I wanted was a style of programming""" start="00:03:22.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""where I could fire off my command,""" start="00:03:25.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""have the Emacs command loop keep working,""" start="00:03:28.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""and only invoke some callback""" start="00:03:31.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""when there was data available.""" start="00:03:33.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""Well, Emacs is famously single threaded,""" start="00:03:35.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""so it shouldn't come as a surprise that it offers""" start="00:03:38.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""a rich set of primitives that enable the sort of""" start="00:03:42.221" video="mainVideo-async" id="subtitle"]]
[[!template text="""network programming that I wanted to do.""" start="00:03:45.654" video="mainVideo-async" id="subtitle"]]
[[!template text="""In particular, it offers""" start="00:03:48.271" video="mainVideo-async" id="subtitle"]]
[[!template text="""a function called `make-network-process`.""" start="00:03:50.671" video="mainVideo-async" id="subtitle"]]
[[!template text="""Now this method offers""" start="00:03:53.404" video="mainVideo-async" id="subtitle"]]
[[!template text="""a bewildering variety of options,""" start="00:03:54.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""but at the heart of the matter""" start="00:03:57.721" video="mainVideo-async" id="subtitle"]]
[[!template text="""it opens a network connection""" start="00:03:59.487" video="mainVideo-async" id="subtitle"]]
[[!template text="""to some endpoint out there,""" start="00:04:01.054" video="mainVideo-async" id="subtitle"]]
[[!template text="""and we can configure it to be non-blocking.""" start="00:04:02.687" video="mainVideo-async" id="subtitle"]]
[[!template text="""It returns a handle that you can use to refer""" start="00:04:06.104" video="mainVideo-async" id="subtitle"]]
[[!template text="""to this network connection with other methods.""" start="00:04:10.421" video="mainVideo-async" id="subtitle"]]
[[!template text="""Other methods such as `process-send-string`,""" start="00:04:13.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""which as the name suggests,""" start="00:04:17.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""allows you to send textual data""" start="00:04:19.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""to the remote endpoint of your network connection.""" start="00:04:21.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""You can also use it with `set-process-filter`,""" start="00:04:26.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""which allows you to associate a callback""" start="00:04:29.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""with your network connection.""" start="00:04:32.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""That callback will be invoked""" start="00:04:33.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""when there is data available""" start="00:04:35.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""in the process's read buffer.""" start="00:04:37.804" video="mainVideo-async" id="subtitle"]]
[[!template text="""In other words,""" start="00:04:40.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""in a request response oriented protocol""" start="00:04:41.254" video="mainVideo-async" id="subtitle"]]
[[!template text="""like that of MPD, you open your socket""" start="00:04:44.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""with `make-network-process`,""" start="00:04:47.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""send your request via `process-send-string`""" start="00:04:50.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""and life will just continue in Emacs""" start="00:04:53.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""until some data shows up""" start="00:04:56.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""in the process's read buffer,""" start="00:04:57.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""at which point your callback will be invoked.""" start="00:05:00.720" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""It turns out this was enough""" start="00:05:05.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""for a purpose built async runtime.""" start="00:05:07.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""Let's work through the sequence of events""" start="00:05:12.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""when opening a connection""" start="00:05:14.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""and firing off a few commands in this style.""" start="00:05:16.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, let's imagine a library""" start="00:05:18.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""that offers a connection object of some sort,""" start="00:05:22.887" video="mainVideo-async" id="subtitle"]]
[[!template text="""a caller, and an MPD server out on the network.""" start="00:05:25.767" video="mainVideo-async" id="subtitle"]]
[[!template text="""The caller will presumably get themselves""" start="00:05:29.487" video="mainVideo-async" id="subtitle"]]
[[!template text="""a connection object by invoking some sort""" start="00:05:31.807" video="mainVideo-async" id="subtitle"]]
[[!template text="""of connect method on our library.""" start="00:05:34.954" video="mainVideo-async" id="subtitle"]]
[[!template text="""We can handle this through `make-network-process`,""" start="00:05:38.327" video="mainVideo-async" id="subtitle"]]
[[!template text="""but we're going to invoke `make-network-process`""" start="00:05:41.047" video="mainVideo-async" id="subtitle"]]
[[!template text="""with nowait equal to true,""" start="00:05:45.487" video="mainVideo-async" id="subtitle"]]
[[!template text="""in other words asynchronously.""" start="00:05:47.167" video="mainVideo-async" id="subtitle"]]
[[!template text="""That means the method is going to return immediately.""" start="00:05:50.567" video="mainVideo-async" id="subtitle"]]
[[!template text="""We won't even know if the connection is up,""" start="00:05:52.927" video="mainVideo-async" id="subtitle"]]
[[!template text="""let alone what the response would be.""" start="00:05:56.327" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""This has some implications.""" start="00:05:59.247" video="mainVideo-async" id="subtitle"]]
[[!template text="""At this point we've returned control to the caller,""" start="00:06:01.807" video="mainVideo-async" id="subtitle"]]
[[!template text="""the Emacs event loop is proceeding""" start="00:06:05.247" video="mainVideo-async" id="subtitle"]]
[[!template text="""quite happily, and so the caller is free""" start="00:06:08.247" video="mainVideo-async" id="subtitle"]]
[[!template text="""to start using our connection object.""" start="00:06:11.304" video="mainVideo-async" id="subtitle"]]
[[!template text="""They might say issue a status command.""" start="00:06:15.047" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, well, in our library""" start="00:06:18.367" video="mainVideo-async" id="subtitle"]]
[[!template text="""we don't have a connection yet.""" start="00:06:20.248" video="mainVideo-async" id="subtitle"]]
[[!template text="""How on earth are we going to service this?""" start="00:06:22.887" video="mainVideo-async" id="subtitle"]]
[[!template text="""Well, we can simply give ourselves a queue""" start="00:06:26.607" video="mainVideo-async" id="subtitle"]]
[[!template text="""and note down the fact that""" start="00:06:29.807" video="mainVideo-async" id="subtitle"]]
[[!template text="""we owe a status command.""" start="00:06:31.820" video="mainVideo-async" id="subtitle"]]
[[!template text="""That's pretty quick.""" start="00:06:34.567" video="mainVideo-async" id="subtitle"]]
[[!template text="""We've now returned control back to our caller,""" start="00:06:35.647" video="mainVideo-async" id="subtitle"]]
[[!template text="""and they are again free to issue more commands.""" start="00:06:38.087" video="mainVideo-async" id="subtitle"]]
[[!template text="""Maybe they issue a play command.""" start="00:06:40.527" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, well, we're going to go deeper into depth,""" start="00:06:42.567" video="mainVideo-async" id="subtitle"]]
[[!template text="""and note that we also owe a play command.""" start="00:06:45.167" video="mainVideo-async" id="subtitle"]]
[[!template text="""At some point in the indeterminate future,""" start="00:06:51.127" video="mainVideo-async" id="subtitle"]]
[[!template text="""the connection will get up,""" start="00:06:54.007" video="mainVideo-async" id="subtitle"]]
[[!template text="""MPD will allocate resources to track a new client.""" start="00:06:57.598" video="mainVideo-async" id="subtitle"]]
[[!template text="""They will write the welcome string into the socket,""" start="00:07:03.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""and those bytes are going to show up""" start="00:07:06.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""in the Emacs process's read buffer,""" start="00:07:07.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""at which point our callback will be invoked.""" start="00:07:10.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""We can parse the welcome string, maybe""" start="00:07:13.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""note the version of that connection object""" start="00:07:17.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""that might come in handy.""" start="00:07:19.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""But the key point is our callback needs to""" start="00:07:20.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""take a look at the queue and notice:""" start="00:07:23.021" video="mainVideo-async" id="subtitle"]]
[[!template text="""&quot;Oh. We owe a status command,&quot;""" start="00:07:25.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""and so we'll invoke `process-send-string`,""" start="00:07:27.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""and send the status command down the pipe.""" start="00:07:29.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""Again, at some indeterminate time in the future""" start="00:07:32.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""some bytes are going to show up""" start="00:07:36.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""in our process's read buffer""" start="00:07:38.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""and our callback will again be invoked.""" start="00:07:41.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""We've got volume is 75 plus a lot of other stuff,""" start="00:07:44.721" video="mainVideo-async" id="subtitle"]]
[[!template text="""and here we come to the next problem.""" start="00:07:48.560" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""If our caller invoked status,""" start="00:07:50.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""they probably wanted to know about the status,""" start="00:07:54.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""so how shall we get them to them?""" start="00:07:56.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""Well, there's really not a lot of options""" start="00:07:59.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""at this point except the callback.""" start="00:08:02.601" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, so change of plan.""" start="00:08:04.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""Our queue is no longer a queue of commands.""" start="00:08:07.568" video="mainVideo-async" id="subtitle"]]
[[!template text="""It's going to be a queue of commands""" start="00:08:11.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""with associated callbacks.""" start="00:08:13.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""We read the response off the socket,""" start="00:08:15.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""invoke our caller supplied callback,""" start="00:08:20.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""and then pop the queue.""" start="00:08:23.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""At this point our callback""" start="00:08:26.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""(the library callback) needs to know""" start="00:08:28.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""that we still have a pending command,""" start="00:08:32.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""we fire that off down the pipe,""" start="00:08:34.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""at some indeterminate time in the future""" start="00:08:35.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""we get a response,""" start="00:08:38.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""our callback is invoked,""" start="00:08:40.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""we invoke the caller supplied callback,""" start="00:08:42.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""and we pop the queue.""" start="00:08:45.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""The structure of such a program""" start="00:08:47.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""is best viewed as a finite state machine.""" start="00:08:53.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""And this is typically where you end up""" start="00:08:55.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""in asynchronous programming,""" start="00:08:57.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""at least when you don't have a runtime grafted""" start="00:08:59.835" video="mainVideo-async" id="subtitle"]]
[[!template text="""onto your program the way you do with Go lang,""" start="00:09:02.551" video="mainVideo-async" id="subtitle"]]
[[!template text="""or when you don't have sort of extensive""" start="00:09:04.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""library support the way you do with Rust.""" start="00:09:07.251" video="mainVideo-async" id="subtitle"]]
[[!template text="""Your data structure exists in one of these states""" start="00:09:09.680" video="mainVideo-async" id="subtitle"]]
[[!template text="""at any given time,""" start="00:09:14.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""and when input shows up on your file descriptor,""" start="00:09:15.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""you transition along one of these edges""" start="00:09:18.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""to a new state.""" start="00:09:21.368" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""Cool. So, let's take a look at some of the code""" start="00:09:24.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""that flows from this.""" start="00:09:28.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, let's hop over to an Emacs""" start="00:09:30.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""and take a look at how we might code this up.""" start="00:09:32.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""If you recall the sequence diagrams I shared,""" start="00:09:35.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""we're going to be scribbling down the command""" start="00:09:38.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""and the callback that we'll be invoking""" start="00:09:40.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""upon its completion.""" start="00:09:42.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""The first thing I did was""" start="00:09:43.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""give myself a little command struct.""" start="00:09:44.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""With that, I was able to define""" start="00:09:47.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""the connection object.""" start="00:09:51.159" video="mainVideo-async" id="subtitle"]]
[[!template text="""We're going to be storing""" start="00:09:52.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""the handle to the connection.""" start="00:09:55.134" video="mainVideo-async" id="subtitle"]]
[[!template text="""We're going to write down the protocol version""" start="00:09:56.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""that we harvest from the welcome message,""" start="00:09:59.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""and of course we'll be recording""" start="00:10:02.117" video="mainVideo-async" id="subtitle"]]
[[!template text="""the command queue as well.""" start="00:10:04.249" video="mainVideo-async" id="subtitle"]]
[[!template text="""And so I gave myself a little connection object,""" start="00:10:06.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""connection struct with those three attributes.""" start="00:10:08.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""With the data model squared away,""" start="00:10:12.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""it was really pretty easy""" start="00:10:15.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""to code up the connect implementation.""" start="00:10:16.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""I'm eliding some details for exposition purposes,""" start="00:10:19.918" video="mainVideo-async" id="subtitle"]]
[[!template text="""but in the event it's really not that more complex""" start="00:10:25.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""than what you see here.""" start="00:10:29.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""We're going to unpack the arguments,""" start="00:10:31.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""figure out where the MPD server is""" start="00:10:32.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""to which you would like us to connect.""" start="00:10:35.651" video="mainVideo-async" id="subtitle"]]
[[!template text="""We'll connect via `make-network-process`.""" start="00:10:37.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""We'll associate a library defined callback""" start="00:10:40.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""with that connection via `set-process-filter`.""" start="00:10:42.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""Then we'll instantiate the connection object""" start="00:10:45.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""and return it to the caller.""" start="00:10:48.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""Once the caller has a connection object,""" start="00:10:51.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""they're free to send commands down that connection.""" start="00:10:53.818" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, what we're doing here is simply""" start="00:10:57.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""instantiating a command object""" start="00:11:00.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""on the basis of the caller supplied arguments""" start="00:11:02.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""and appending it to the queue.""" start="00:11:05.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""Then the last thing we do,""" start="00:11:07.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""and I've just indicated this with a comment,""" start="00:11:08.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""is we kick the queue.""" start="00:11:10.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""This kind of goes back""" start="00:11:12.680" video="mainVideo-async" id="subtitle"]]
[[!template text="""to the state transition diagram""" start="00:11:14.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""I laid out earlier.""" start="00:11:16.680" video="mainVideo-async" id="subtitle"]]
[[!template text="""What this means is the logic for saying,""" start="00:11:18.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""well, if we're awaiting the completion""" start="00:11:22.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""of a previously sent command,""" start="00:11:24.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""there's really not much more to be done.""" start="00:11:25.851" video="mainVideo-async" id="subtitle"]]
[[!template text="""We're just going to push this command""" start="00:11:27.501" video="mainVideo-async" id="subtitle"]]
[[!template text="""onto the queue and return.""" start="00:11:30.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""On the other hand, if the queue is empty""" start="00:11:31.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""on entry to `elmpd-send`,""" start="00:11:34.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""there's no reason not to just""" start="00:11:37.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""immediately send the command.""" start="00:11:38.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""This is an example of the sort of client side code""" start="00:11:40.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""that results from this API.""" start="00:11:46.600" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""So, you can see here, we are giving ourselves""" start="00:11:48.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""a connection to the MPD server on the localhost.""" start="00:11:51.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""We're going to send the get volume command""" start="00:11:54.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""down that connection.""" start="00:11:57.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""And if that command completes and all is well,""" start="00:11:58.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""we'll just send a message to Emacs.""" start="00:12:02.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""Unfortunately, you can't see my minibuffer,""" start="00:12:05.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""so I'll hop over to the Messages buffer.""" start="00:12:07.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""And there's our result.""" start="00:12:11.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""The volume is 43.""" start="00:12:12.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""Great, I thought.""" start="00:12:15.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""Simple, clean, responsive, easy to code to.""" start="00:12:18.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""That is unfortunately not the end of the story.""" start="00:12:24.160" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""Let's continue this example a little bit.""" start="00:12:27.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""Let's imagine that""" start="00:12:31.736" video="mainVideo-async" id="subtitle"]]
[[!template text="""if the volume comes back from the server""" start="00:12:33.301" video="mainVideo-async" id="subtitle"]]
[[!template text="""and it is less than 50,""" start="00:12:35.501" video="mainVideo-async" id="subtitle"]]
[[!template text="""we would like to set it to 50.""" start="00:12:37.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, this is interesting""" start="00:12:39.335" video="mainVideo-async" id="subtitle"]]
[[!template text="""because we have two commands""" start="00:12:41.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""and whether or not we send the second command""" start="00:12:43.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""it is going to depend on""" start="00:12:45.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""the response we get from the first.""" start="00:12:46.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay. I thought, well, that's fine.""" start="00:12:48.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""I can simply put that logic in the callback""" start="00:12:51.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""that I specified for the get volume command.""" start="00:12:55.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, here we are we check the return code,""" start="00:12:58.701" video="mainVideo-async" id="subtitle"]]
[[!template text="""we parse the volume, we compare it to 50,""" start="00:13:01.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""and if it's less we just invoke `elmpd-send` again""" start="00:13:04.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""from the first command's callback.""" start="00:13:08.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""Okay, I could live with that""" start="00:13:10.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""it's not the worst thing I've ever seen.""" start="00:13:14.051" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""Let's extend this example a little further,""" start="00:13:15.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""and this is contrived, but bear with me.""" start="00:13:19.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""Let us suppose that if we do set the volume to 50,""" start="00:13:21.868" video="mainVideo-async" id="subtitle"]]
[[!template text="""we'd like to get the volume one more time just to""" start="00:13:25.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""make sure that our change took on the server.""" start="00:13:28.635" video="mainVideo-async" id="subtitle"]]
[[!template text="""We can play the same game.""" start="00:13:30.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""We will put that logic in the callback""" start="00:13:33.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""that we specified for the set volume command.""" start="00:13:37.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""And here we are, we check the return code,""" start="00:13:40.168" video="mainVideo-async" id="subtitle"]]
[[!template text="""we send a message to Emacs,""" start="00:13:43.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""we send the get volume command again""" start="00:13:45.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""along with its own callback.""" start="00:13:49.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""And at this point I hope it's clear""" start="00:13:51.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""(the problem that is emerging),""" start="00:13:55.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""and if it's not yet, let me note that so far""" start="00:13:57.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""we're only handling the happy path""" start="00:14:01.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""in each of these callbacks.""" start="00:14:03.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""We really ought to do something about the error path.""" start="00:14:04.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""For purposes of illustration, let's just say,""" start="00:14:06.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""we send a message to Emacs""" start="00:14:10.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""that means it would look like this.""" start="00:14:12.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""At this point that I really think""" start="00:14:14.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""it's impossible to deny that this API is actually""" start="00:14:17.335" video="mainVideo-async" id="subtitle"]]
[[!template text="""not that easy to program to.""" start="00:14:20.851" video="mainVideo-async" id="subtitle"]]
[[!template text="""If there are any JavaScript devs watching,""" start="00:14:23.280" video="mainVideo-async" id="subtitle"]]
[[!template text="""you're probably chuckling right now""" start="00:14:27.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""because I have discovered for myself""" start="00:14:28.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""what they call &quot;callback hell&quot;.""" start="00:14:30.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""If you are returning""" start="00:14:33.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""the results of asynchronous function invocations""" start="00:14:37.868" video="mainVideo-async" id="subtitle"]]
[[!template text="""to their caller via callbacks,""" start="00:14:40.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""you pretty much inevitably end up in this sort of""" start="00:14:42.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""deeply nested sequence of callbacks""" start="00:14:45.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""that is difficult to write, difficult to read,""" start="00:14:48.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""and difficult to reason about.""" start="00:14:50.768" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""And yet when I was stuck in this situation""" start="00:14:53.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""it just seemed like it really shouldn't be this bad.""" start="00:14:57.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""If I give myself this sort of tabular data structure,""" start="00:15:00.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""I felt that this expressed precisely the same logic""" start="00:15:05.320" video="mainVideo-async" id="subtitle"]]
[[!template text="""just in a much easier to read manner.""" start="00:15:10.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""I could, in my mind's eye,""" start="00:15:12.868" video="mainVideo-async" id="subtitle"]]
[[!template text="""see the code for transforming this data structure,""" start="00:15:15.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""which is really just a list,""" start="00:15:19.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""into the code that you just""" start="00:15:21.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""saw in the previous slide,""" start="00:15:23.585" video="mainVideo-async" id="subtitle"]]
[[!template text="""and really if Lisp is good at anything""" start="00:15:25.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""it is list processing right.""" start="00:15:29.440" video="mainVideo-async" id="subtitle"]]
[[!template text="""And it was really at this point""" start="00:15:31.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""that a little bit of enlightenment dawned.""" start="00:15:33.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""I learned that Lisp is homo iconic,""" start="00:15:36.368" video="mainVideo-async" id="subtitle"]]
[[!template text="""which is just means that the language itself""" start="00:15:40.800" video="mainVideo-async" id="subtitle"]]
[[!template text="""is a data structure in that language.""" start="00:15:46.040" video="mainVideo-async" id="subtitle"]]
[[!template text="""Lisp code is, after all, just a list.""" start="00:15:49.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""And the power of Lisp macros""" start="00:15:53.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""is taking that data structure,""" start="00:15:57.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""some data structure that you've defined,""" start="00:15:59.760" video="mainVideo-async" id="subtitle"]]
[[!template text="""and doing exactly what I wanted to do.""" start="00:16:02.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""Transforming it from one list into another,""" start="00:16:04.640" video="mainVideo-async" id="subtitle"]]
[[!template text="""the destination list being Lisp code.""" start="00:16:07.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, I got busy, and I coded up my first Lisp macro,""" start="00:16:11.835" video="mainVideo-async" id="subtitle"]]
[[!template text="""which I called `elmpd-chain`.""" start="00:16:16.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""And that lengthy list of, you know,""" start="00:16:19.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""three or four nested callbacks""" start="00:16:21.600" video="mainVideo-async" id="subtitle"]]
[[!template text="""gets turned into this,""" start="00:16:24.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""which I hope you'll agree is much simpler""" start="00:16:25.920" video="mainVideo-async" id="subtitle"]]
[[!template text="""much easier to read, much easier to reason about.""" start="00:16:29.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""And if you're morbidly curious,""" start="00:16:32.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""you can expand your macros,""" start="00:16:36.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""and this invocation of `elmpd-chain` expands to this.""" start="00:16:40.160" video="mainVideo-async" id="subtitle"]]
[[!template text="""So, that's my story.""" start="00:16:44.200" video="mainVideo-async" id="subtitle"]]
[[!template new="1" text="""In all fairness, I should note that""" start="00:16:46.400" video="mainVideo-async" id="subtitle"]]
[[!template text="""the MPD protocol has some subtleties""" start="00:16:50.840" video="mainVideo-async" id="subtitle"]]
[[!template text="""and complexities that I didn't really get into,""" start="00:16:53.501" video="mainVideo-async" id="subtitle"]]
[[!template text="""both due to time constraints""" start="00:16:56.880" video="mainVideo-async" id="subtitle"]]
[[!template text="""and because they're not terribly relevant""" start="00:16:58.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""to the points I wanted to touch on.""" start="00:17:00.520" video="mainVideo-async" id="subtitle"]]
[[!template text="""I should also note that there's""" start="00:17:02.000" video="mainVideo-async" id="subtitle"]]
[[!template text="""a fair amount of work in the library itself""" start="00:17:05.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""around accumulating partial responses""" start="00:17:07.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""as they show up in the buffer""" start="00:17:11.240" video="mainVideo-async" id="subtitle"]]
[[!template text="""and dispatching them piecemeal to the caller.""" start="00:17:12.560" video="mainVideo-async" id="subtitle"]]
[[!template text="""That was really too complex to get into here.""" start="00:17:16.120" video="mainVideo-async" id="subtitle"]]
[[!template text="""If you would like to see the code,""" start="00:17:20.468" video="mainVideo-async" id="subtitle"]]
[[!template text="""it's available on GitHub as well as MELPA.""" start="00:17:22.360" video="mainVideo-async" id="subtitle"]]
[[!template text="""I'll be putting a version of this talk""" start="00:17:25.080" video="mainVideo-async" id="subtitle"]]
[[!template text="""on my personal site,""" start="00:17:29.200" video="mainVideo-async" id="subtitle"]]
[[!template text="""and you can always reach out to me personally,""" start="00:17:30.480" video="mainVideo-async" id="subtitle"]]
[[!template text="""I hang out on IRC as sp1ff,""" start="00:17:33.720" video="mainVideo-async" id="subtitle"]]
[[!template text="""or you can just email me as sp1ff@pobox.com.""" start="00:17:36.960" video="mainVideo-async" id="subtitle"]]
[[!template text="""Thank you very much.""" start="00:17:41.920" video="mainVideo-async" id="subtitle"]]



Captioner: bhavin192

Questions or comments? Please e-mail [emacsconf-org-private@gnu.org](mailto:emacsconf-org-private@gnu.org?subject=Comment%20for%20EmacsConf%202022%20async%3A%20Emacs%20was%20async%20before%20async%20was%20cool)


<!-- End of emacsconf-publish-after-page -->