WEBVTT 1 00:00:00.000 --> 00:00:02.720 Hey everyone, I'm Michael 2 00:00:02.720 --> 00:00:04.480 and I'm going to be talking to you today 3 00:00:04.480 --> 00:00:07.640 about asynchronous programming in Emacs Lisp. 4 00:00:07.640 --> 00:00:10.360 I'm located in the San Francisco Bay Area 5 00:00:10.360 --> 00:00:12.040 where I'm a developer as well as 6 00:00:12.040 --> 00:00:14.160 a long time Emacs user. 7 00:00:14.160 --> 00:00:18.760 You may have heard of async or asynchronous programming. 8 00:00:18.760 --> 00:00:21.360 The idea has been around for decades 9 00:00:21.360 --> 00:00:24.400 but it first gained widespread attention in JavaScript 10 00:00:24.400 --> 00:00:26.720 back in the aughts. 11 00:00:26.720 --> 00:00:29.680 Then in the teens it gained tremendous popularity 12 00:00:29.680 --> 00:00:31.720 in the DevOps world with Golang. 13 00:00:31.720 --> 00:00:33.800 And just in the last few years 14 00:00:33.800 --> 00:00:37.880 support for async programming has landed in Rust. 15 00:00:37.880 --> 00:00:40.080 Well it can be done in Emacs as well 16 00:00:40.080 --> 00:00:42.040 and this talk will demonstrate that 17 00:00:42.040 --> 00:00:44.600 by walking you through a little problem 18 00:00:44.600 --> 00:00:47.200 that I actually solved for myself. 19 00:00:47.200 --> 00:00:49.040 Like a lot of these stories 20 00:00:49.040 --> 00:00:51.920 it begins with scratching a personal itch. 21 00:00:51.920 --> 00:00:55.320 In my case automating my music server. 22 00:00:55.320 --> 00:00:58.240 I use something called the music player daemon locally 23 00:00:58.240 --> 00:01:00.240 and as the name suggests 24 00:01:00.240 --> 00:01:03.560 it just kind of hangs out in the background. 25 00:01:03.560 --> 00:01:08.040 Reads music files and talks to assorted sound drivers. 26 00:01:08.040 --> 00:01:09.640 In fact it is so focused on 27 00:01:09.640 --> 00:01:12.440 that mission that it doesn't even offer a user interface. 28 00:01:12.440 --> 00:01:14.400 Instead it serves an API 29 00:01:14.400 --> 00:01:16.120 and invites application developers 30 00:01:16.120 --> 00:01:19.360 to build clients on top of that API. 31 00:01:19.360 --> 00:01:22.200 Okay so let's hop into a vterm 32 00:01:22.200 --> 00:01:25.080 and I'd like to show you the MPD client I use 33 00:01:25.080 --> 00:01:26.600 for my daily driver. 34 00:01:26.600 --> 00:01:29.520 Something called ncmpcpp. 35 00:01:29.520 --> 00:01:31.800 Doesn't exactly roll off the tongue 36 00:01:31.800 --> 00:01:33.720 but I've got a playlist. 37 00:01:33.720 --> 00:01:36.560 I can browse the file system. 38 00:01:36.560 --> 00:01:39.240 Looks like I can search my music library. 39 00:01:39.240 --> 00:01:40.000 Yada yada yada. 40 00:01:40.000 --> 00:01:42.600 It's got all the basic features. 41 00:01:42.600 --> 00:01:44.640 The point that I want to make is that 42 00:01:44.640 --> 00:01:51.920 ncmpcpp is a completely independent project of MPD. 43 00:01:51.920 --> 00:01:53.720 Separate and distinct. 44 00:01:53.720 --> 00:01:55.680 It does all of its work 45 00:01:55.680 --> 00:01:57.200 by simply communicating with 46 00:01:57.200 --> 00:02:01.400 the music player daemon over the API. 47 00:02:01.400 --> 00:02:03.440 Well I wanted to program to that API 48 00:02:03.440 --> 00:02:05.840 only from within Emacs. 49 00:02:05.840 --> 00:02:09.520 Now there are already Emacs MPD clients out there 50 00:02:09.520 --> 00:02:11.560 but I didn't really want a full blown client. 51 00:02:11.560 --> 00:02:14.320 I just wanted a few small tweaks 52 00:02:14.320 --> 00:02:16.320 over my current configuration. 53 00:02:16.320 --> 00:02:19.280 A command to skip to the next song. 54 00:02:19.280 --> 00:02:22.360 Maybe shove the current track into the mode line. 55 00:02:22.360 --> 00:02:24.160 Things like this. 56 00:02:24.160 --> 00:02:28.560 I needed an elisp API that would let me do this. 57 00:02:28.560 --> 00:02:32.000 Okay well let's get out of ncmpcpp 58 00:02:32.000 --> 00:02:37.560 and let's get into a netcat session 59 00:02:37.560 --> 00:02:39.400 with my local MPD server. 60 00:02:39.400 --> 00:02:43.840 As you can see we get a welcome string. 61 00:02:43.840 --> 00:02:46.800 So it is a server goes first protocol. 62 00:02:46.800 --> 00:02:49.640 But after that it's a very familiar 63 00:02:49.640 --> 00:02:53.960 text based request response oriented protocol. 64 00:02:53.960 --> 00:02:56.240 I can ask for the volume. 65 00:02:56.240 --> 00:02:58.160 I can ask for the status. 66 00:02:58.160 --> 00:03:06.000 But in particular I wanted an asynchronous API. 67 00:03:06.000 --> 00:03:07.800 If I issue a command like 68 00:03:07.800 --> 00:03:11.840 find every track in my library 69 00:03:11.840 --> 00:03:15.360 that's going to produce a lot of data 70 00:03:15.360 --> 00:03:18.920 that's a human perceptible pause 71 00:03:18.920 --> 00:03:22.080 as Emacs processes all the input. 72 00:03:22.080 --> 00:03:25.560 What I wanted was a style of programming 73 00:03:25.560 --> 00:03:28.080 where I could fire off my command 74 00:03:28.080 --> 00:03:31.560 have the Emacs command loop keep working 75 00:03:31.560 --> 00:03:33.440 and only invoke some callback 76 00:03:33.440 --> 00:03:35.280 when there was data available. 77 00:03:35.280 --> 00:03:39.560 Well Emacs is famously single threaded 78 00:03:39.560 --> 00:03:41.840 so it shouldn't come as a surprise 79 00:03:41.840 --> 00:03:44.080 that it offers a rich set of primitives 80 00:03:44.080 --> 00:03:46.720 that enable the sort of network programming 81 00:03:46.720 --> 00:03:49.320 that I wanted to do. 82 00:03:49.320 --> 00:03:50.760 In particular it offers 83 00:03:50.760 --> 00:03:53.280 a function called make network process. 84 00:03:53.280 --> 00:03:57.800 Now this method offers a bewildering variety of options. 85 00:03:57.800 --> 00:03:59.320 But at the heart of the matter 86 00:03:59.320 --> 00:04:01.040 it opens a network connection 87 00:04:01.040 --> 00:04:03.120 to some endpoint out there 88 00:04:03.120 --> 00:04:06.640 and we can configure it to be non blocking. 89 00:04:06.640 --> 00:04:09.840 It returns a handle that you can use to refer to 90 00:04:09.840 --> 00:04:14.880 this network connection with other methods. 91 00:04:14.880 --> 00:04:17.760 Other methods such as process and string 92 00:04:17.760 --> 00:04:19.600 which as the name suggests 93 00:04:19.600 --> 00:04:21.960 allows you to send textual data 94 00:04:21.960 --> 00:04:26.320 to the remote endpoint of your network connection. 95 00:04:26.320 --> 00:04:29.400 You can also use it with set process filter 96 00:04:29.400 --> 00:04:32.160 which allows you to associate a callback 97 00:04:32.160 --> 00:04:33.240 with your network connection. 98 00:04:33.240 --> 00:04:35.920 That callback will be invoked 99 00:04:35.920 --> 00:04:40.480 when there is data available 100 00:04:40.480 --> 00:04:41.960 in the processes read buffer. 101 00:04:41.960 --> 00:04:44.960 In other words in a request response oriented protocol 102 00:04:44.960 --> 00:04:47.800 like that of MPD you open your socket 103 00:04:47.800 --> 00:04:50.960 with make network process 104 00:04:50.960 --> 00:04:53.760 send your request via process send string 105 00:04:53.760 --> 00:04:56.360 and life will just continue in emacs 106 00:04:56.360 --> 00:04:57.560 until some data shows up 107 00:04:57.560 --> 00:05:00.720 in the processes read buffer 108 00:05:00.720 --> 00:05:05.200 at which point your callback will be invoked. 109 00:05:05.200 --> 00:05:07.560 It turns out this was enough 110 00:05:07.560 --> 00:05:12.280 for a purpose built async runtime. 111 00:05:12.280 --> 00:05:14.800 Let's work through the sequence of events 112 00:05:14.800 --> 00:05:16.480 when opening a connection 113 00:05:16.480 --> 00:05:18.720 and firing off a few commands in this style. 114 00:05:18.720 --> 00:05:22.120 So let's imagine a library 115 00:05:22.120 --> 00:05:25.520 that offers a connection object of some sort 116 00:05:25.520 --> 00:05:28.720 a caller and an MPD server out on the network. 117 00:05:28.720 --> 00:05:31.880 The caller will presumably get themselves 118 00:05:31.880 --> 00:05:34.760 a connection object by invoking some sort of 119 00:05:34.760 --> 00:05:38.080 connect method on our library. 120 00:05:38.080 --> 00:05:41.160 We can handle this through make network process 121 00:05:41.160 --> 00:05:45.360 but we're going to invoke make network process 122 00:05:45.360 --> 00:05:47.200 with no weight equal to true 123 00:05:47.200 --> 00:05:48.520 in other words asynchronously. 124 00:05:48.520 --> 00:05:52.240 That means the method is going to return immediately. 125 00:05:52.240 --> 00:05:56.320 We won't even know if the connection is up 126 00:05:56.320 --> 00:05:57.920 let alone what the response would be. 127 00:05:57.920 --> 00:06:01.560 This has some implications. 128 00:06:01.560 --> 00:06:05.280 At this point we've returned control to the caller 129 00:06:05.280 --> 00:06:09.400 the emacs event loop is proceeding quite happily 130 00:06:09.400 --> 00:06:11.320 and so the caller is free 131 00:06:11.320 --> 00:06:14.920 to start using our connection object. 132 00:06:14.920 --> 00:06:17.640 They might say issue a status command. 133 00:06:17.640 --> 00:06:20.600 Okay well in our library 134 00:06:20.600 --> 00:06:22.680 we don't have a connection yet. 135 00:06:22.680 --> 00:06:25.920 How on earth are we going to service this? 136 00:06:25.920 --> 00:06:29.440 Well we can simply give ourselves a queue 137 00:06:29.440 --> 00:06:33.360 and note down the fact that we owe a status command. 138 00:06:33.360 --> 00:06:35.560 That's pretty quick. 139 00:06:35.560 --> 00:06:38.120 We've now returned control back to our caller 140 00:06:38.120 --> 00:06:40.640 and they are again free to issue more commands. 141 00:06:40.640 --> 00:06:41.840 Maybe they issue a play command. 142 00:06:41.840 --> 00:06:45.160 Okay well we're going to go deeper into debt 143 00:06:45.160 --> 00:06:48.160 and note that we also owe a play command. 144 00:06:48.160 --> 00:06:56.160 At some point in the indeterminate future MPDU 145 00:06:56.160 --> 00:06:57.320 is the connection will get up 146 00:06:57.320 --> 00:07:03.000 MPDU will allocate resources to track a new client. 147 00:07:03.000 --> 00:07:06.160 They will write the welcome string into the socket 148 00:07:06.160 --> 00:07:07.920 and those bytes are going to show up 149 00:07:07.920 --> 00:07:10.360 in the emacs process read buffer 150 00:07:10.360 --> 00:07:13.160 at which point our callback will be invoked. 151 00:07:13.160 --> 00:07:17.440 We can parse the welcome string maybe 152 00:07:17.440 --> 00:07:19.240 note the version that connection object 153 00:07:19.240 --> 00:07:20.400 that might come in handy 154 00:07:20.400 --> 00:07:21.720 but the key point is 155 00:07:21.720 --> 00:07:24.080 our callback needs to take a look at the queue 156 00:07:24.080 --> 00:07:25.240 and notice 157 00:07:25.240 --> 00:07:27.200 oh we owe a status command 158 00:07:27.200 --> 00:07:29.880 and so we'll invoke process and string 159 00:07:29.880 --> 00:07:32.280 and send the status command down the pipe. 160 00:07:32.280 --> 00:07:36.760 Again at some indeterminate time in the future 161 00:07:36.760 --> 00:07:38.600 some bytes are going to show up 162 00:07:38.600 --> 00:07:41.200 in our processes read buffer 163 00:07:41.200 --> 00:07:43.160 and our callback will again be invoked. 164 00:07:43.160 --> 00:07:48.560 We've got volume is 75 plus a lot of other stuff 165 00:07:48.560 --> 00:07:50.480 and here we come to the next problem. 166 00:07:50.480 --> 00:07:54.440 If our caller invoked status 167 00:07:54.440 --> 00:07:56.960 they probably wanted to know about the status 168 00:07:56.960 --> 00:07:59.880 so how shall we get them to them? 169 00:07:59.880 --> 00:08:03.040 Well there's really not a lot of options at this point 170 00:08:03.040 --> 00:08:04.280 except the callback. 171 00:08:04.280 --> 00:08:09.000 Okay so change of plan our queue 172 00:08:09.000 --> 00:08:11.720 is no longer a queue of commands 173 00:08:11.720 --> 00:08:13.840 it's going to be a queue of commands 174 00:08:13.840 --> 00:08:15.880 with associated callbacks. 175 00:08:15.880 --> 00:08:20.280 We read the response off the socket 176 00:08:20.280 --> 00:08:23.440 invoke our caller supplied callback 177 00:08:23.440 --> 00:08:26.080 and then pop the queue. 178 00:08:26.080 --> 00:08:28.920 At this point our callback 179 00:08:28.920 --> 00:08:32.160 the library callback needs to know 180 00:08:32.160 --> 00:08:34.040 that we still have a pending command 181 00:08:34.040 --> 00:08:35.720 we fire that off down the pipe 182 00:08:35.720 --> 00:08:38.520 at some indeterminate time in the future 183 00:08:38.520 --> 00:08:40.360 we get a call we get a response 184 00:08:40.360 --> 00:08:42.640 our callback is invoked 185 00:08:42.640 --> 00:08:45.720 we invoke the caller supplied callback 186 00:08:45.720 --> 00:08:47.240 and we pop the queue. 187 00:08:47.240 --> 00:08:53.760 The structure of such a program 188 00:08:53.760 --> 00:08:55.800 is best viewed as a finite state machine 189 00:08:55.800 --> 00:08:57.640 and this is typically where you end up 190 00:08:57.640 --> 00:08:59.200 in asynchronous programming at least 191 00:08:59.200 --> 00:09:03.360 when you don't have a runtime grafted onto your program 192 00:09:03.360 --> 00:09:04.960 the way you do with Golang 193 00:09:04.960 --> 00:09:08.240 or when you don't have sort of extensive library support 194 00:09:08.240 --> 00:09:09.680 the way you do with Rust. 195 00:09:09.680 --> 00:09:14.480 Your data structure exists in one of these states 196 00:09:14.480 --> 00:09:15.440 at any given time 197 00:09:15.440 --> 00:09:18.960 and when input shows up on your file descriptor 198 00:09:18.960 --> 00:09:24.240 you transition along one of these edges to a new state. 199 00:09:24.240 --> 00:09:28.160 Cool so let's take a look at some of the code 200 00:09:28.160 --> 00:09:29.480 that flows from this. 201 00:09:29.480 --> 00:09:32.240 Okay let's hop over to an Emacs 202 00:09:32.240 --> 00:09:33.920 and take a look at how we might code this up. 203 00:09:33.920 --> 00:09:38.360 If you recall the sequence diagrams I shared 204 00:09:38.360 --> 00:09:40.120 we're going to be scribbling down the command 205 00:09:40.120 --> 00:09:42.160 and the callback that will be invoking 206 00:09:42.160 --> 00:09:43.240 upon its completion. 207 00:09:43.240 --> 00:09:45.440 So the first thing I did was give myself 208 00:09:45.440 --> 00:09:47.400 a little command struct 209 00:09:47.400 --> 00:09:52.280 with that I was able to define the connection object. 210 00:09:52.280 --> 00:09:56.280 We're going to be storing the handle to the connection. 211 00:09:56.280 --> 00:09:59.400 We're going to write down the protocol version 212 00:09:59.400 --> 00:10:02.000 that we harvest from the welcome message 213 00:10:02.000 --> 00:10:03.560 and of course we'll be recording 214 00:10:03.560 --> 00:10:05.760 the command queue as well. 215 00:10:05.760 --> 00:10:08.640 And so I gave myself a little connection object 216 00:10:08.640 --> 00:10:10.960 with a connection struct 217 00:10:10.960 --> 00:10:12.240 with those three attributes. 218 00:10:12.240 --> 00:10:15.000 With the data model squared away 219 00:10:15.000 --> 00:10:17.840 it was really pretty easy to code up 220 00:10:17.840 --> 00:10:21.160 the connect implementation. 221 00:10:21.160 --> 00:10:24.880 I'm deleting some details for exposition purposes 222 00:10:24.880 --> 00:10:29.520 but in the event it's really not that more complex 223 00:10:29.520 --> 00:10:30.520 than what you see here. 224 00:10:30.520 --> 00:10:32.840 We're going to unpack the arguments, 225 00:10:32.840 --> 00:10:35.040 figure out where the MPD server is 226 00:10:35.040 --> 00:10:37.280 to which you would like us to connect. 227 00:10:37.280 --> 00:10:39.920 We'll connect via make network process. 228 00:10:39.920 --> 00:10:42.640 We'll associate a library defined callback 229 00:10:42.640 --> 00:10:45.920 with that connection via set process filter. 230 00:10:45.920 --> 00:10:48.440 Then we'll instantiate the connection object 231 00:10:48.440 --> 00:10:50.120 and return it to the caller. 232 00:10:50.120 --> 00:10:53.800 Once the caller has a connection object 233 00:10:53.800 --> 00:10:56.880 they're free to send commands down that connection. 234 00:10:56.880 --> 00:10:59.120 So what we're doing here 235 00:10:59.120 --> 00:11:02.320 is simply instantiating a command object 236 00:11:02.320 --> 00:11:05.200 on the basis of the caller supplied arguments 237 00:11:05.200 --> 00:11:06.640 and appending it to the queue. 238 00:11:06.640 --> 00:11:07.920 And then the last thing we do 239 00:11:07.920 --> 00:11:11.040 and I've just indicated this with a comment 240 00:11:11.040 --> 00:11:12.040 is we kick the queue. 241 00:11:12.040 --> 00:11:14.560 This kind of goes back to 242 00:11:14.560 --> 00:11:18.200 the state transition diagram I laid out earlier. 243 00:11:18.200 --> 00:11:22.680 What this means is the logic for saying well 244 00:11:22.680 --> 00:11:24.280 if we're waiting the completion 245 00:11:24.280 --> 00:11:25.480 of a previously sent command 246 00:11:25.480 --> 00:11:27.280 there's really not much more to be done. 247 00:11:27.280 --> 00:11:31.000 We're just going to push this command onto the queue 248 00:11:31.000 --> 00:11:31.600 and return. 249 00:11:31.600 --> 00:11:33.120 On the other hand 250 00:11:33.120 --> 00:11:37.120 if the queue was empty on entry to LMPD send 251 00:11:37.120 --> 00:11:39.160 there's no reason not to just 252 00:11:39.160 --> 00:11:43.400 immediately send the command. 253 00:11:43.400 --> 00:11:44.680 And this is an example of 254 00:11:44.680 --> 00:11:46.520 the sort of client side code 255 00:11:46.520 --> 00:11:48.080 that results from this API. 256 00:11:48.080 --> 00:11:51.360 So you can see here we are giving ourselves 257 00:11:51.360 --> 00:11:54.240 a connection to the MPD server on the local host 258 00:11:54.240 --> 00:11:56.600 and we're going to send the get volume command 259 00:11:56.600 --> 00:11:58.160 down that connection. 260 00:11:58.160 --> 00:12:02.840 And if that command completes and all is well 261 00:12:02.840 --> 00:12:05.360 we'll just send a message to Emacs. 262 00:12:05.360 --> 00:12:07.800 Unfortunately you can't see my mini buffer 263 00:12:07.800 --> 00:12:10.960 so I'll hop over to the messages buffer 264 00:12:10.960 --> 00:12:12.720 and there's our result. 265 00:12:12.720 --> 00:12:15.160 The volume is 43. 266 00:12:15.160 --> 00:12:17.960 Great I thought. 267 00:12:17.960 --> 00:12:22.520 Simple clean responsive easy to code to. 268 00:12:22.520 --> 00:12:27.760 That is unfortunately not the end of the story. 269 00:12:27.760 --> 00:12:32.320 Let's continue this example a little bit. 270 00:12:32.320 --> 00:12:33.560 Let's imagine that 271 00:12:33.560 --> 00:12:35.920 if the volume comes back from the server 272 00:12:35.920 --> 00:12:37.360 and it is less than 50 273 00:12:37.360 --> 00:12:38.600 we would like to set it to 50. 274 00:12:38.600 --> 00:12:41.560 So this is interesting 275 00:12:41.560 --> 00:12:43.200 because we have two commands 276 00:12:43.200 --> 00:12:45.840 and whether or not we send the second command 277 00:12:45.840 --> 00:12:46.840 is going to depend on 278 00:12:46.840 --> 00:12:48.560 the response we get from the first. 279 00:12:48.560 --> 00:12:51.640 Okay I thought well that's fine 280 00:12:51.640 --> 00:12:55.080 I can simply put that logic in the callback 281 00:12:55.080 --> 00:12:57.920 that I specified for the get volume command. 282 00:12:57.920 --> 00:13:01.560 So here we are we check the return code 283 00:13:01.560 --> 00:13:04.400 we parse the volume we compare it to 50 284 00:13:04.400 --> 00:13:08.360 and if it's less we just invoke LMPD send again 285 00:13:08.360 --> 00:13:10.800 from the first command's callback. 286 00:13:10.800 --> 00:13:13.440 Okay I could live with that 287 00:13:13.440 --> 00:13:15.520 it's not the worst thing I've ever seen. 288 00:13:15.520 --> 00:13:19.400 Let's extend this example a little further 289 00:13:19.400 --> 00:13:21.480 and this is contrived but bear with me. 290 00:13:21.480 --> 00:13:25.480 Let us suppose that if we do set the volume to 50 291 00:13:25.480 --> 00:13:27.800 we'd like to get the volume one more time 292 00:13:27.800 --> 00:13:30.640 just to make sure that our change took on the server. 293 00:13:30.640 --> 00:13:33.560 Okay we can play the same game. 294 00:13:33.560 --> 00:13:37.280 We will put that logic in the callback 295 00:13:37.280 --> 00:13:39.520 that we specified for the set volume command. 296 00:13:39.520 --> 00:13:43.480 And here we are we check the return code 297 00:13:43.480 --> 00:13:45.480 we send a message to Emacs 298 00:13:45.480 --> 00:13:49.200 we send the get volume command again 299 00:13:49.200 --> 00:13:51.080 along with its own callback 300 00:13:51.080 --> 00:13:55.280 and at this point I think you know I hope it's clear 301 00:13:55.280 --> 00:13:57.520 the problem that is emerging 302 00:13:57.520 --> 00:14:01.360 and if it's not yet let's let me note that so far 303 00:14:01.360 --> 00:14:03.000 we're only handling the happy path 304 00:14:03.000 --> 00:14:04.520 in each of these callbacks. 305 00:14:04.520 --> 00:14:06.840 We really ought to do something about the error path 306 00:14:06.840 --> 00:14:10.120 for purposes of illustration let's just say 307 00:14:10.120 --> 00:14:12.120 we send a message to Emacs 308 00:14:12.120 --> 00:14:14.320 that means it would look like this 309 00:14:14.320 --> 00:14:16.560 and it's at this point 310 00:14:16.560 --> 00:14:19.400 that I really think it's impossible to deny 311 00:14:19.400 --> 00:14:23.280 that this API is actually not that easy to program to 312 00:14:23.280 --> 00:14:27.160 and if there are any JavaScript devs watching 313 00:14:27.160 --> 00:14:28.840 you're probably chuckling right now 314 00:14:28.840 --> 00:14:30.720 because I have discovered for myself 315 00:14:30.720 --> 00:14:33.880 what they call callback hell. 316 00:14:33.880 --> 00:14:36.040 If you are returning 317 00:14:36.040 --> 00:14:40.160 the results of asynchronous function invocations 318 00:14:40.160 --> 00:14:42.200 to their caller via callbacks 319 00:14:42.200 --> 00:14:45.640 you pretty much inevitably end up in this sort of 320 00:14:45.640 --> 00:14:48.040 deeply nested sequence of callbacks 321 00:14:48.040 --> 00:14:49.880 that is difficult to write difficult to read 322 00:14:49.880 --> 00:14:53.520 and difficult to reason about. 323 00:14:53.520 --> 00:14:57.480 And yet when I was stuck in this situation 324 00:14:57.480 --> 00:15:00.080 it just seemed like it really shouldn't be this bad. 325 00:15:00.080 --> 00:15:05.320 If I give myself this sort of tabular data structure 326 00:15:05.320 --> 00:15:10.160 I felt that this expressed precisely the same logic 327 00:15:10.160 --> 00:15:11.960 just in a much easier to read manner. 328 00:15:11.960 --> 00:15:15.840 I could in my mind's eye 329 00:15:15.840 --> 00:15:19.720 see the code for transforming this data structure 330 00:15:19.720 --> 00:15:21.040 which is really just a list 331 00:15:21.040 --> 00:15:25.600 into the code that you just saw in the previous slide 332 00:15:25.600 --> 00:15:29.440 and really if Lisp is good at anything 333 00:15:29.440 --> 00:15:31.080 it is list processing right 334 00:15:31.080 --> 00:15:33.080 and it was really at this point 335 00:15:33.080 --> 00:15:35.240 that a little bit of enlightenment dawned. 336 00:15:35.240 --> 00:15:40.800 I learned that Lisp is homo iconic 337 00:15:40.800 --> 00:15:46.040 which is just means that the language itself 338 00:15:46.040 --> 00:15:49.360 is a data structure in that language. 339 00:15:49.360 --> 00:15:53.160 Lisp code is after all just a list 340 00:15:53.160 --> 00:15:57.160 and the power of Lisp macros 341 00:15:57.160 --> 00:15:59.760 is taking that data structure 342 00:15:59.760 --> 00:16:02.400 some data structure that you've defined 343 00:16:02.400 --> 00:16:04.640 and doing exactly what I wanted to do 344 00:16:04.640 --> 00:16:07.520 transforming it from one list into another 345 00:16:07.520 --> 00:16:11.080 the destination list being Lisp code. 346 00:16:11.080 --> 00:16:16.000 So I got busy and I coded up my first Lisp macro 347 00:16:16.000 --> 00:16:19.160 which I called LMPD chain 348 00:16:19.160 --> 00:16:21.600 and that lengthy list of you know 349 00:16:21.600 --> 00:16:24.200 three or four nested callbacks 350 00:16:24.200 --> 00:16:25.920 gets turned into this 351 00:16:25.920 --> 00:16:29.520 which I hope you'll agree is much simpler 352 00:16:29.520 --> 00:16:32.240 much easier to read much easier to reason about. 353 00:16:32.240 --> 00:16:36.000 And if you're morbidly curious 354 00:16:36.000 --> 00:16:40.160 you can you can expand your macros 355 00:16:40.160 --> 00:16:44.200 and this invocation of LMPD chain expands to this. 356 00:16:44.200 --> 00:16:46.400 So that's my story. 357 00:16:46.400 --> 00:16:50.840 In all fairness I should note that 358 00:16:50.840 --> 00:16:55.160 the MPD protocol has some subtleties and complexities 359 00:16:55.160 --> 00:16:56.880 that I didn't really get into 360 00:16:56.880 --> 00:16:58.360 both due to time constraints 361 00:16:58.360 --> 00:17:00.520 and because they're not terribly relevant 362 00:17:00.520 --> 00:17:02.000 to the points I wanted to touch on 363 00:17:02.000 --> 00:17:05.360 I should also note that there's 364 00:17:05.360 --> 00:17:07.720 a fair amount of work in the library itself 365 00:17:07.720 --> 00:17:11.240 around accumulating partial responses 366 00:17:11.240 --> 00:17:12.560 as they show up in the buffer 367 00:17:12.560 --> 00:17:16.120 and dispatching them piecemeal to the caller 368 00:17:16.120 --> 00:17:19.720 that was really too complex to get into here. 369 00:17:19.720 --> 00:17:22.360 If you would like to see the code 370 00:17:22.360 --> 00:17:25.080 it's available on GitHub as well as Melpa. 371 00:17:25.080 --> 00:17:29.200 I'll be putting a version of this talk 372 00:17:29.200 --> 00:17:30.480 on my personal site 373 00:17:30.480 --> 00:17:33.720 and you can always reach out to me personally 374 00:17:33.720 --> 00:17:36.960 I hang out on IRC as SPIF 375 00:17:36.960 --> 00:17:41.920 or you can just email me as SPIF at P.O.Box dot com. 376 00:17:41.920 --> 00:17:47.880 Thank you very much.