[[!sidebar content=""]]
[[!meta title="Emacs was async before async was cool"]]
[[!meta copyright="Copyright © 2022 Michael Herstine"]]
[[!inline pages="internal(2022/info/async-nav)" raw="yes"]]

<!-- Initially generated with emacsconf-generate-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->


# Emacs was async before async was cool
Michael Herstine (IRC: sp1ff)

[[!inline pages="internal(2022/info/async-before)" raw="yes"]]

While async is all the rage in the JavaScript, Go and Rust
communities, Emacs has had the ability to (certain) asynchronous
processing all along. I learned this while writing an Emacs package to
program to my music server's API.

Music Player Daemon is a project that does what it says: hangs out in
the background & plays your music. It has an API that I wanted to call
from Emacs. I had just learned async Rust, so I decided to see if I
could do something similar here. It turns out I could: I wound-up
writing a little async runtime that allows callers to "send" commands
to the daemon by queueing them up and returning immediately. In the
background, as output from previous commands comes in, new commands
are dequeued and sent.

This required callers to provide callbacks in order to receive the
results of their commands, which led to a situation all too familiar
to Javascript programmers: callback hell. The coda to my story was
(finally) coming to understand Lisp macros in order to extend Emacs
Lisp with a more convenient construct for sending multiple commands.

This talk will dive into the details of the Emacs Lisp process API,
specifically the Low-Level Network Access API, with illustrations as
to how it can be used to build an asynchronous queue. It will also
introduce Lisp macros and how powerful they can be.

Michael is a developer and long-time Emacs user from the San Francisco
Bay area. He hacks in C++, Lisp & Rust and thinks a lot about writing
provably correct code. You can find him at:

  - his [home page](https://www.unwoundstack.com)
  - on IRC: sp1ff on Libera.Chat
  - through [e-mail](mailto:sp1ff@pobox.com)
  - or on [Github](https://github.com/sp1ff)

# Discussion

-   Q: (Referencing the first half of your talk): How does this approach
    compare to using tq.el, Emacs' built-in library for transaction
    queues?
    -   A: Great question; should have mentioned that... I took a look,
        but chose to just do it "by hand"; I wouldn't have used many
        of the features offerred by tq.
-   Q: Have you considered using the aio.el library (written by Chris
    Wellons) that implements async/await for Emacs lisp using promises?
    It's implemented using Elisp's record data structure, and turns
    the nested callback structure into regular-looking Elisp code
    (without extra keywords etc). +1
    -   A: I wasn't aware, but thanks for the pointer-- will
        definitely take a look
-   Q: not to take away from your excellent work, but are you aware that
    EMMS has an MPD client? There's also mpc.el built into Emacs.
    -   A: Another great point; I am, along with mpdel (another MPD
        client for Emacs). They are all full-fledge applications-- I
        just wanted a small, tight toolkit
-   Q:Have you seen the Lonesome Pine Specials? I saw your music library
    and figured you would be interested. My favorite is the one with
    Edgar Meyer, Sam Bush, Jerry Douglas, and I think Bela Fleck and
    Mark O'Connor?
    -   A: LOL I haven't, but I I think I will be!
-   Q: can you share the code to the macro that creates the callback
    tree?
    -   A: <https://github.com/sp1ff/elmpd/blob/master/elmpd.el#L898>
        -   thanks!
-   Q: would using dynamic/special vars add anything interesting /
    easier to async elisp in your opinion? i noticed you using `let` a
    lot, but if you defined a variable hmm... not sure if i can :)  i
    was just wondering if having dynamic binding in Elisp opposed to
    something like JS adds some power to async programming
    -   A: lexical binding is easier to reason about :)
-   Q: There's another package (chuntaro?) in addition to wellon's aio
    that also implements a coroutine trampoline on the emacs event loop.
    any thoughts on the async/await paradigm generally red/blue
    functions, etc?
    -   A: Longer discussion in the chat room, but I think it's a
        promising if over-used approach to concurrency.
- Q: How does your project compare to some of the other MPD clients?
- Q: Any thoughts on the async await paradigm generally, red-blue functions, etc.?
- Q: Do you think it's a viable future for Emacs to get out of callback hell?

Comments from YouTube:

- Thank you for the informative video Michael! Now I know a way to avoid callback hell in Emacs lisp!
- Nice talk, thank you Michael!

[[!inline pages="internal(2022/info/async-after)" raw="yes"]]

[[!inline pages="internal(2022/info/async-nav)" raw="yes"]]