The Sound of Emacs, Emms, The Emacs Multimedia System.
Hi, I'm Yoni Rabkin and I'll be talking about Emms;
the Emacs Multimedia System.
What is Emms?
Emms displays and plays media from within Emacs
using a variety of external players
and from different media sources.
Emms can run as a minimalistic player
which is controlled with no more than
a handful of simple M-x commands,
or as a fully-fledged interactive media browser and player.
Emms can display album art, play streaming audio,
tag music files, search for lyrics,
provide MPD connectivity, control the volume,
and more. Much more.
The Emms project acts like Emacs in microcosm.
It slowly but surely grows bigger
and gets ever more features.
Perhaps Emms will one day even have a text editor.
The structure of this talk:
We'll start with an introduction to Emms.
This is the practical part.
Then, a bit about how Emms works. That's the technical part.
Finally, how we work. All about Emms development.
Introduction to Emms: The practical part:
I want this talk to be of immediate use to people,
so I'm going to present a quick TL;DR of the Emms manual
concerning installation and use.
By the end of this part you should be able to
install, configure, and use Emms in a variety of ways.
Where can I get Emms?
Emms is distributed primarily via GNU ELPA.
So it's really only a M-x list-packages away at any moment.
There's also a website hosted at gnu.org.
Among other things on the website, you'll find
a copy of the friendly, robust, and up-to-date user manual.
Installing Emms has become progressively easier over time
and will continue to get easier.
In the bad old days, it required downloading a tarball
and compiling a C language shim
to enable reading metadata from media files.
But those days are long gone, and installing Emms is now
as easy as invoking M-x list-packages,
installing the Emms package, and placing as few as
2 or 3 lines of configuration in your Emacs initialization.
So after the package is installed via ELPA,
you can add these few lines.
`emms-all` will make available all of the stable features
which are shipped with Emms.
The `emms-player-list` variable is a list of players
like MPV, MPlayer, VLC, etc.
Emms will call and control these external players
to play your media.
The variable `emms-info-functions` is a list of ways
for Emms to read the metadata in your media files
so that Emms can display song title, artist name,
year of production, etc.
The `emms-info-native` feature in the setup example
is the built-in metadata reader
written entirely in Emacs Lisp.
But there are also other backends
which can call external programs for info
such as TinyTag, the TagLib library, exiftool, and so on.
You can then old-school restart your Emacs
or simply evaluate the above couple of lines to get going.
Now that we have Emms installed and configured,
we should load some media for player.
There are multiple ways to load media into Emms for playing.
They can be directories with local files,
synchronized from a remote instance of
a music player daemon, PLS or M3U playlists,
a list of URLs for streaming,
or even Emms' own native playlist format
which is unsurprisingly a just serialized Emacs Lisp.
No matter how you add tracks to Emms,
you'll end up with a playlist.
A fundamental strength of Emms is that each playlist
is a regular Emacs buffer and the track listing therein
is nothing more than text lines with property overlays.
This means that you can navigate, search, copy,
and edit an Emms playlist buffer
just as you would any Emacs buffer.
If you want to reorganize the tracks in the playlist,
then you can simply kill yank the tracks
just as you would any buffer with lines of text,
and the same can be done between multiple playlist buffers.
One of the most straightforward ways to add media
is to invoke a command like `M-x emms-add-directory-tree`.
You can point it to the top of a set of directories
with playable files for Emms to traverse.
Another rather convenient method is to mark files in Dired
and to invoke `emms-add-dired`.
I definitely use this one a lot.
The Emms playlist mode binds
a number of useful keys and commands.
It's highly recommended that you either
read the friendly manual
or hit "C-h m" in a playlist buffer to discover them.
Now we have a playlist buffer with a number of tracks,
so the next step is going to be playback.
Emms can be used as a minimalistic player
with nothing more than a handful of commands.
Once there is a current Emms playlist,
invoking emms-start will begin playing the current track.
Now of course in a new playlist
that would be the first track.
Now emms-next, emms-pause, and emms-stop
do exactly what you think they do.
To visit the current playlist,
you can invoke M-x emms-playlist-mode-go,
which is a long command I personally bind to "M-f12".
You'll be taken to the current playlist buffer.
While you can have multiple playlist buffers,
only one is current for the purposes of playback commands.
The playlist buffer has keys bound
to control the media being played.
`emms-seek-forward` and `emms-seek-backwards` allow you
to scrub along the media being played.
Which commands are available is a function of
the player backend being employed.
The simplest of players may have nothing more
than the ability to play, stop, and seek,
but others may implement a plethora of commands.
The Modeline: Emms will by default display
the name of the currently playing track in the mode line
with information such as playing time.
The mode line format is controlled
via the `emms-mode-line-format` variable
and the `emms-mode-line-playlist-current` function.
Metadata and the cache.
It would be sufficient for emms to simply list
the file names or urls of each piece of media,
but unless you name your music and media
with obsessive consistency and precision,
not that there is anything wrong with that
then the resulting list will be a bit of an eyesore.
Moreover, there are a lot of other useful metadata
in the media files, including cool stuff like album art.
So instead of just files, Emms will try
to extract metadata from each track
and display a nicely-formatted track listing.
The format can be controlled by customizing
the variable `emms-track-description-function`.
Emms uses so-called info methods to extract
the metadata from each file.
`emms-info-native`, which I mentioned before,
is the built-in metadata reader written in Emacs Lisp.
It provides support for Ogg Vorbis, Ogg Opus, FLAC, and MP3.
However, if you have media in other formats,
you can also add info methods
to the `emms-info-functions` list,
which call external programs such as exiftool,
the LibTag library, tiny-tag, etc. to read file metadata.
Since reading metadata takes time
and that metadata doesn't change very often,
Emms builds a cache as it extracts
the information from each file.
The first time loading of thousands of tracks
into the emms cache may take a while,
but as is the nature of caching, subsequent loads
will be nearly instantaneous.
To ease loading huge media collections,
emms also can populate the cache asynchronously,
so that your emacs isn't locked up in the interim.
Let's talk about streams and URLs.
Not all playlist entries need to be associated with files.
It's possible to add streaming playlists
and URLs to any playlist.
Emms also comes with a built-in eclectic list
of streaming audio stations to get you started.
Any playlist entry can be a URL,
and that URL will be passed on to the media player backend,
which can play it, if any.
Meta-playlist mode:
Emms also has meta-playlist mode
to help manage multiple playlists.
When you invoke meta-playlist mode,
you will see a listing of all of the current Emms playlists,
and this mode binds a handful of useful keybindings
to help manage those playlists.
The Browser:
Music doesn't always lend itself to being viewed
as a series of discrete files.
While there may be a good taxonomy of music
that can be reflected using directories and filenames,
there are other aspects which cannot.
This is especially true when you consider that
unlike many computer file taxonomies,
music files may contain
a lot of self-descriptive information
in the form of metadata,
such as the year a work was published, the composer,
the performing artist, etc.
Therefore, it makes sense for Emms to enable
a different view into a media collection
which is based on the cached metadata.
The browser interface binds a host of keys
to help navigate the tree structure
of the metadata information.
Since browser display
is not predicated upon directory structure,
you can invoke functions such as `emms-browse-by-album`,
or `emms-browse-by-artist`, etc.
to view the collection in different ways.
Emms can do a lot more,
but covering it all would take too much time.
I do recommend opening the fine Emms manual
and getting to know some additional features
such as sorting tracks in playlists,
sorting and filtering in the browser,
editing track information,
deriving a new playlist from an existing playlist,
the music player daemon, lyrics display, volume control,
bookmarks, GNU FM, and Dbus/Mpris support.
I hope this was a useful introduction to Emms.
How Emms Works: The technical part:
This part is an overview of how Emms works.
By the end of this, you should be familiar enough
with Emms internals to hack on it. Hint hint.
A short history of Emms
Emms is 20 years old as of the time of writing.
Old enough to drink in many countries.
This means it was developed back in 2003
for emacs 21.2 or thereabouts.
As developers, we don't go around looking to
replace code just because it's old.
On the other hand, some parts were inadequate
or just didn't age gracefully.
And we have been partially or completely rewriting those.
I became the maintainer of Emms about a decade ago,
but I didn't start the project.
Jorgen Schäfer started the project.
I reached out to Jorgen
and he kindly shared some of his recollections.
Jorgen states that Emms was born back
when the music format wars raged.
MP3 was the standard, but overshadowed with patent issues.
In fact, Technicolor and Fraunhofer IIS
only stopped licensing their patents for MP3
as recently as April of 2017.
Jorgen said that, and I quote,
"I needed a tool that was player agnostic
and that could deal with a large collection of music files.
And I did not want any of the GUI music players
that existed back then.
Primarily, actually, because I did not want
to be switching windows to skip to the next song.
If I remember correctly,
I had just a shell script before that.
But I figured I lived in Emacs, so why not write a tool
that I can control my music from Emacs
without ever having to leave Emacs?" Unquote.
We can see that Jorgen's motivations were of the best kind,
to stay in Emacs.
Emms, an architecture of sensible abstractions.
Emms can be divided into a number of parts.
The core, tracks, playlists, sources, players,
info, cache, and ancillary.
Now David J. Wheeler once said
that all problems in computer science
can be solved by another level of indirection,
except of course for the problem
of too many layers of indirection.
Emms core has survived this long
because it makes sensible and flexible coding abstractions.
Keep this in mind as we explore the implementation.
This following part of the talk will also be invaluable
if you want to hack on Emacs.
Another hint.
The Emms core.
The core defines tracks, playlists,
a way to start and stop playback,
as well as ways to proceed to the next track.
Tracks:
Emms tracks consist of a list whose CAR is the symbol track,
and CADR is an alist starting with
the association of `type'.
Type can be something like file, streamlist, URL, etc.
A track of classical music from Bach's Art of Fugue
may look something like this.
While a track may contain many associations,
the number of associations remains a small constant
from the perspective of computational steps required
to find any particular association.
Playlist:
An Emms playlist consists of an Emacs buffer
with a buffer-local non-nil variable,
`emms-playlist-buffer-p`.
The buffer can contain anything, any amount or type of text,
or anything else.
Emms tracks are stored in text properties within the buffer,
with the unimaginatively named text property `emms-track`.
For Emms, to go to the next track consists of
nothing more than looking for the next text property change
containing `emms-track`, wherever that is.
That means that there is a healthy decoupling between
the visual representation of a playlist
and its contents as far as Emms is concerned.
This decoupling allows Emms playlist buffers
to look like anything as long as that anything consists of
one or more `emms-track` text properties.
Sources:
A source is how you tell Emms:
"Go and get those things and turn them into tracks."
More specifically, an Emms source is a function called in
a playlist buffer in order to add tracks.
And even more specifically, a source is really
a family of related functions
defined by the macro `define-emms-source`.
A straightforward example
is the function `emms-add-directory`,
which adds an entire directory of files
to the current playlist.
It accepts, or interactively queries for, a directory
and iterates over each file in that directory,
adding them as tracks to the playlist buffer as it goes.
Emms comes with sources for files, directories, URLs,
playlists of various formats,
files from dired mode, and etc.
Players:
An Emms player is, at its simplest, a data structure
with three functions.
One to start playing, one to stop,
and one which returns true if the player knows
how to play a given track.
However, if your player also knows how to pause, resume,
seek, etc, then additional functions can be added
to the player data structure.
This is abstract enough to be able to, for example,
define a simple player for images with the help of
the `define-emms-simple-player` macro.
The above will define a player called `emms-player-display`,
which would call ImageMagick's `display` command
on each file in our playlist
with the image file extension we listed.
Info:
As previously described, Emms comes with info methods,
which are functions to add
descriptive information to tracks.
Emms is set up so that
the hook `emms-track-initialize-functions` is called
when a track is created, and that ends up calling
the info methods listed in the `emms-info-functions` list.
These will modify the track data structure to add metadata.
One of the coolest recent features of Emms
is `emms-info-native`, written by Petteri Hintsanen;
again, sorry for the pronunciation.
`emms-info-native` is a purely Emacs Lisp implementation
which reads Ogg Vorbis, Ogg Opus, FLAC, and MP3 files
and parses out the metadata.
This is in comparison with other info readers
which Emms supports, which all involve calling out
to external processes and parsing the values returned.
`emms-info-native` works by unpacking and examining
the binary data in the media file headers
and parsing the data layout specifications.
The Cache:
The Emms cache is a mapping between a full path name
and its associated information.
Once information is extracted from a file
using an info method, that information is then
associated with that file in the cache.
One thing to bear in mind is that the caching system
was originally written back
when slow spinning disks were common.
A 32GB SSD drive cost close to $700 in 2006,
which is the equivalent of about $1,000
at the time of writing.
But despite the speed of modern drives,
the caching system is still worth using
for larger music collections.
The caching system is also a prerequisite
for being able to use the Emms browser.
The cache implementation is relatively naive.
For instance, moving a file will invalidate
that cache entry for that file
and will require a refresh.
However, relatively little work has been done
to the cache implementation over the years
since it has proven to be good enough
for the majority of situations.
Which is to say, nobody complained.
Healthy back and forth. MPV, MPD, GNU.FM
Process communication with a simple media player
can be as straightforward
as starting an asynchronous process
and waiting for that process to complete
in order to move to the next track.
This is how the example above
with ImageMagick's display binary worked.
However, Emms also handles asynchronous
two-way communication with processes.
A simple example of this would be sending strings
to a running process such as the pause command to VLC.
MPV:
MPV is a popular media player forked
in a roundabout way from mplayer.
One of its most notable features is
support for a robust client API.
Mike Kazantsev has been working since 2018
to develop the excellent `emms-player-mpv.el'.
It can communicate with a long running MPV process
via Unix sockets or IP sockets.
This allows for MPV to do things
like update ICY metadata for streaming audio.
So that, for example, when a song changes
while you're listening to a streaming audio via Emms,
the song title displayed in the mode line
and track listing can update as well.
This means that deep inside the code
there is an Emacs `make-network-process` call.
The fact that Mike has put this together
in fewer than 1,000 lines of legible Emacs Lisp
is a testament to some serious coding ability.
MPD:
Similar to MPV but potentially
on a completely different machine
is Emms support for the Music Player Daemon.
Music Player Daemon or MPD is a media player
with an explicit client-server design
and communicates with Emms via a network process.
Unfortunately, MPD support has never been all that great.
But this isn't the emms developers fault!
Because unlike every other media player
that Emms interfaces with MPD is designed around
its own internal playlist database.
This is a surprising design decision
on the MPD developers' part
since it goes against the client-server mindset.
A consequence is that we end up having to try and coordinate
and harmonize the MPD playlist with the Emms playlist.
I can foresee writing a completely new MPD mode for Emms
which is designed to be a true pure MPD client.
Unless of course someone volunteers to beat me to it.
Hint hint.
GNU FM and Libre FM:
Libre FM is a music community which allows you
to share your listening habits with other users of the site.
A kind of online listening party.
In the case of `emms-librefm-scrobber.el`
we use Emacs' `url-retrieve` function
to asynchronously send to a URL
and then fire a callback function to process the response.
This represents numerous challenges
to implement within Emacs.
The primary issue being that Emacs itself
is pretty weak at doing anything
truly and really asynchronously.
I can say with confident sarcasm
and with tongue firmly planted in cheek
that it is almost as if the original designers
of Emacs didn't foresee their text editor
needing to play music
while interacting with a remote network server.
How myopic!
How we work: Emms development:
This part is an overview of how Emms is developed.
By the end of this part you should be able to understand
how we hacked this project, and how you can too.
Where it's at.
How to find our forge.
Emms has been hosted at the FSF's forge, Savannah,
since around 2003.
Emms is distributed via GNU ELPA and integrated into Emacs.
Before ELPA it was distributed as a tarball
via ftp.gnu.org but that stopped back in 2020.
I was initially resistant to ELPA but around the time
when the thousandth person asked me why Emms isn't on ELPA,
I realized that it had to happen.
Emms can also be found in other places
such as Melpa or GitHub but we, the developers of Emms,
have nothing to do with that
and we don't monitor those channels.
If you want the source straight from, well, the source,
then go to the Savannah Git repository.
Look who's talking: Where development discussion happens.
If you want to talk to us, discussions all happen
on emms-help@gnu.org.
We used to use emms-patches@gnu.org
but didn't feel like the volume of incoming patches
justified a separate mailing list.
The Rime Of The Ancient Maintainer:
There are a number of activities
particular to being a maintainer.
These are all part of a project's lifecycle.
Let's review some of them.
The life and times of an Emms patch:
A maintainer needs to be able to accept, critique,
and integrate patches from contributors and developers.
This means, among other things, that the maintainer
needs to keep on top of copyright issues.
Before being able to add Emms to GNU/ELPA,
we had to make sure that the copyright situation
was in order.
This long process required reaching out to people
and having them assign the copyright
for their work to the FSF,
or even removing their code entirely
if they couldn't be reached.
The experience left me with the conviction
that the easiest way to fix
the copyright situation of your package
is to ensure that it never gets broken in the first place.
Often a person will write in to the emms-help mailing list,
or perhaps raise an issue on IRC.
If it's a bug report or feature request, we'll discuss it,
and when it's fixed, we'll ask the reporter
to test the result and provide feedback.
If it's a patch, then we'll typically go one of three ways.
A trivial patch, such as fixing a typo
or corrections on a single line of code,
will simply be applied by one of the developers.
A non-trivial, but one-time patch,
will have to be cleared from a copyright perspective.
This means assigning copyright for the changes to the FSF.
Once that's cleared, then the patch will be applied.
Finally, if it's a non-trivial patch,
which looks like it would be the start
of a long-term development work (my favorite),
then after copyright is cleared,
that person will be offered to be added
to the members with Git repo access on Savannah.
From there, we usually use a dedicated branch
to do all the playing around
before merging it with the main Git repo.
If you have ever sent a patch, feature request,
or bug report into Emms (small or large), we thank you.
Let It Go, The Release Process:
The maintainer is responsible for the release process.
I found that a consistent schedule works well,
which is not to say that we have to release on schedule,
but that aiming for a consistent release schedule
provides structure and a goal.
The main Git branch in the repository is stable
and more often than not of release quality.
Releases are done about every three months.
And with such a stable main branch,
the process of releasing often involves little more
than writing a NEWS entry.
As a consequence, new and wonderful features
which aren't quite ready for prime time
when a release comes around,
will remain safely in their branch on the Git repo
until after the ELPA release.
It Is Not In Our Stars, But In Ourselves; Future Directions:
One aspect of Emms that needs to improve is ease of setup.
Now that might surprise you, since at the time of writing,
it's already pretty easy.
But my ideal is that the user would need to do
nothing at all after installation.
And with that, as a goal in mind,
there is more work to be done.
We are working on a player discovery feature.
The idea is simple.
The code looks for binaries of popular media players
on the user's machine,
and for each one found, it asks the user
if they want the associated Emms player backend
to be configured.
In effect, this code is already working,
but currently an undocumented, unofficial feature.
You can try it for yourself with
`emms-setup-discover-players`.
So what's the holdup?
`emms-setup-discover-players` currently configures
the `emms-player-list` variable,
but doesn't write it to disk.
And that means that the configuration
isn't preserved between Emacs sessions.
The question then becomes,
what is the best way to preserve this setting?
I personally don't like anything
to edit my .emacs except me,
and I wouldn't do that to anyone else.
Now we already write state to the .emacs.d/emms/ directory,
but that would require care not to
clobber a user's existing setup.
Having the user set up their system in one place,
such as a .emacs or a .emmsrc,
while saving state to a different place
is asking for confusion.
This is a good example which I bring up
of where a maintainer needs to
solicit opinions from developers,
both the Emacs developers,
asking them where packages should save state,
and the Emms developers, and also users.
Then, the maintainer needs to
carefully choose a path forward.
It is typical of the kind of issue you have to have in mind
when you're maintaining a package.
Development Policies: Interface Language.
A maintainer of an interactive program such as Emms
needs to think about user interaction.
Emms doesn't use key bindings which are familiar
to people who are used to GUI media players,
and that can, and has, caused friction.
Some new users are confused when they press the spacebar
on an entry in the Emms browser,
only to find that nothing starts playing.
Indeed, all that does is to expand the browser tree
at that point.
Then they might press RET on the same entry,
and be further frustrated at the continuing silence.
Since what return does is just to add that entry at point
to the current playlist.
The discussion then arises
about how Emms should handle that situation.
On one hand, we want to make it as easy as possible
for new users to learn Emms,
and adopt a do-what-I-mean interface approach.
On the other hand, this is an Emacs project.
It isn't a stand-alone GUI media player,
and should integrate into Emacs,
and serve Emacs users first and foremost.
Development policies: Freedom.
Another maintainer job is to think of Emms' posture
in regards to software freedom.
Here are a few examples.
Back with MP3 was still a patent encumbered format,
we pushed hard for Vorbis everywhere
along with the PlayOgg campaign.
A then popular music streaming service,
which will remain unnamed,
changed their stance towards third-party applications,
and required individual API keys which could not be shared.
We stood firm, said "no",
and removed support for that service.
A recent suggestion to add support for YouTube
was also nixed,
because the particular backend
was found to download and run proprietary javascript
on the user's machine.
Saying no to potentially useful or wanted features
because it involves non-free software
is often an unpopular decision and can alienate people.
A maintainer needs to think carefully
about each of these decisions,
as they are rarely straightforward and one-sided.
And as you see above, they also change over time
and need to be re-evaluated.
One of the most useful things a maintainer can do
is to coordinate the development effort
and help new people join the project.
In light of that, if you want to work on a project
which has a bit of everything,
you could do worse than hacking on Emms.
There is inter-process communication,
displaying graphics, parsing binary files,
caching, asynchronous processes, user interface design.
We also are a project that insists on
keeping a well-written and up-to-date manual.
If you can write English or hack Emacs Lisp at all,
chances are that there is something you can do for Emms.
Just saying.
Acknowledgements:
I'd like to express my deep gratitude for all of the people
who have hacked on Emms
during my time as a maintainer and before it.
It is often the case that I'm just the person
holding the rudder and steering the ship,
with all of these developers
rowing furiously to provide the power
which actually moves the ship forward.
Thank you to all.