Welcome to Emacs as a Shell, a talk by Christopher Howard for
Emacs Conference 2024.
In this talk, I would like to explore, or advocate for, a
particular perspective. I want to encourage people to
think of Emacs not as simply an editor or a development
environment, but rather as a shell, or at least something
that allows us to do most of the things that we might
otherwise want to do from a shell.
What do I mean by shell? By
shell, I mean basically an interface that allows us to
interact with the rest of our system by entering commands.
That definition is, perhaps, a little too broad, and so I
will try to narrow it down with a list of features that,
historically, we have come to expect from a shell. The Bash
shell is one very portable and well-known shell, and for
many of us it is maybe the prototypical example. But in the
past there have been many other shells, and there are other
shells available today. If we are willing to be flexible in
our thinking, we can think of Emacs as being a shell, or at
least providing most of the functionality that we expect
from a shell.
Before further expanding on this idea, I must emphasize
what I do not mean. First of all, I am not talking about
running Bash, or some other external shell, from within
Emacs, although this is certainly possible. I am arguing,
rather, for using Emacs as a shell, instead of other shells.
Second, I do not mean running a terminal emulator from
within Emacs. Emacs has a built-in terminal emulator, but
this is not what I mean. A terminal emulator is essentially a
program designed to control the cursor and text appearance
in response to various control codes in order to mimic a
terminal display device. There are certainly legitimate
reasons to do this. Nevertheless, in general, it does not
make much sense to run a terminal emulator within Emacs,
because Emacs has its own commands for controlling the
cursor and text appearance. Also, due to the way Emacs was
designed historically, Emacs itself believes that it is
running on a terminal. So you end up with layers upon layers
of terminal emulation. Anyhow, at the end of the day, Emacs
will not perform as well as a dedicated terminal emulator
program. I also think that, as we try to force ANSI terminal
emulation into our Emacs workflow, this ultimately will be
a hindrance to us in taking advantage of the natural and
pleasant interfaces that are already available to us
within Emacs. In brief, if your goal is simply to figure out
how to be able to do all your normal Bash command line
wizardry from within an Emacs window instead of a GNOME
console window, you are headed down a different set of train
tracks than I am. Also, something which I fear may confuse
the issue for some viewers is the fact that Emacs ships with
its own unique built-in shell, called the Emacs shell, or
Eshell.
Eshell aims to be a legitimate shell, and provides a very
similar experience to other shells like Bash, while being
well integrated into the Emacs interface, and without
giving up the power of the Emacs Lisp engine. Eshell will be
mentioned multiple times in this talk. The entire talk
could, in fact, be about Eshell, except that I want the talk
to cover all aspects of Emacs shell-like functionality
through its other tools, such as interactive commands and
special modes. So, hopefully we can keep distinct in our
mind the ideas of Emacs as a shell versus the Emacs shell,
though the latter is an important part of the former.
Let's get back to the fundamental idea of what is a shell. In
the broadest definition, a shell is an interface which
allows you to interact with your operating system through
commands. However, from a historical perspective, there
are a few basic capabilities which we expect to be part of
every shell.
First of all, the shell provides a means of launching
external programs. Some internal or built-in commands
might also be made available. Second, the shell provides a
means of managing environment variables. In the past,
environment variables often played a critical role as a
means of passing in options, file names, device names, and
suchlike to external programs. This is not quite as common
today, but the environment still plays a critical role in
managing things such as the path to executables and
libraries, as well as various other user, desktop, and
system settings. The shell modifies the environment and
passes it on to external programs.
Historically, job control was expected to be either a
function of the shell, or easily accessible from it.
Usually today, our personal computing is not
batch-oriented. But typically, shells can run multiple
processes simultaneously, as well as provide means to
suspend and terminate processes, which are useful
features. Shells should be able to redirect and pipeline
process input and output. This allows the user to connect
process input and output with files, devices, or other
processes. Finally, shells are expected to have some
limited scripting capability, such as the
POSIX-compliant set of program statements and
conditionals that Bash provides.
As command-line wizards, there are a number of tasks we
expect to be able to do quickly and easily from our shell,
even though these tasks are not the domain of the shell
itself. A common task is file management and navigation. We
quickly navigate and manipulate the file system with
standard utilities that do things like change the current
working directory, rename files, move files, and delete
files. We usually expect to have access to some additional
process management utilities. These allow us to do things
such as find out the status of all processes running on the
system, and send signals to processes. Finally, we expect
to have access to some basic networking utilities. For
example, we should be able to run commands that set up
network interfaces, ping computers, and download files.
With a little reflection, we can see that Emacs can provide
all or nearly all of the functionality we have described so
far. And the functionality can be called conveniently
through one of several methods.
Either a normal interactive call, like M-x something, or a
call to an elisp function,
or through Eshell commands, or through some special buffer
mode, such as the directory editor, which provides its own
interface to some functionality.
It may be going too far to say that Emacs is a full replacement
for shells like Bash. Nevertheless, we can see that Emacs
can do most of the things that we might otherwise do with our
shell.
Let us give some examples. First, can Emacs launch
external commands? Of course. Now, there are something
like a half-dozen different ways to do that within Emacs, and
some are more convenient than others. From any Elisp
program, we can call functions like make-process and
call-process to launch external processes.
These, however, generally are not convenient for quick,
one-off commands. Another option would be to run Eshell,
which would allow us to call the external program from a
familiar command line prompt.
If we do not actually want to drop into Eshell just to run one
command, we also have the interactive command,
eshell-command,
which would allow us to call the external program from a
familiar command line prompt.
If we do not actually want to drop an
Eshell just to run one command, as I just mentioned, we also
have the interactive command eshell-command, which allows
us to enter in a one-off command and run that immediately.
And finally, there is also an interactive command called
simply shell-command.
Shell command is like Eshell command, but instead passes
the command off to our system shell, for example, bash.
This is cheating, of course, but it might be useful or convenient
in some scenarios.
Regarding environment variables, Emacs can read and
manipulate the environment variables, which in turn get
passed on to processes which it launches. The
general-purpose interactive commands for this are
getenv and setenv. These commands
deal with the one environment that is available throughout
all parts of your running Emacs session. In other words,
these functions deal with a global environment, which is
the same wherever you are running getenv or
setenv.
An important exception is that every instance of Eshell
maintains a distinct environment that will not be affected
by setenv calls run in other buffers. Also, Eshell
has some additional syntax for dealing with its
environment, including the set and export syntax.
Regarding job control and process management, Emacs does
not provide job control in the way that Bash users are used
to. We can, however, launch asynchronous processes, and do
various things to them. From Eshell, or an eshell-command
call, we can append the ampersand symbol to the command, and
this will cause the process to run asynchronously in a
dedicated buffer.
Now, if the command is launched from Eshell, it will not
actually run in a separate buffer, but the output will go to
the Eshell buffer.
We can run the interactive command list-processes to see
all the processes running for our current Emacs session.
In Eshell, we can run the command "jobs" to get the same list.
This will show the process name, process buffer name,
process ID, and some other information. We can select the
process buffer in the process list to bring up that process
buffer.
We can also use the interactive command signal-process to
send any signal to a process, including "stop" to suspend the
process, "continue" to resume the process, and "interrupt" or
kill to terminate the process.
Regarding redirecting and pipelining input and output,
Eshell does support redirection similar to Bash, so you can
overwrite and append to files and some other objects. Input
redirection is not yet implemented, but it is on the Eshell
to-do list. Eshell also has pipes. The default pipe, which
uses the familiar vertical bar symbol, pipes the data
between the commands using an intermediate Emacs buffer.
This, while usually quite practical, is less efficient
than the system pipe. Therefore, Eshell also makes
available a star-modified version, which uses the system
pipe through a call to your system shell.
So we can do things like
direct output to a file.
We're unfortunately not able to do input redirection, but
we can use pipes.
Elisp can manipulate and tie together processes in various
ways, such as process filters and pipe processes, but I
won't attempt to cover that. I feel like you should mention
again that we have two kinds of pipes here available. So this
pipe, the standard one, will pipe the data through Emacs
buffers. That's very practical in most cases, but it is less
efficient than piping through the system pipe. So Eshell
makes available another symbol for that, star, vertical
bar, that allows you to explicitly use the system pipe.
Regarding scripting: Of course, using Emacs makes
available all the power of the Elisp API and third-party
packages, so we have that out of the gate. Eshell also has
control flow statements, like an "if" construct and a "for"
construct. See the Eshell info manual, section 3.7, for
more details.
And if you wish to write a script entirely in Eshell syntax,
and store it in a separate file, this is possible with recent
versions of Emacs.
Here's an example of a brief script that I wrote.
Unfortunately, an eshell mode for proper syntax
highlighting is not yet available, but hopefully that will
be forthcoming. Note that Eshell syntax allows elisp
forms to be interspersed with regular command form for
additional scripting power. We will discuss this a little
more later.
Regarding file system management. In Emacs, many of the
common file system operations are available as
interactive commands. For example, M-x cd, to change your
buffer's current working directory, and other M-x commands
such as make-directory
chmod,
and delete-file. Of course, you can also drop into Eshell,
or use M-x eshell-command to run the usual external commands
for file system manipulation. Also, a file manager is built
into Emacs, which can be run by calling M-x dired.
The directory editor is powerful, but it is a bit strange to
folks expecting something like Midnight Commander or the
GNOME file manager. It gives us a number of helpful features
like the ability to mark files, and to run elisp functions on
them, and some other interesting ways to manipulate and
rename the files. However, third-party Emacs extensions
such as Midnight Commander Mode and Sunrise Commander are
available to provide a Midnight Commander experience, for those who
prefer that sort of file management.
Emacs also has the nifty TRAMP functionality built in,
which allows you, most of the time, to easily edit files on
other computers, as well as manipulate the file system.
This transparently works through SSH and some other
protocols that you can specify.
Regarding networking features, I don't have a lot of
interesting things to say about this at the present, so I'll
skip through this quickly. But if you do a little research,
you will see that Emacs has a lot of functionality relating
to making network connections, interacting with the web,
and such like, both built-in and in available packages, as
well as modes for doing things like Web browsing and Gemini
browsing. And of course, you can run the usual standard
networking commands for your system through Eshell.
So having put forward the main arguments for this talk, I
would like to take some time now to give a brief tour of a few of
the features of Eshell, the Emacs shell. It bears
emphasizing that Eshell is not a drop-in replacement for
Bash, or even a Bash clone, though I believe the developers
are trying to make much of the syntax very similar. Also,
Eshell is not a terminal emulator, and it will not display
correctly applications which use advanced ANSI control
codes. However, Eshell can be configured to be aware of such
applications, and to run them automatically within the
Emacs terminal emulator when launched. See section 5.1 of
the Eshell manual titled Visual Commands.
Though Eshell is not Bash, it has multiple features,
pertaining mainly to its by-design Emacs integration,
which may make it more appealing to use than Bash or another
shell.
For one, Eshell allows entering commands on the command
line that are space and new line separated, without
parentheses. Of course, all the other shells do this. But
within Eshell, it is possible to enter internal Emacs
functions, as well as external commands.
This allows us to do things like this.
As far as I understand, it is possible to enter any Emacs
function on the Eshell command line. However, some special
syntax may be required if you are trying to pass in something
that is not a string or a number.
As you might have noticed in the last example, Eshell makes
it possible to use an Emacs buffer as a sink for output. It
also allows using a buffer as a source of input, though this
is slightly more complicated, since the buffer must be
converted to a string first. I have distilled this down into
my own function, named with the "at" symbol.
And I will provide the brief snippet of code for this later.
So to give an example, here's our messages buffer.
And from Eshell, we can do something like this.
Let's say here we wanted to grab our messages buffer to see
everything that we had been loading during the startup
process.
So you can see how that could be very handy in a number of
scenarios.
I wanted to briefly mention that we have a helpful function
here called eshell-insert-buffer-name,
which allows us to insert a buffer name into the current
buffer at point using completion,
which can save you a lot of typing.
Another nice feature of Eshell
is that it allows integrating ELisp into the command line
call. Let's give another example. Say we wanted to echo the
date to an event file or an event log.
I should probably take a moment to explain this asterisk
that I'm occasionally using. So since Emacs, or excuse me,
since Eshell can use internal or external Emacs, excuse me,
internal Emacs commands or external commands, it may
sometimes be necessary to clarify which one you want to use,
since the names may overlap. Since my Eshell is configured
by default to prefer the internal Emacs functions, then
sometimes I have to use the asterisk to specify that I want
the external version.
Here I can insert a bit of Elisp,
and then redirect the output to the event log.
Last, I want to mention that there are some optional Eshell
modules in Emacs, not turned on by default, which provide
additional nifty features.
On my system, I have most of the optional modules turned on.
An interesting module is eshell-smart, which does various
things with cursor positioning and scrolling, so as to make
editing commands and reviewing output easier.
Let's say I was to change directory to my boot directory
and use a command which involves lots of output.
You'll notice right away that the cursor positioning is set
such that I'm immediately able to view the top of the output.
Also, I'm able to use the space bar to page through the
output.
So this is an opinionated feature, which assumes that
you're likely going to want to review the output
immediately, or that you often will. Of course, you can
always jump to the end.
Also, after a command is entered, the cursor is immediately
repositioned to make it easy to edit the command.
And also, if I don't want to edit the command, and I do not want
to review the output, I can simply start typing another
command.
So that covers the brief tour of Eshell features.
And that basically ends my talk.
However, a handful of
viewers might be wondering, is it possible to set Emacs to be
my login shell to completely replace bash in your login
experience? The answer is yes, but in practice there are
various difficulties involved which might make it not
worth the trouble.
Before doing this, you'll have to answer a few initial
questions. Do you want to make a new Emacs instance every
time you log in, or do you want it to connect to an Emacs
server? Which is popular among Emacs users, to reuse the
session, or to connect to the existing session. Also, do you
want a different result, whether in graphical or a terminal
environment? And are you okay with your initialization
file being run every time you log in, including every new tab
you open in a terminal emulator? If we assume that you are
using a system with /etc/passwd user management, you get
one field to specify the name of the shell program that you
want to use, and no arguments are allowed. So maybe you can
see how this might be challenging, depending on your
answers to the previous questions. You can work around
these issues in various ways, like modifying the
authentication system, or by specifying a script for your
login shell. But if your normal workflow is to simply log in
and start Emacs and run that Emacs session until your next
reboot, then it probably isn't worth the bother.
So thank you for listening to my talk, Emacs as a Shell, by
Christopher Howard for Emacs Conference 2024.
At the bottom of this page, you can see a link to the
repository containing the brief amount of code that was
featured here in this video, as well as a link to my personal
Gemini gemlog, as well as to a Web portal version of that.
Thank you.