WEBVTT captioned by mohsen
NOTE Introduction
00:00:00.000 --> 00:00:09.719
Greetings. Salaam. This is Mohsen Banan. محسن بنان.
00:00:09.720 --> 00:00:12.559
I am a software and internet engineer.
00:00:12.560 --> 00:00:14.519
I have been interested in email and
00:00:14.520 --> 00:00:17.199
Emacs for a very long time.
00:00:17.200 --> 00:00:21.159
My interest in email started with X.400
00:00:21.160 --> 00:00:27.599
and the Red and Blue CCITT books -- circa 1988.
00:00:27.600 --> 00:00:31.999
Early on, in the very early 1990s, I jumped ship
00:00:32.000 --> 00:00:35.279
and joined the Internet email movement.
00:00:35.280 --> 00:00:38.399
I am the primary author of two mobile email
00:00:38.400 --> 00:00:45.839
related Internet RFCs, RFC-2188 and RFC-2524.
00:00:45.840 --> 00:00:49.919
My interest in Emacs started in 1986 --
00:00:49.920 --> 00:00:54.959
It was Emacs version 17 then. By around 1988
00:00:54.960 --> 00:00:58.479
when Emacs version 18 was well in place,
00:00:58.480 --> 00:01:01.799
I started living inside of Emacs.
00:01:01.800 --> 00:01:06.479
My primary digital environment has been Emacs ever since.
00:01:06.480 --> 00:01:11.319
It has been a good life.
00:01:11.320 --> 00:01:16.999
It turns out that Emacs and email mix up really well.
00:01:17.000 --> 00:01:21.519
Here, in this presentation and in the context of
00:01:21.520 --> 00:01:26.799
Revisiting The Anatomy of Emacs Mail User Agents,
00:01:26.800 --> 00:01:30.319
With MARMEE (Multi-Account Resident
00:01:30.320 --> 00:01:33.399
Message Exchange Environment)
00:01:33.400 --> 00:01:35.559
I am offering my thoughts on this topic
00:01:35.560 --> 00:01:41.079
in this Emacs Conference 2022.
NOTE Mail and the digital ecosystem
00:01:41.080 --> 00:01:43.359
Long ago, I asked myself:
00:01:43.360 --> 00:01:49.119
"What should my ultimate mail environment be?"
00:01:49.120 --> 00:01:52.140
Over the past 20+ years, I have been exploring
00:01:52.141 --> 00:01:58.519
the concept of the "Ultimate Mail User Agent (MUA)".
00:01:58.520 --> 00:02:01.439
We do care about privacy, autonomy,
00:02:01.440 --> 00:02:05.039
morality, ethics, society and philosophy,
00:02:05.040 --> 00:02:10.679
so from the get go, proprietary (Haraam) environments
00:02:10.680 --> 00:02:14.199
such as Microsoft Office's Outlook
00:02:14.200 --> 00:02:20.159
and Google Office's Gmail were non-starters for me.
00:02:20.160 --> 00:02:23.159
But these are significant realities
00:02:23.160 --> 00:02:27.879
and we need to deal with these realities.
00:02:27.880 --> 00:02:30.199
Notice how Microsoft and Google
00:02:30.200 --> 00:02:36.079
have both framed their MUAs in the context of "office".
00:02:36.080 --> 00:02:38.719
That type of framing is correct.
00:02:38.720 --> 00:02:41.719
an MUA must be fully integrated
00:02:41.720 --> 00:02:46.559
in the totality of one's digital ecosystem.
00:02:46.560 --> 00:02:49.759
So, the Ultimate Mail User Agent
00:02:49.760 --> 00:02:53.399
must be part of the Ultimate Usage Environment
00:02:53.400 --> 00:02:57.599
of the Ultimate Digital Ecosystem.
00:02:57.600 --> 00:03:02.799
In the non-proprietary (Halaal) universe, clearly
00:03:02.800 --> 00:03:07.199
the ultimate usage environment is Emacs.
00:03:07.200 --> 00:03:10.879
Emacs is today's most potent and convivial
00:03:10.880 --> 00:03:15.159
non-proprietary usage environment.
00:03:15.160 --> 00:03:19.399
So, clearly, the ultimate Mail User Agent
00:03:19.400 --> 00:03:22.879
must be an integral part of Emacs.
NOTE Emacs and mail
00:03:22.880 --> 00:03:24.157
Having reached that conclusion,
00:03:24.158 --> 00:03:28.124
we then need to determine the specifics
00:03:28.125 --> 00:03:33.599
of the shape and the anatomy of Emacs' MUAs.
00:03:33.600 --> 00:03:36.039
We could have arrived at this conclusion
00:03:36.040 --> 00:03:38.239
from the reverse direction as well.
00:03:38.240 --> 00:03:41.599
Zawinski's Law states:
00:03:41.600 --> 00:03:46.479
Every program attempts to expand until it can read mail.
00:03:46.480 --> 00:03:49.119
Those programs which cannot so expand
00:03:49.120 --> 00:03:52.679
are replaced by ones which can.
00:03:52.680 --> 00:03:56.719
Jamie's point is very simple and obvious.
00:03:56.720 --> 00:03:59.759
The "App" that you "live in" all day
00:03:59.760 --> 00:04:05.439
should be your MUA and mail environment.
00:04:05.440 --> 00:04:09.559
I ask those who jumped ship, who abandoned Emacs
00:04:09.560 --> 00:04:15.879
in favor of VS Code: What about mail?
00:04:15.880 --> 00:04:21.279
Long ago, Emacs expanded to including MUAs.
00:04:21.280 --> 00:04:27.039
In fact there are many Emacs MUAs that you can choose from.
00:04:27.040 --> 00:04:30.599
If you are already hip with Emacs And Linux,
00:04:30.600 --> 00:04:34.639
you should definitely consider doing email in Emacs.
00:04:34.640 --> 00:04:39.079
But if you are not already hip with Emacs,
00:04:39.080 --> 00:04:43.359
I mean for new Emacs users, unfortunately,
00:04:43.360 --> 00:04:48.599
setting up and using email is not straight forward.
00:04:48.600 --> 00:04:54.519
We (I mean, Emacs developers) should work on that!
00:04:54.520 --> 00:04:57.359
Emacs offers a good number of MUAs with
00:04:57.360 --> 00:05:01.959
different characteristics to suit differing tastes.
00:05:01.960 --> 00:05:06.119
As of 2022, you can choose from the following MUAs:
00:05:06.120 --> 00:05:15.079
Gnus, VM, WanderLust, Mew, mu4e, notmuch.el, mh-e and Rmail.
00:05:15.080 --> 00:05:17.719
Over the years I have tried several of these
00:05:17.720 --> 00:05:20.119
and eventually landed on Gnus.
00:05:20.120 --> 00:05:25.519
The relevance column in this table simply and only
00:05:25.520 --> 00:05:28.079
reflects my taste.
00:05:28.080 --> 00:05:32.399
Throughout the rest of this presentation, I focus on Gnus.
NOTE The audience for this presentation
00:05:32.400 --> 00:05:36.199
I have 3 types of audiences in mind for this presentation.
00:05:36.200 --> 00:05:39.959
First, if you are already using Emacs
00:05:39.960 --> 00:05:42.439
as more than an editor,
00:05:42.440 --> 00:05:47.399
it makes good sense for you to also use Emacs as your MUA.
00:05:47.400 --> 00:05:50.759
There may well be some relevant information here for you
00:05:50.760 --> 00:05:52.319
in that situation.
00:05:52.320 --> 00:05:57.599
Second, for those interested in philosophy of Emacs,
00:05:57.600 --> 00:06:00.759
I go on some bigger picture tangents
00:06:00.760 --> 00:06:02.839
that may be of value to you.
00:06:02.840 --> 00:06:08.839
Third, I address some Emacs developers with some feedback,
00:06:08.840 --> 00:06:11.319
some suggestions, and some requests.
00:06:11.320 --> 00:06:14.599
The general model here is that
00:06:14.600 --> 00:06:17.079
we would collectively work towards
00:06:17.080 --> 00:06:19.799
improving what is on the table.
NOTE Monolithic mail user agents
00:06:19.800 --> 00:06:22.719
When a Mail User Agent is self-contained
00:06:22.720 --> 00:06:26.519
and includes implementation of mail protocols,
00:06:26.520 --> 00:06:29.079
we call it a Monolithic-MUA.
00:06:29.080 --> 00:06:33.879
Just as it is with the physical mail postal service,
00:06:33.880 --> 00:06:36.919
sending mail and receiving mail
00:06:36.920 --> 00:06:40.639
are fundamentally separate activities.
00:06:40.640 --> 00:06:42.839
And then there is mail processing.
00:06:42.840 --> 00:06:45.959
Based on these categorizations,
00:06:45.960 --> 00:06:49.679
Emacs has a set of mature libraries
00:06:49.680 --> 00:06:53.359
for composing mail, sending mail, and receiving mail.
00:06:53.360 --> 00:06:58.119
These are all independently well-documented
00:06:58.120 --> 00:07:02.479
and are part of the basic emacs Distribution.
00:07:02.480 --> 00:07:06.239
Emacs MUAs then use these common libraries
00:07:06.240 --> 00:07:09.519
to process mail (each somewhat differently).
00:07:09.520 --> 00:07:15.399
The primary benefit of the Monolithic-MUA approach
00:07:15.400 --> 00:07:19.599
is that Emacs MUAs then become self-contained
00:07:19.600 --> 00:07:22.959
and therefore multi-platform.
NOTE Split mail user agents
00:07:22.960 --> 00:07:25.559
But, when it comes to the question of merits of
00:07:25.560 --> 00:07:30.559
implementation of mail protocols in Elisp inside of Emacs,
00:07:30.560 --> 00:07:33.959
there is also another approach:
00:07:33.960 --> 00:07:36.159
that of a Split-MUA.
00:07:36.160 --> 00:07:40.559
Concept of a split-MUA is that of
00:07:40.560 --> 00:07:44.959
splitting the MUA into two different parts:
00:07:44.960 --> 00:07:47.159
One being the usage environment,
00:07:47.160 --> 00:07:50.319
and the other being mail protocols processing.
00:07:50.320 --> 00:07:54.719
The interface between these can be either
00:07:54.720 --> 00:07:57.839
direct (the upper box)
00:07:57.840 --> 00:08:00.279
or through protocols (the lower box).
00:08:00.280 --> 00:08:05.159
With Gnus, we primarily use the direct interface.
00:08:05.160 --> 00:08:09.639
The split-MUA model has many advantages
00:08:09.640 --> 00:08:12.599
over the monolithic-MUA model.
00:08:12.600 --> 00:08:17.279
With Split-MUAs, your messages are local,
00:08:17.280 --> 00:08:19.839
you can search them privately
00:08:19.840 --> 00:08:22.319
and access to your email is faster.
NOTE Gnus can be both
00:08:22.320 --> 00:08:28.119
Gnus can be used as both a Monolithic-MUA
00:08:28.120 --> 00:08:30.599
and also as a Split-MUA.
00:08:30.600 --> 00:08:35.399
Gnus and other Emacs MUAs are flexible enough
00:08:35.400 --> 00:08:39.119
to allow you to create your own split-MUA.
00:08:39.120 --> 00:08:42.519
For outgoing mail, Gnus can
00:08:42.520 --> 00:08:45.119
invoke a sendmail-like interface program.
00:08:45.120 --> 00:08:50.007
For incoming mail, Gnus can access Maildirs directly
00:08:50.008 --> 00:08:53.724
and let other programs imap-retrieve
00:08:53.725 --> 00:08:56.439
and update into maildirs.
00:08:56.440 --> 00:08:59.719
You can then search through your maildirs
00:08:59.720 --> 00:09:01.374
locally and privately
00:09:01.375 --> 00:09:04.559
with various mail-oriented search engines,
00:09:04.560 --> 00:09:07.319
and many have done so.
00:09:07.320 --> 00:09:11.399
For example, what we are seeing on this slide
00:09:11.400 --> 00:09:15.879
is from a 2014 Do It Yourself (DIY) recipe
00:09:15.880 --> 00:09:20.319
that one of our fellow Emacs conference participants,
00:09:20.320 --> 00:09:24.879
Adolfo, had published at the mentioned URL.
00:09:24.880 --> 00:09:29.719
The recipe in that slide is based on the following tools:
00:09:29.720 --> 00:09:35.079
mbsync, mu, mu4e, and msmtp.
00:09:35.080 --> 00:09:38.279
All our choices are different.
00:09:38.280 --> 00:09:42.399
There are many such recipes out there on the web.
NOTE Proprietary universes
00:09:42.400 --> 00:09:46.199
So, here, I don't want to provide
00:09:46.200 --> 00:09:50.439
yet another Emacs Split-MUA recipe.
00:09:50.440 --> 00:09:51.559
I want to do more.
00:09:51.560 --> 00:09:56.679
Instead, I want to target the contours of the ultimate MUA
00:09:56.680 --> 00:10:01.319
in the non-proprietary universe of digital ecosystems.
00:10:01.320 --> 00:10:05.439
But, first, let's take a look at what is
00:10:05.440 --> 00:10:07.479
happening in the proprietary universe.
00:10:07.480 --> 00:10:11.439
The 5 big American proprietary tech companies
00:10:11.440 --> 00:10:14.399
(Google, Microsoft, Apple, Facebook and Amazon)
00:10:14.400 --> 00:10:20.439
have created 5 competing enclaves as mostly separate
00:10:20.440 --> 00:10:22.999
and isolated digital ecosystem.
00:10:23.000 --> 00:10:27.839
In this slide, I am focusing on the first 3
00:10:27.840 --> 00:10:31.439
and each of their office and email environments.
00:10:31.440 --> 00:10:36.319
Let's clearly recognize that the economic model
00:10:36.320 --> 00:10:40.159
of these proprietary digital ecosystems is:
00:10:40.160 --> 00:10:42.159
"Surveillance Capitalism".
00:10:42.160 --> 00:10:45.799
So, when any of us goes there to get
00:10:45.800 --> 00:10:47.959
a free-of-charge email account,
00:10:47.960 --> 00:10:53.039
he has chosen to voluntarily forgo much of his privacy.
00:10:53.040 --> 00:10:55.799
And many have done so.
00:10:55.800 --> 00:10:59.919
Sadly, the rest of the world is becoming
00:10:59.920 --> 00:11:02.719
Americanized through the American Internet.
00:11:02.720 --> 00:11:08.439
As of 2022, almost %90 of Facebook's
00:11:08.440 --> 00:11:11.959
daily active users come from outside of the US.
00:11:11.960 --> 00:11:17.159
Also, with respect to email, each of the enclaves
00:11:17.160 --> 00:11:20.039
have MUAs that are fully integrated
00:11:20.040 --> 00:11:22.439
in their digital ecosystems
00:11:22.440 --> 00:11:24.599
in the form of an office environment
00:11:24.600 --> 00:11:29.839
comprising of address book, calendar, time management
00:11:29.840 --> 00:11:33.199
and planning tools and multi-lingual authoring
00:11:33.200 --> 00:11:36.239
and various other integrated tools.
NOTE Non-proprietary universes
00:11:36.240 --> 00:11:40.839
Now, let's focus on the right side of this picture.
00:11:40.840 --> 00:11:43.519
On the non-proprietary side,
00:11:43.520 --> 00:11:46.239
based on the Western FLOSS model,
00:11:46.240 --> 00:11:49.479
we have ended up with lots of components.
00:11:49.480 --> 00:11:52.239
We have Debian as a platform,
00:11:52.240 --> 00:11:58.919
we have Emacs as an editor-centered office environment
00:11:58.920 --> 00:12:03.439
and we have Gnus as an incredibly powerful MUA.
00:12:03.440 --> 00:12:07.839
But on the non-proprietary side we don't have anything
00:12:07.840 --> 00:12:12.079
that can reasonably be considered a digital ecosystem.
00:12:12.080 --> 00:12:16.439
I mean, the services aspect is missing.
NOTE By*
00:12:16.440 --> 00:12:20.799
Over the past two decades I have created
00:12:20.800 --> 00:12:24.399
quite an elaborate digital ecosystem for myself.
00:12:24.400 --> 00:12:26.759
It is called: By*.
00:12:26.760 --> 00:12:30.839
The Libre-Halaal ByStar Digital Ecosystem
00:12:30.840 --> 00:12:35.439
is being built to provide autonomy-oriented services
00:12:35.440 --> 00:12:37.359
on internet scale.
00:12:37.360 --> 00:12:42.919
The * in ByStar stands for Unix's globbing symbol,
00:12:42.920 --> 00:12:46.919
signifying that our scope is everything.
00:12:46.920 --> 00:12:52.039
Notice in this bigger picture that in the red box,
00:12:52.040 --> 00:12:58.439
our focus remains to be Emacs, Gnus and the ultimate MUA.
00:12:58.440 --> 00:13:03.439
I am not here to sell you ByStar, but perhaps
00:13:03.440 --> 00:13:06.239
you should be in the market for something like that.
00:13:06.240 --> 00:13:10.879
We need non-proprietary digital ecosystems.
00:13:10.880 --> 00:13:13.359
Very briefly, I'll give you
00:13:13.360 --> 00:13:16.319
some pointers to the full ByStar story.
00:13:16.320 --> 00:13:23.239
The full ByStar story is a 250 plus pages book titled:
00:13:23.240 --> 00:13:26.079
Nature Of Polyexistentials,
00:13:26.080 --> 00:13:28.399
Basis For Abolishment Of
00:13:28.400 --> 00:13:31.319
The Western Intellectual Property Rights Regime,
00:13:31.320 --> 00:13:33.479
And Introduction Of
00:13:33.480 --> 00:13:36.359
The Libre-Halaal ByStar Digital Ecosystem.
00:13:36.360 --> 00:13:42.039
I have it self-published on my own ByName public web page.
00:13:42.040 --> 00:13:46.639
The ByStar story starts with understanding of the
00:13:46.640 --> 00:13:48.959
Nature Of Polyexistentials.
00:13:48.960 --> 00:13:53.839
Polyexistentials inherently exist in multiples.
00:13:53.840 --> 00:13:56.959
Software is a polyexistential.
00:13:56.960 --> 00:14:01.919
Polyexistentials are naturally non-scarce,
00:14:01.920 --> 00:14:06.279
and making polyexistential artificially scarce,
00:14:06.280 --> 00:14:08.599
which is what the Western
00:14:08.600 --> 00:14:11.119
intellectual property rights regime does,
00:14:11.120 --> 00:14:13.599
is counter to nature.
00:14:13.600 --> 00:14:17.639
Polyexistentials are unownable
00:14:17.640 --> 00:14:20.039
and should not be considered property.
00:14:20.040 --> 00:14:25.079
The Western IPR regime is in conflict with nature.
00:14:25.080 --> 00:14:29.039
But, the book is more than just philosophy.
00:14:29.040 --> 00:14:32.359
In that book I also cover
00:14:32.360 --> 00:14:35.799
the bigger picture of healthy digital ecosystems
00:14:35.800 --> 00:14:39.359
which also includes the topic of this presentation.
00:14:39.360 --> 00:14:43.399
I'd be interested in your thoughts and your feedback,
00:14:43.400 --> 00:14:45.679
if you choose to dig deeper.
00:14:45.680 --> 00:14:50.999
And if you want to dig deeper, here are some links.
00:14:51.000 --> 00:14:55.079
By* is about re-decentralization
00:14:55.080 --> 00:14:57.159
of Internet application services.
00:14:57.160 --> 00:15:00.759
Among other things, ByStar provides
00:15:00.760 --> 00:15:05.159
complete own-your-email services. I mean,
00:15:05.160 --> 00:15:10.319
private Hillary-Clinton-Style mail servers for everyone.
00:15:10.320 --> 00:15:16.519
There is an overview of ByStar at by-star.net.
NOTE Libre-Halaal
00:15:16.520 --> 00:15:21.199
You may have noticed that I consistently use
00:15:21.200 --> 00:15:25.479
the "Libre-Halaal" label with ByStar.
00:15:25.480 --> 00:15:28.919
Halaal is a very sensitive word.
00:15:28.920 --> 00:15:30.719
I am a Moslem.
00:15:30.720 --> 00:15:35.919
But my use of Halaal is not in a religious context.
00:15:35.920 --> 00:15:39.079
It is in a philosphical context.
00:15:39.080 --> 00:15:42.759
And the scope of the "Libre-Halaal" label
00:15:42.760 --> 00:15:46.439
is manner-of-existence of Software and Services.
00:15:46.440 --> 00:15:50.599
It is not about Halaal-ness with respect to
00:15:50.600 --> 00:15:54.959
function and use of Software and Services.
00:15:54.960 --> 00:15:58.239
Unfortunately, the word Halaal
00:15:58.240 --> 00:16:02.319
and the concept of Halaal does not exist in English.
00:16:02.320 --> 00:16:06.759
So, first I introduce it into Globish.
00:16:06.760 --> 00:16:12.799
I have done so in PLPC-120039.
00:16:12.800 --> 00:16:18.199
Further, I explain as to why labels
00:16:18.200 --> 00:16:22.639
of Open Source and Free Software are both ill-directed.
00:16:22.640 --> 00:16:25.839
We then carefully define
00:16:25.840 --> 00:16:29.999
"Libre-Halaal Software" and "Libre-Halaal Services".
00:16:30.000 --> 00:16:33.919
Notice that last link.
00:16:33.920 --> 00:16:36.199
I bet, this is the first time
00:16:36.200 --> 00:16:40.679
that anyone includes a link to his "Open Business Plan"
00:16:40.680 --> 00:16:43.159
in an Emacs Conference.
00:16:43.160 --> 00:16:46.559
I hope others would do this as well.
00:16:46.560 --> 00:16:49.679
There is appetite out there
00:16:49.680 --> 00:16:53.639
for privacy- and autonomy-oriented digital ecosystems,
00:16:53.640 --> 00:16:57.759
and there is no conflict between honest business,
00:16:57.760 --> 00:17:01.039
honest profit, and Libre-Halaal Software
00:17:01.040 --> 00:17:02.919
and Libre-Halaal Services.
00:17:02.920 --> 00:17:08.039
The sub-title of our open business plan is:
00:17:08.040 --> 00:17:12.879
"An Inversion to the Proprietary Internet Services Model".
00:17:12.880 --> 00:17:20.879
And here are the same links as a native Reveal slide.
00:17:20.880 --> 00:17:24.679
If instead of a video, you are viewing
00:17:24.680 --> 00:17:27.399
this presentation as a Reveal web page,
00:17:27.400 --> 00:17:31.319
you can just click on the pointers and URLs.
NOTE The BISOS integration framework
00:17:31.320 --> 00:17:36.879
So, what was the point of bringing ByStar
00:17:36.880 --> 00:17:38.119
into this presentation?
00:17:38.120 --> 00:17:42.319
In tangible terms, what have we gotten out of
00:17:42.320 --> 00:17:45.639
the tangent we took on the ByStar bigger picture?
00:17:45.640 --> 00:17:50.839
Of course we have the ByStar Digital Ecosystem itself.
00:17:50.840 --> 00:17:54.879
But that is not immediately relevant to this presentation.
00:17:54.880 --> 00:17:59.959
Here, through BISOS we now have
00:17:59.960 --> 00:18:04.359
an integration framework, which we definitely needed.
00:18:04.360 --> 00:18:07.879
We now have BISOS-MARMEE,
00:18:07.880 --> 00:18:11.519
Multi-Account Resident Mail Exchange Environment,
00:18:11.520 --> 00:18:13.679
which is a consistent set
00:18:13.680 --> 00:18:17.639
of MUA-related software components --- which we need.
00:18:17.640 --> 00:18:23.159
We also needed to augment Emacs in our own terms,
00:18:23.160 --> 00:18:25.919
so we have Blee for that,
00:18:25.920 --> 00:18:29.959
ByStar Libre-Halaal Emacs Environment,
00:18:29.960 --> 00:18:32.919
is ByStar ecosystemized Emacs.
00:18:32.920 --> 00:18:37.679
And finally Blee-Gnus, which is
00:18:37.680 --> 00:18:40.639
Gnus and MARMEE integrated with Blee.
00:18:40.640 --> 00:18:46.479
With these in place, we can now dive deeper into MARMEE.
00:18:46.480 --> 00:18:53.119
The idea of MARMEE, is that of packaging together
00:18:53.120 --> 00:18:56.079
the mail protocols parts of the Split-MUA.
00:18:56.080 --> 00:19:00.799
MARMEE (which is of course in the context of BISOS)
00:19:00.800 --> 00:19:03.679
is the green box in this slide.
00:19:03.680 --> 00:19:08.519
For outgoing mail, we use an altered qmail.
00:19:08.520 --> 00:19:12.679
We will be looking deeper into qmail a bit later.
00:19:12.680 --> 00:19:17.559
For incoming mail, we are using offlineimap
00:19:17.560 --> 00:19:20.119
which is oauth2 aware.
NOTE BISOS-MARMEE
00:19:20.120 --> 00:19:23.439
Before going into more details,
00:19:23.440 --> 00:19:26.519
let's take a look at the parts lists for
00:19:26.520 --> 00:19:29.359
BISOS-MARMEE and Blee-Gnus.
00:19:29.360 --> 00:19:33.719
MARMEE is a collection of Python-based libraries
00:19:33.720 --> 00:19:37.479
and Debian packages that provide for rich sending
00:19:37.480 --> 00:19:40.279
and receiving of email outside of Emacs.
00:19:40.280 --> 00:19:44.239
Here is our BISOS-MARMEE parts list.
00:19:44.240 --> 00:19:48.599
MARMEE features include tracked mail Sending
00:19:48.600 --> 00:19:51.919
for confirmed mail communications
00:19:51.920 --> 00:19:54.599
and email distribution facilities
00:19:54.600 --> 00:19:57.559
(say, similar to Constant Contact).
00:19:57.560 --> 00:20:02.079
For Delivery Status Notification (DSN),
00:20:02.080 --> 00:20:06.039
we have adopted flufl.bounce.
00:20:06.040 --> 00:20:10.879
I'll be touching on everything that is qmail-related,
00:20:10.880 --> 00:20:17.319
namely qmail-remote.cs and mailfront, in a separate slide.
00:20:17.320 --> 00:20:21.759
notmuch is our choice of mail search engine.
NOTE Blee-Gnus
00:20:21.760 --> 00:20:27.319
Similarly, here is our Blee-Gnus Parts List.
00:20:27.320 --> 00:20:33.439
Blee-Gnus is Gnus and MARMEE integrated with BISOS and Blee.
00:20:33.440 --> 00:20:38.599
Notice mentions of org-msg and polymode here.
00:20:38.600 --> 00:20:42.879
Later, I'll expand on these in the context of
00:20:42.880 --> 00:20:47.679
transitioning from Message-Mode to Message-Polymode.
NOTE In combination
00:20:47.680 --> 00:20:52.199
With these parts in place,
00:20:52.200 --> 00:20:55.279
now let's see how they will all come together.
00:20:55.280 --> 00:20:59.999
Gnus is very flexible, and in combination with MARMEE,
00:21:00.000 --> 00:21:03.639
it can create an incredibly powerful MUA.
00:21:03.640 --> 00:21:07.279
On this slide, note the boxes
00:21:07.280 --> 00:21:10.319
that include the FPs label.
00:21:10.320 --> 00:21:14.839
FP stand for File Parameters.
00:21:14.840 --> 00:21:18.519
It is the basis of BISOS's configuration
00:21:18.520 --> 00:21:20.039
and secrets management.
00:21:20.040 --> 00:21:24.319
Notice that it has consistent agents
00:21:24.320 --> 00:21:27.839
inside of Emacs and on the OS.
00:21:27.840 --> 00:21:29.919
This is a big deal
00:21:29.920 --> 00:21:34.719
in that it can reduce user visible configuration complexity.
NOTE X822-Bus
00:21:34.720 --> 00:21:39.759
Also, notice the X822-Bus here.
00:21:39.760 --> 00:21:43.999
The idea of X822-Bus is that of
00:21:44.000 --> 00:21:49.839
allowing for communication among user's preferences, Gnus
00:21:49.840 --> 00:21:53.599
and MARMEE-qmail through addition of X- fields
00:21:53.600 --> 00:21:57.959
in RFC-822 message headers.
00:21:57.960 --> 00:22:03.599
X822-Bus is used for selection of mail sending agents
00:22:03.600 --> 00:22:08.839
and specification of delivery status parameters.
NOTE bystar-qmail
00:22:08.840 --> 00:22:12.279
Of key significance in this picture
00:22:12.280 --> 00:22:15.479
is our choice of qmail for outgoing mail.
00:22:15.480 --> 00:22:22.519
Compared to sendmail, postfix, exim,
00:22:22.520 --> 00:22:25.399
and other conventional MTAs;
00:22:25.400 --> 00:22:31.719
the qmail ecosystem is far more flexible and potent.
00:22:31.720 --> 00:22:34.599
We are not using qmail as is.
00:22:34.600 --> 00:22:37.999
Ours is called bystar-qmail.
00:22:38.000 --> 00:22:41.799
When we use it as a traditional MTA,
00:22:41.800 --> 00:22:45.439
we refer to it as PALS-qmail.
00:22:45.440 --> 00:22:52.599
And when we use it on the MUA side, we call it MARMEE-qmail.
00:22:52.600 --> 00:22:56.599
Just like Emacs, qmail has
00:22:56.600 --> 00:22:59.959
a solid core and a flexible periphery.
00:22:59.960 --> 00:23:04.479
All our alterations have been on the periphery.
00:23:04.480 --> 00:23:07.890
We have replaced qmail-remote
00:23:07.891 --> 00:23:14.479
with our own Python implementation called qmail-remote.cs.
00:23:14.480 --> 00:23:20.919
By being in Python, it can do a lot more a lot more easily.
00:23:20.920 --> 00:23:26.540
For example, qmail-remote.cs interacts with
00:23:26.541 --> 00:23:33.079
Google Oauth2 APIs and allows you to send through Gmail.
00:23:33.080 --> 00:23:36.399
This is shown with the red circle.
00:23:36.400 --> 00:23:43.639
We have also replaced qmail-smtpd with mailfront,
00:23:43.640 --> 00:23:46.159
shown with a blue circle.
00:23:46.160 --> 00:23:51.359
This allows us to use MARMEE Split-MUA
00:23:51.360 --> 00:23:53.999
through protocol interfaces.
00:23:54.000 --> 00:23:56.559
Let's take a look at that.
NOTE Using MARMEE with other MUAs outside Emacs
00:23:56.560 --> 00:24:02.719
Previously we looked at the "Direct Interface" of MARMEE,
00:24:02.720 --> 00:24:08.479
specifically, qmail-inject and Maildir for Gnus.
00:24:08.480 --> 00:24:11.479
But what if we wanted to use
00:24:11.480 --> 00:24:15.159
MARMEE with other MUAs outside of Emacs?
00:24:15.160 --> 00:24:18.799
That can be done through the "Protocol Interface".
00:24:18.800 --> 00:24:22.039
MARMEE also includes "mailfront"
00:24:22.040 --> 00:24:27.439
which can function as an SMTP submit server for localhost.
00:24:27.440 --> 00:24:30.599
This way, we can configure
00:24:30.600 --> 00:24:36.239
the outgoing mail part of any MUA to point to the localhost
00:24:36.240 --> 00:24:41.399
and have MARMEE-qmail function as an outgoing proxy.
NOTE Incoming mail
00:24:41.400 --> 00:24:47.919
For incoming mail, MARMEE-Split-MUA-Protocol-Interface
00:24:47.920 --> 00:24:51.359
includes "Courier", which can function
00:24:51.360 --> 00:24:54.079
as an IMAP server for localhost.
00:24:54.080 --> 00:24:58.519
This way, we can configure the incoming mail part
00:24:58.520 --> 00:25:02.319
of any MUA to point to the localhost
00:25:02.320 --> 00:25:06.519
and have MARMEE function as an incoming proxy
00:25:06.520 --> 00:25:10.759
by serving the local Maildir to the MUA.
NOTE Licensing and project status
00:25:10.760 --> 00:25:18.079
All sources for all of ByStar, BISOS,
00:25:18.080 --> 00:25:23.439
Blee and MARMEE are subject to Affero GPL.
00:25:23.440 --> 00:25:28.319
The sources and documentation are all republished
00:25:28.320 --> 00:25:30.559
under various "Organizations"
00:25:30.560 --> 00:25:35.439
under github.com/mohsenBanan
00:25:35.440 --> 00:25:40.679
All of ByStar, BISOS, Blee and MARMEE
00:25:40.680 --> 00:25:42.839
reflect work in progress,
00:25:42.840 --> 00:25:46.319
and we are NOT recruiting users at this time.
00:25:46.320 --> 00:25:49.279
For more than two decades,
00:25:49.280 --> 00:25:53.159
I have been using these all in that bigger context.
00:25:53.160 --> 00:25:56.239
They are mostly real,
00:25:56.240 --> 00:26:01.399
but so far, just for myself and a few other engineers.
00:26:01.400 --> 00:26:06.519
Our model is similar to God's early days.
00:26:06.520 --> 00:26:08.319
You may ask:
00:26:08.320 --> 00:26:12.119
"How did God create all of this in just 7 days?"
00:26:12.120 --> 00:26:17.199
Well, easy, He did not have an installed base to deal with.
NOTE Installing MARMEE
00:26:17.200 --> 00:26:24.519
You can obtain and install MARMEE in two ways. As is:
00:26:24.520 --> 00:26:29.439
as standalone-MARMEE, you can just
00:26:29.440 --> 00:26:31.919
pip install bisos.marmee.
00:26:31.920 --> 00:26:35.879
For the Gnus part you are completely on your own.
00:26:35.880 --> 00:26:40.279
Or on a Debian-11, you can just run
00:26:40.280 --> 00:26:43.399
the bisos bootstrap script.
00:26:43.400 --> 00:26:48.359
That way you will have all of BISOS, which includes MARMEE
00:26:48.360 --> 00:26:52.159
and you will have Blee, which includes Blee-Gnus.
00:26:52.160 --> 00:26:54.719
If you plan to do so,
00:26:54.720 --> 00:26:58.839
I suggest that you first try it in a disposable VM.
00:26:58.840 --> 00:27:02.159
BISOS and Blee are large.
00:27:02.160 --> 00:27:06.359
Many apt and pip packages will be installed!
00:27:06.360 --> 00:27:11.839
And here are the same links as a native Reveal slide.
00:27:11.840 --> 00:27:17.519
If you are viewing this presentation as Reveal.js web page,
00:27:17.520 --> 00:27:20.479
you can just click on the pointers and URLs.
NOTE MARMEE as an Emacs "Common Agent"
00:27:20.480 --> 00:27:25.359
Let's consider MARMEE as an Emacs "Common Agent".
00:27:25.360 --> 00:27:28.919
By "Common-Agent" I mean a capability
00:27:28.920 --> 00:27:30.959
which Emacs builds on
00:27:30.960 --> 00:27:33.759
and which other Apps can also use.
00:27:33.760 --> 00:27:38.479
Emacs has a very rich applications development framework
00:27:38.480 --> 00:27:41.119
for absorbing common-agents.
00:27:41.120 --> 00:27:45.474
Consider how magit has absorbed git,
00:27:45.475 --> 00:27:49.774
or how flycheck has absorbed mypy
00:27:49.775 --> 00:27:54.199
or how EAF does its work outside of Emacs ---
00:27:54.200 --> 00:27:57.919
that too can be considered a common-agent.
00:27:57.920 --> 00:28:02.039
The common-agent model permits us
00:28:02.040 --> 00:28:04.599
to do more outside of Emacs.
00:28:04.600 --> 00:28:08.639
Common-agents maximize social benefits
00:28:08.640 --> 00:28:10.559
and are more convivial.
00:28:10.560 --> 00:28:15.479
For example, any MUA can profit from MARMEE.
00:28:15.480 --> 00:28:18.439
But we don't have good ways of
00:28:18.440 --> 00:28:21.399
packaging Emacs and its packages
00:28:21.400 --> 00:28:23.639
with their common-agents.
00:28:23.640 --> 00:28:28.359
Instead, we usually end up with DIY recipes.
00:28:28.360 --> 00:28:32.479
This is why I am contextualizing
00:28:32.480 --> 00:28:35.439
Emacs inside of Blee and BISOS.
00:28:35.440 --> 00:28:37.999
That is what they are for.
00:28:38.000 --> 00:28:40.424
And that is why I consider them
00:28:40.425 --> 00:28:43.199
immediately relevant to this presentation.
00:28:43.200 --> 00:28:47.719
With an incredibly powerful Display Engine,
00:28:47.720 --> 00:28:51.279
and an incredibly powerful Elisp Engine,
00:28:51.280 --> 00:28:55.039
and an incredibly powerful Input Methods Engine,
00:28:55.040 --> 00:28:59.559
and an incredibly powerful Common-Agents paradigm,
00:28:59.560 --> 00:29:02.707
Emacs has the potential of being
00:29:02.708 --> 00:29:06.407
any non-proprietary digital ecosystem's
00:29:06.408 --> 00:29:08.879
preferred usage environment.
00:29:08.880 --> 00:29:14.090
I am in favor of putting more around Emacs
00:29:14.091 --> 00:29:17.440
and strengthening integration of Emacs
00:29:17.441 --> 00:29:19.719
with Debian, explicitly,
00:29:19.720 --> 00:29:22.839
perhaps even at the cost of
00:29:22.840 --> 00:29:26.599
de-emphasizing its multi-platform attribute.
00:29:26.600 --> 00:29:30.319
A smaller Emacs is a better Emacs.
00:29:30.320 --> 00:29:33.719
Notice that in this slide,
00:29:33.720 --> 00:29:37.279
I have used many arrows in many colors.
00:29:37.280 --> 00:29:45.599
Much of Emacs's power comes from its ability
00:29:45.600 --> 00:29:47.759
to absorb and to integrate.
00:29:47.760 --> 00:29:51.919
Tomohiro is right on the mark when he says,
00:29:51.920 --> 00:29:55.359
"The reason why Emacs platform is good
00:29:55.360 --> 00:29:58.119
is that it cooperates with OS,
00:29:58.120 --> 00:30:00.919
not because it is good by itself."
00:30:00.920 --> 00:30:03.999
I am suggesting that we should
00:30:04.000 --> 00:30:06.519
raise the bar from the OS
00:30:06.520 --> 00:30:09.839
to the entirety of our digital ecosystem.
00:30:09.840 --> 00:30:13.039
There are many models
00:30:13.040 --> 00:30:15.839
for Emacs to cooperate with the OS
00:30:15.840 --> 00:30:19.319
and with applications and with services.
00:30:19.320 --> 00:30:25.439
The colors of arrows in the previous slide correspond to
00:30:25.440 --> 00:30:28.879
the model of interface of the common-agent:
00:30:28.880 --> 00:30:33.999
for example, sub-process invocation, pipe-based
00:30:34.000 --> 00:30:39.359
asynchronous interface, or file-based interactions.
NOTE Consistent configuration
00:30:39.360 --> 00:30:44.759
One important aspect of common-agent paradigm is that
00:30:44.760 --> 00:30:50.399
both the common-agent and its Emacs App
00:30:50.400 --> 00:30:53.039
need to be configured consistently.
00:30:53.040 --> 00:30:57.959
In MARMEE and Blee-Gnus,
00:30:57.960 --> 00:31:01.919
we use File-Params to accomplish this.
00:31:01.920 --> 00:31:06.959
In BISOS, there is a Python interface to File-Params,
00:31:06.960 --> 00:31:10.239
there is a Bash interface to File-Params,
00:31:10.240 --> 00:31:15.719
and in Blee, there is an Elisp interface to File-Params.
00:31:15.720 --> 00:31:18.919
So, configurations are extended.
00:31:18.920 --> 00:31:23.279
Furthermore, File-Params can be encrypted,
00:31:23.280 --> 00:31:26.959
and credentials can be protected and shared.
00:31:26.960 --> 00:31:33.919
This is a significant improvement over .authinfo
00:31:33.920 --> 00:31:36.959
and its more recent incarnations.
NOTE Feedback and requests
00:31:36.960 --> 00:31:41.199
EmacsConf could be a great place
00:31:41.200 --> 00:31:44.374
for users to provide feedback to developers
00:31:44.375 --> 00:31:47.879
and for developers to suggest to developers.
00:31:47.880 --> 00:31:52.759
In that spirit, my primary audience in this part
00:31:52.760 --> 00:31:54.839
are fellow Emacs developers.
00:31:54.840 --> 00:32:00.039
BISOS-MARMEE and Blee-Gnus are starting points.
00:32:00.040 --> 00:32:02.439
We can collectively work
00:32:02.440 --> 00:32:04.679
towards improving what is in place.
00:32:04.680 --> 00:32:07.919
Some such improvements involve
00:32:07.920 --> 00:32:11.919
collaboration among various Emacs developers.
00:32:11.920 --> 00:32:16.599
Here, I am making some explicit requests
00:32:16.600 --> 00:32:19.559
from some of the relevant emacs developers.
00:32:19.560 --> 00:32:24.159
At most, these are requests and invitations.
00:32:24.160 --> 00:32:28.519
For each of these requests, I am providing links
00:32:28.520 --> 00:32:30.119
for additional details.
00:32:30.120 --> 00:32:33.039
In due course, I'll follow up
00:32:33.040 --> 00:32:35.279
in the Emacs developers mailing list.
NOTE X-Message-SMTP-Method: qmail
00:32:35.280 --> 00:32:41.879
Gnus uses X-Message-SMTP-Method
00:32:41.880 --> 00:32:45.119
for selection of Mail-Sending-Agent.
00:32:45.120 --> 00:32:50.519
Even though all the qmail injection code is still in Gnus,
00:32:50.520 --> 00:32:58.279
support for "X-Message-SMTP-Method: qmail" is missing.
00:32:58.280 --> 00:33:02.759
It takes 2 lines of code to revive it.
00:33:02.760 --> 00:33:07.399
With regards to (1), qmail was previously supported in Gnus.
00:33:07.400 --> 00:33:11.319
Lars, can you please reactivate it? Thanks.
NOTE X-Message-Send-Method
00:33:11.320 --> 00:33:16.439
(2) is a terminology suggestion.
00:33:16.440 --> 00:33:21.559
The term X-Message-SMTP-Method violates conceptual layering.
00:33:21.560 --> 00:33:27.079
Please consider changing it to X-Message-Send-Method.
00:33:27.080 --> 00:33:33.719
In a Split-MUA setup, Gnus need not know about SMTP at all.
00:33:33.720 --> 00:33:36.599
We just need to pass information
00:33:36.600 --> 00:33:39.319
to a Mail-Sending-Agent selector.
NOTE Sharing config info and secrets with common agents
00:33:39.320 --> 00:33:44.439
(3) is simply a design suggestion for
00:33:44.440 --> 00:33:46.759
which I prepared the context.
00:33:46.760 --> 00:33:51.839
.authinfo and Emacs auth-source library
00:33:51.840 --> 00:33:54.199
are too Emacs-centric.
00:33:54.200 --> 00:33:57.839
We need to share config info and secrets
00:33:57.840 --> 00:34:00.799
between common-agents and Emacs.
00:34:00.800 --> 00:34:03.639
The File Parameters approach
00:34:03.640 --> 00:34:05.799
can be a general-purpose solution.
00:34:05.800 --> 00:34:10.839
Is it reasonable to extend auth-source library to
00:34:10.840 --> 00:34:12.719
support File Params?
00:34:12.720 --> 00:34:16.519
I'll cover (4) in the next slide.
00:34:16.520 --> 00:34:21.159
(5) is a philosophical common suggestion
00:34:21.160 --> 00:34:26.439
to all Emacs developers. We need to better cultivate
00:34:26.440 --> 00:34:30.879
the model of Common-Agents integration with Emacs.
00:34:30.880 --> 00:34:39.599
And here are the same links as a native Reveal slide.
NOTE message-polymode
00:34:39.600 --> 00:34:42.799
A mail message comprises of
00:34:42.800 --> 00:34:45.599
Envelope, Header and BodyParts.
00:34:45.600 --> 00:34:49.439
Each of these have their own syntax (their own mode).
00:34:49.440 --> 00:34:53.639
Conceivably Each BodyPart has its own mode.
00:34:53.640 --> 00:34:59.439
So, we need to evolve Message-Mode into Message-Polymode.
00:34:59.440 --> 00:35:03.719
More or less by default, org-mode has become
00:35:03.720 --> 00:35:08.999
the beginnings of "Emacs Native Markup Language -- ENML".
00:35:09.000 --> 00:35:14.399
With org-msg you can write your emails in org-mode ---
00:35:14.400 --> 00:35:16.559
destined as html.
00:35:16.560 --> 00:35:19.559
org-msg needs to become
00:35:19.560 --> 00:35:22.239
an integral part of Message-Polymode.
00:35:22.240 --> 00:35:25.119
It would be heavenly
00:35:25.120 --> 00:35:29.959
if Lars, Jérémy and Vitalie could collaborate
00:35:29.960 --> 00:35:34.079
and give us the needed Message-Polymode. Thank you.
NOTE Vertical slice use cases
00:35:34.080 --> 00:35:38.119
One way to verify that we have not gone astray
00:35:38.120 --> 00:35:42.759
in our horizontal bigger pictures is to verify them
00:35:42.760 --> 00:35:46.919
through the concept of "Vertical Slice Use Cases".
00:35:46.920 --> 00:35:50.799
Let one use case be reading and writing
00:35:50.800 --> 00:35:54.639
of mail on multiple gmail accounts with Gnus.
00:35:54.640 --> 00:35:59.319
Google now requires use of oauth2 tokens
00:35:59.320 --> 00:36:02.439
which MARMEE can do outside of emacs.
00:36:02.440 --> 00:36:05.679
There is a recent email thread
00:36:05.680 --> 00:36:09.119
on that in the emacs-devel mailing list.
00:36:09.120 --> 00:36:14.279
Let another use case be that of tracking delivery
00:36:14.280 --> 00:36:18.679
and non-delivery reports for custom envelope addresses
00:36:18.680 --> 00:36:26.039
of byname.net (part of ByStar) autonomous mail services.
00:36:26.040 --> 00:36:30.319
I would have loved to walk you through these
00:36:30.320 --> 00:36:32.959
vertical slice use cases
00:36:32.960 --> 00:36:36.439
as screen captures of my Blee environment.
00:36:36.440 --> 00:36:40.639
For that, I need at least another 20 minutes.
00:36:40.640 --> 00:36:43.079
But my time is up.
00:36:43.080 --> 00:36:46.719
So, let's consider this as the first
00:36:46.720 --> 00:36:48.919
in a series of presentations
00:36:48.920 --> 00:36:51.799
where next in this series could be
00:36:51.800 --> 00:36:55.479
the mentioned two vertical slice use cases.
00:36:55.480 --> 00:36:59.279
Perhaps there could be another presentation
00:36:59.280 --> 00:37:02.879
on this topic in EmacsConf 2023.
00:37:02.880 --> 00:37:06.759
This document was produced entirely with
00:37:06.760 --> 00:37:10.799
Libre-Halaal Software, and is published using
00:37:10.800 --> 00:37:13.079
Libre-Halaal Internet Services.
00:37:13.080 --> 00:37:17.959
I want to thank all the EmacsConf Organizers
00:37:17.960 --> 00:37:19.519
for their great work,
00:37:19.520 --> 00:37:30.960
and Sacha, Leo, and Amin in particular.