Hey everyone, I'm Michael,
and I'm going to be talking about
Org mode and the IndieWeb.
I am located in the San Francisco Bay Area,
where I'm a developer as well as
a longtime Emacs user.
So, I maintain a personal website using Org mode.
If you're watching this talk,
I'm going to guess that you probably are too.
For anybody who isn't,
let me explain exactly what I mean by that.
I have a static website. I author the pages
in Org mode's markup language,
otherwise known as Orgdown.
I use the Org export facility
to transcode that markup to HTML.
And then I just use rsync to push
the resulting HTML pages up to a VPS.
I like the workflow.
It's familiar to me as a coder.
I get to use familiar tools like Git and Make.
Publishing and then pushing the site
to a local test server is just `make`.
Pushing it to the live site is just `make prod`.
That said, certain problems made themselves apparent
with this arrangement pretty quickly.
Commenting was one. It's very difficult
to support commenting on a static website.
I've got no database.
I have no real server, and so on.
Yes, of course, there is Disqus
and other third party services
that will do this for you,
but I was uncomfortable outsourcing that job.
And it was more than just comments.
There was a general sense of isolation.
There's no connection to places
like Reddit, or Mastodon, or Twitter,
you know, where all the people are.
I guess you can post, then Tweet a link to it.
But suppose somebody responds to your Tweet.
Now you've got a conversation going on on Twitter
that you're a visitor, and
that a non-Twitter-using visitor to your site
would be completely disconnected from.
I am seeing people using Reddit effectively
as the comment section for their sites.
But unless you've got an audience, you know,
the size of Derek's or Amos's,
I don't think that's really feasible either.
Now, after casting about for some time,
I stumbled upon something called the IndieWeb.
In their own words, the IndieWeb is
a community of independent and personal websites
connected by simple standards based on
the principles of owning your domain
and using it as your primary identity,
publishing on your own site,
and optionally syndicating elsewhere,
and owning your data.
I would describe it as a collection of individuals
who've chosen to own their own platforms,
alongside a loosely specked set of standards
that tie those sites together.
And it's really those standards
that make the IndieWeb
more than just a call for everybody
to go back to the arts
and blog on on their own sites.
Now, this presentation isn't going to
focus on the IndieWeb as such.
It's more about using Org mode
to put your site on the IndieWeb.
I'm a little limited by time here,
so I'm going to focus on
just one of those protocols,
and that's Webmentions.
So, what's a Webmention?
Let's begin with the inveterate Alice,
who has a website
and has posted content to that site.
Her old friend Bob comes along,
notices that content
and wishes to say something about it.
He posts to his site
and his publication software will,
if it supports Webmentions,
will notice that he's mentioned Alice's post.
At that point, his publication software
reaches out to Alice's site,
asks for the mentioned document,
and will examine it to see
if Alice advertises an endpoint at her site
capable of receiving Webmentions.
In this case, it does.
So, Bob's publishing software does it.
At the end of the day,
a Webmention is really just
an HTTP post request with two parameters,
a source and a target.
On receipt, Alice's server will
reach out to Bob's site,
request the document that contains the mention,
and validate it, decide whether or not
she wants to accept the Webmention.
In this case, it's legit, it's accepted,
and Alice chooses to make a note,
to update her content,
to make a note of the fact
that it was mentioned by Bob.
Now a couple of things to note here.
The first is that this is effectively
decentralized commenting.
Both parties own their content,
and there's no third party involved,
trusted or otherwise.
Now at this point, you might object that, well,
as the owner of a statically generated site,
I have essentially none of the infrastructure
I need to implement this.
I have no server above and beyond Apache.
I can't really…, I have no database.
I guess you could send Webmentions with curl,
but now to do Webmention endpoint discovery,
you're going to be parsing arbitrary HTML.
It's a lot of work.
That's okay. There are sites out there
that offer Webmentions as a service.
So, let's take a look at how that goes.
We return to our original example.
Alice continues to advertise an endpoint
capable of receiving Webmentions,
but it's no longer on her site.
It's at webmention.io.
Bob is essentially in the same position,
but he now has in his world
a third party site called Telegraph.
When Bob wants to publish,
he no longer needs to go through all the work
of sending Webmention,
and of carrying out Webmention endpoint discovery.
He sends one API request to Telegraph,
effectively saying, please send a Webmention for me.
Asynchronously, telegraph.io is going to
retrieve Alice's post,
do Webmention endpoint discovery,
and send the Webmention on Bob's behalf.
Except this time he's sending it to webmention.io.
Now, at some arbitrary point in the future,
Alice can ask webmention.io,
"Hey, do I have any new Webmentions?"
And if she does, she may choose
to update her content and publish.
Okay, so let's code this up.
Now, I'm recording this talk
about a month ahead of time
and I haven't been able to put together
a little demo project.
Hopefully, I can hack something together
before this video streams.
But in the meantime,
I'd like to imagine a little test site.
It has a single page,
maybe we'll call it index.org,
and we're going to publish it.
We're going to export it to
a subdirectory of our project directory,
simply called www.
So, the entry point to the Org export system
is the function org-publish,
whose docstring helpfully says,
it publishes all projects.
The set of all projects is defined by
the variable `org-publish-project-alist`,
which is a very flexible association list
that lets you define
what files are in your project,
how you wish to export them,
where they're going to go, etc, etc, etc.
So great. This is actually pretty straightforward.
We just give ourselves a little Elisp file
and a single function.
I'll call it publish,
and all it will do is define
`org-publish-project-alist`
and invoke `org-publish-all`.
At that point, exporting is a one-liner.
We just invoke Emacs, load up our site.el file,
and invoke the publish function.
And if we want to publish to the live server,
that's just another one-liner of ours.
So, that's the publication framework.
Now, let's take a look at sending Webmentions.
The idea is that we're going to
get our fingers into the publication process.
Note when we see a Webmention in
the process of exporting our Orgdown,
and write it out to disk for sending later on.
Now, I didn't want to send a Webmention
for every single link in the post.
I wanted this to be an intentional choice.
And it turns out there are
different sorts of Webmentions
you can make on a page.
For instance, if you add
the CSS class u-in-reply-to to your link,
the recipient will
interpret this Webmention as a reply.
There are similar CSS classes
for likes, reposts, and generalized mentions.
When the recipient gets your Webmention,
if they want to know who's talking to them,
they need to parse your page
and look for DOM elements with certain CSS classes
defined by the protocol as well.
I'm a visual thinker,
so I kind of drew out the process of publication,
and exactly where we're going to
get our fingers into this.
So, this is me invoking make,
which of course fires up Emacs.
Just as before, my publish function will define
`org-publish-project-alist` with one difference.
There is an attribute,
a property in the list called `publishing-function`.
We're going to need to customize that.
As usual, we then call…,
we kick off the process by calling `org-publish-all`.
org-publish-all will invoke
your publishing function
for each page, and it will hand to
your publishing function for each page.
The name of the file you're publishing,
where it's going, and a parameter entitled plist.
This is not super well documented.
There are points in the docs
that refer to this as a communication channel.
What I discovered by simply reading the code was that
it's a property list that is initialized
for each invocation of your publication function.
The initial properties are
inherited from your project,
but you are free to add properties as you go
to communicate between different portions
of the publication process.
My publication function really does one thing,
and that's simply swap out the HTML template
that's passed to `org-publish-to`.
Now, in order to take note of
each Webmention that I send,
I took advantage of another
Org mode extension point
called defining new link types.
Here you can see
I've created a new link type called reply.
One of the attributes that you can attach to this
is the function that is used to export your link.
I've elided the code for
mentions, likes, and reposts.
If you look at my export function,
you can see that it ultimately yields
the appropriate HTML for this link.
Before that, it calls a little helper function
that will pull out the actual
target URL of the link
and shove it into this communication channel
under the property name `sp1ff/mentions`.
So, `org-publish-to` is really the workhorse
of the publication process.
The first thing it's going to do is
transcode from the parsed Orgdown,
which is an intermediate representation
known as Org elements, to HTML.
In particular, for every one of my new links
that I'm using to mark Webmentions,
it's going to invoke my little export function.
And so as we work our way through the post,
I'm going to accumulate all the Webmentions
that I've made in the property list.
The second step is to actually render
the final HTML document,
and that's where my specialized template comes in.
All I do there is, use it to get
my authorship information into the page.
And then the last step is called finalization.
At this point, we have the rendered HTML document,
and Org mode gives you an extension point here,
where you can do arbitrary post-processing
on that document.
I arguably abuse it to retrieve
all the Webmentions I've made
out of the communication channel
and write them to disk.
At this point, when we type make,
we wind up with the rendered HTML
for our Orgdown document,
along with a little text file
in which we've recorded all the Webmentions
that need to be sent.
The next step is to send said Webmentions.
This is surprisingly easy in Emacs Lisp.
This is my actual implementation.
I use the request.el package to talk to Telegraph.
And at this point, we really don't need to
add a lot to our little site Elisp file.
I sketched out a `send-webmentions` implementation
that just goes through in a loop
and calls my send-webmention function.
And now publication becomes a two-step process.
First, the org-publish, then sending Webmentions.
Okay, so I realize this has been
a bit of a whirlwind.
So, where are we now?
We have a sample site that we can publish
and have sent Webmentions.
And we've done it with just Emacs, Org mode,
a little Lisp, and a make file.
If you'd like to see more,
I've put my library up on GitHub.
It has logic for both
sending and receiving Webmentions
as well as something
that on the IndieWeb is called POSSE,
which is an acronym standing for
Publish (on your) Own Site, Syndicate Elsewhere.
What that means is that
you turn the publication step
from merely publishing new content to your site
to also replicating it to places
like Twitter and Facebook and so forth.
And also when people like, comment,
and retweet your content,
that gets fed back to your site,
where you can display it as comments.
In terms of the future,
I feel like I'm at a decision point.
Org mode is admirably flexible,
and I'm confident that I can continue to
add support for IndieWeb protocols.
On the other hand, it is so flexible
that the process of figuring out
which extension points to use in any situation
is very challenging.
When I started down this path,
my mindset was keep it simple
and let's just see how far I could get with Org mode.
And I feel like I might be bumping up
against the limitations of that approach now.
Thank you very much.