And so this little application--
well, I'll skip that and just kind of
jump right into my thesis for those of you
that might be planning to duck out for the RMS talk,
starting in a little bit.
So essentially, my thesis here is really that
the Emacs toolchain can easily be combined
with other skills and used in kind of
a Unix paradigm of having sort of different tools
to do different steps.
We might actually use the same tool
to implement a couple of steps.
But with that paradigm, each step
is an individual item that can be sort of
dropped in and replaced.
So over the course of the talk,
hopefully I'll come back to that thesis.
But I'll now jump back and start walking through
what is orgvm?
So this is a very simple proof of concept program.
We'll just jump over to perhaps
a prettier view of the source code for it.
This is implemented-- oops.
There's some cruft, I think, in my local.
All right, so this config block at the top...
And we'll be jumping back and forth
between the code and the documentation.
So the first thing I want to point out
is that this is written in Node.js,
but I think you'll find it'd be pretty trivial to implement
in any language.
Certainly, you're more than welcome to use this.
I'd be happy to accept your patches or feature requests
and things like that.
Of course, bug reports.
But I'd also encourage others to roll their own.
You might well come up with a different version of this
that's even cooler.
And we can learn from each other.
If you heard one of my talks before,
you probably recognize a common theme.
I'm a big fan of head-first development
as a way to get invested in both
the tool chain and a culture.
All right, so let's come back to orgvm.
First of all, we'll start with
the itch I was trying to scratch.
I wanted to be able to quickly use a web browser
to browse my Org documents.
It's particularly handy when the documents
are full of cross links to each other.
That meant I wanted to automatically export,
particularly to HTML.
But it made sense for me to include Markdown, PDF,
or whatever format I want.
Because many times, I'm going to look at that file
and then pop it into an email or upload it somewhere.
And then finally, it should be, therefore,
pretty easy to download the document
rather than view it, once I'm done.
So let's just run a quick demo.
You'll see I'm still a Windows user.
Yeah, I'm working on it.
So all right, first thing that we're going to do
is fire up the program.
Actually, for simplicity, let's just
admit we live in a DOS world.
And as you can see, there's not much to it
to get the application running.
So with that done, then, I can run out to my localhost.
And we'll just start by plugging in the name of an Org file.
So I've got a little Org file that I prepared
that just kind of provides a proof of concept to this.
And you can see, as imagined, we're automatically
turning that Org file...
Let's just take a quick look at it.
And here's that file now.
You can see, nothing up my sleeve.
This is a very basic Org file
that I use for testing this program.
Images work.
We've got some nicely syntax-highlighted code blocks
in a couple different languages,
and not really that much going on there.
All right, let's come back to the documentation.
I pretty well covered this, I think.
But you'll need a relatively recent version of Emacs.
I haven't taken any pains to make this backward compatible.
To be fair, I haven't tested it extensively.
It may well work on Emacs 26 or older versions.
I'm personally running 27.1 and 28,
as well as recent builds of 29.
There's some quick start instructions here,
which I'm going to take as read.
You probably saw the key element of this,
which involves starting the program.
You do-- I will call out Yale.
If you're trying to play with this yourself,
don't forget to run the npm install command.
That'll bring in express.js,
which the JavaScript we're about to look at is built on.
So let's just take a look at the usage patterns real quick.
To use this, we're simply giving the document name
without the .org extension in whatever file path--
or I'm sorry, whatever we've configured the server
to run on, in this case, port 3000.
I also want to call attention to the fact
that nothing in this program protects you
from damaging yourself.
This isn't meant as a production capability.
This is something that's used to publish
your own note files
and roll them out to yourself.
That's something I'll definitely look at adding,
but I want people to be careful of it
while this is in an alpha state.
So the default response is HTML, and we saw that here.
But we also can modify the response format.
We're currently supporting HTML, Markdown, and PDF.
And that's really enough to select a different format.
That's really nothing more than adding type. Okay.
Not sure what's going on there.
Okay, well, there goes my demo.
Shows me for doing my talk live.
But this, fortunately, this error message
is a nice segue to the part of the talk
that I'd really like to focus on,
hopefully bringing me back to that thesis.
So as we start to look at code, what we're looking for
is really this Emacs Lisp that's getting generated here.
And you'll notice that's the stuff
I thought was important to produce as diagnostics
for the programs running as well.
So, spoiler, this Elisp is dynamically
generated by the program.
And that's really the core of the way
orgvm or my orgvm works.
So this should look pretty similar to the view of the code
we had a moment ago.
You can see I've got some bases.
This is all hard-coded into the program,
nothing fancy going on here.
The debug is simply controlling that diagnostic output
that we looked at.
There's some other, hopefully fairly self-explanatory
programs or properties, where to find Emacs and so forth.
And then finally, we come in to the meat of it,
the variables that are used to control what Elisp
we can generate dynamically.
So here, we're controlling the extension
that it should look for Org files.
Hopefully not too many people out there
with a weird extension for the Org files,
but this should support that.
I'm afraid that is something I've been known to do.
Then we define a list of additional export types.
Here's one that ought to work.
Let's take a look at type=org.
And, aha, it's giving us the file.
So I'm not going to open that up,
but now we can see that that's definitely working,
for certain versions of working.
So this list of type parameters is
controlling the supported types.
Hopefully it should be fairly easy to add in different ones.
The fancy footwork here is just a list
of the types that we're going to be using.
The fancy footwork here involves, first of all,
there's the extension and the MIME type.
That's, as you might guess, used to control
the response content type.
We also have this replace variable.
This prevents-- there's an optimization
to send an existing PDF or HTML file
if that's already there,
but only if the original source Org file
hasn't been modified since.
This `replace` effectively can turn that off.
If I remove the `replace: true` attribute,
then I'll be prevented from overwriting that.
In other words, I'll always send a cached version.
That might be helpful if, for example,
you've got hand-tuned PDFs
and you don't want to accidentally overwrite them.
All right, let's get into the code a little bit more.
I'm going to skip past the really good stuff
and jump into the boring parts
so that we have them as context.
Here's the default path.
And it is going to send me the readme from the project--
from the project repo if I don't specify a path.
And then we have a couple of different endpoints
that we support.
We'll come back to this first one.
For now, let's start with the more normal one,
which is just giving us a file name.
So we can see we start by figuring out
what the physical file name should be called.
And assuming that that exists--
sorry, I've confused myself.
So this is the caching or the optimization
that I mentioned, sending the existing file.
This file exists is where the optimization is
that regenerates the file if the source
or document for the HTML generator has changed.
Again, this is a short talk, so I'm not
going to go into all the nuances of this JavaScript code.
It's pretty far from an Emacs-related thing.
So with that said, then, the rest of this program
is really mostly just handling the different errors:
"I didn't understand that type."
"I don't know the document."
"I failed."
Otherwise, there's the caching.
And here's really where things get interesting,
where we've generated some Elisp,
and then we're calling Emacs with that Elisp.
If everything works, we'll send the file.
If it doesn't, we'll send the 500.
And we've already seen the 500, so we know that works.
All right, let's get to the interesting part.
Sorry, one more footnote.
There is a capability built in that will
allow us to execute an Org block.
Let's see if that's working in our local.
I'll remind myself how to do it.
It's run.
I think it's called test.
And that's returning a 500.
I'm suspecting that's running because I'm running
in command instead of bash.
Oh, yeah, so the failure is happening
after I generate the Elisp.
I'm pretty confident that is what the actual problem is.
If we have time, I'll jump back over there
and relaunch it in mingw bash.
And we can see it actually work.
But this works pretty well for me on my work laptop.
I didn't have to make any changes to it.
So I have a fairly high amount of confidence,
at least in trivial cases, this works pretty well.
All right, so what I actually wanted to talk about today--
and I'm going to be kind of hand-waving around
this ES5 class that I've got and kind of the way that works.
Hopefully, this will be pretty familiar to you
if you are a JavaScript programmer.
The interesting stuff comes when we want to build some Lisp.
Here, you can see that I really don't have
a whole lot of code around formatting LISP.
You can see that I've special-cased
whether the arguments that were passed
happen to be a function.
If they are, I'm going to call that function.
And then the result will be formatted as Lisp.
So this would be a recursive call here.
Otherwise, I'm just going to return the arguments.
Sorry, otherwise, I will slap a pair of parentheses
around the result of walking that list if I get...
formatting each element of the list of arguments
that this `formatLisp` process calls
and separating them with spaces.
So in short form, this program walks through a list.
If the list it receives is a function,
it calls that function.
Once that's handled or otherwise,
we simply walk the list, taking the arguments,
concatenating them on strings, and finally,
wrap the results in parentheses.
So what I didn't mention there but might be obvious
is if I have a nested list, the inner list
will be subjected to the same treatment.
So this is a recursive sort of algorithm.
All right, so now when I go to export,
actually, in the interest of time,
I'm going to avoid walking through that piece of code
and let's focus instead on the more interesting part
of how that Lisp gets encoded.
So coming back to the PDF is a good example here,
because it's got a special case.
You can see I've specified this `exportFun`
or export function.
That's a property none of these other types have.
And you can see it contains some Elisp telling us
how to call the export for it.
Let's go see how that's used.
At the very end of what I just skipped over,
the detailed "how the Org export process works,"
you'll see that I am ending with a step
to call the export function.
Here, I look to see whether I have
an export function property.
If I do, I call that function.
And if I don't, I build this list with the default
`org-export-to-file` function
using the filename and an output filename.
So this, hopefully, is pretty familiar to anybody
that's manually messed around
with calling `org-export-to-file`.
If it isn't, you can pretty well trust me for it.
There's nothing very special going on.
This looks rather like...
Poor example there.
Let's go back to our markdown.
And there, we can see--
[Leo]: I'm going to make a quick announcement.
Can you hear me?
[Corwin]: Yes, go for it.
[Leo]: OK, let me just show my face.
Oh, I'm not showing my face.
Damn it.
OK, I'll make the announcement.
You won't see my face quite yet.
We are about to get started.
Well, we actually just got started on dev
with the talk by RMS.
So if you want to hop over to watch the talk by RMS,
feel free to do so.
Otherwise, we will be continuing on Gen with Corwin
to finish his talk and have a Q&A.
Corwin, you can feel free to go now.
[Corwin]: Okay, bye, everybody.
And for those sticking around,
I'm just going to keep pressing on with this.
In fact, I'm going to dive back into the part
that I skipped here, which is the rest of how
this export functionality works.
So just to make sure the dot is tied together,
the core of how this program works
is generating some Elisp and then passing it
to Emacs in batch mode.
So if that wasn't perfectly clear,
that's really what's going on with this program.
The rest of the implementation is just
a way to do that or certain features that
are supported in that generated Elisp, if you will.
So this is, you could say, the minimum implementation
I could come up with to create a web server
for my local Org documents.
And I will also interrupt myself to just pull up
the Etherpad real quick.
Actually, if somebody is listening
and can share a link to that, I closed my browser window
with my links in it.
But sure, I'm happy to take questions at any point, Leo,
if there are any questions for me.
Are you hanging out with me,
instead of watching RMS? You can go.
I'm teasing.
[Leo]: No, I mean, we know that some people can
have both streams open.
It's fine.
And right now, it's not the Q&A with RMS.
It's just the presentation.
So feel free to hang out a little longer
if you just want the live stuff.
Don't worry about it.
You're fine.
[Corwin]: Yeah, and forgive me, everybody,
if you were hoping for a quick, succinct talk.
I happen to know I was going to be opposite RMS,
so I awarded myself the liberty of rambling.
So if you do have a question, something that I alluded to
and haven't come back to yet, you should, by all means,
prompt me.
[Leo]: Corwin, I might do--
I'm just giving you a little heads up.
I might need to go help at some point of dev.
So if I need to do so, I will let you know right now
inside the BBB room,
and you'll be on your own to manage the chat.
And you can just talk backstage to us
to manage what we do with the stream, OK?
[Corwin]: Yep, that should be no problem at all.
I've got my pad up now.
Thank you, ??.
And I'm sorry about butchering your name there.
And yep, I've got my chat open.
And I think I'm pretty well set to self-manage.
Oh, I don't have a camera on.
So you can't see me giving you the thumbs up.
[Leo]: Okay, good.
All right, so let's just walk through,
because it's sort of an interesting code.
Let's just take a look real quick
at how we generated our Elisp here,
because it is--
there we go.
It is a little bit interesting.
So here is the method.
So I didn't get into detail on this.
But there's an ES5 class that represents
an Org mode document.
It has the static debug property that,
as you might imagine,
can be overridden by that debug setting
we looked at in the defaults.
We also have a static variable that--
a static property that does nothing more than getting
the path to Emacs out of those defaults.
Similarly, we have a class method
to spawn out an Emacs, as I mentioned, in batch mode,
eval-ing some arbitrary Lisp that's passed in.
All right, so the type--
this is where things start to get interesting.
So this is an implementation detail,
but-- that it's written as a static method.
But essentially, what's going on here
is looking up from that type list
to try to find a type that's passed in,
and that's returning one of these blocks.
Let's say I requested HTML, which would be the default.
Then I'm going to get this set of properties back.
All right.
Essentially, this program generates a program
or a little block of executable elisp.
However, in some cases, where if the `load-path` has
been customized in that type block,
or I think that's the only case I supported.
There was another complexity I removed.
So in that case, then I can simply
replace that program with a let.
Either way, I'm going to have everything I generate
be encapsulated in a single block.
The-- then I'm calling that formatLisp process
that we talked about, appending to that--
or inserting into, you could say, the outer scope.
And we start by finding the file.
We then load any libraries that might be needed.
In some cases, the type might not
have any external libraries.
So we just-- so that's a no-op.
And then finally, we're going to execute
that logic I mentioned before about selecting
either the default org-export-to-file,
or else whatever Elisp we've staged for exporting
that particular file type.
And again, in the case of PDF, there's a special function
that's used to trigger that export.
Or you may be aware that that's a little more complicated.
There's intermediate forms there.
All right.
So just reminding myself if there's anything else
I have to cover on background.
And I think that pretty well covers the basics.
All right, let's look at that source block execute.
This is the other use of the format list function.
So here, rather than looking at the type
and passing that through our Org export method,
and then that type is used to get the list
that we want to create.
In the case of source block execute,
we're kind of rolling it a lot more by hand.
So this gives us a good chance to sort of unwind
how that list looks when it's staged as JavaScript data.
So here again, I wrap everything in a `progn`.
I start by preventing an interactive prompt
for the Babel execution.
And then we load languages.
This relates to another piece of our configuration
where we've specified a set of languages
that it's OK to execute.
So if that type isn't in this list,
then we won't be able to execute it in line
through our trivial little web server.
All right.
With that done, then, loading the selected language,
we then once again open the file.
And we're-- whoops.
Let-bind a return value, which is
calculated by using Org source block execute [`org-sbe`]
on the name of the block that's given.
And then we use a temp buffer to write that out
to a temporary file.
This is actually a little clumsy,
but I haven't put the effort in to have this written out
to the standard output cleanly instead of using a temp file.
So under-- this is another example of where it may not
be production-- well, it definitely
is not production-worthy code in that under heavy load,
this would certainly break with collisions
on the Babel file,
the name of the Babel file.
In any case, once we've staged up our Elisp, which is--
this is basically variable interpolation,
then we just call Emacs on that.
And if we look down to where that's called,
you can see that the Org Babel filename calculated here.
[Leo]: Is there a problem?
[Corwin]: No, I'm fine.
I'm just lost in my code.
[Leo]: OK, cool.
Uh-oh means, oh, I need to intervene.
What is going on?
Carry on, please.
[Corwin]: No, I'm fine, Leo.
Thank you.
All right, so then--
so you can see we get--
we send the Babel file here,
which is calculated manually.
A bit sloppy there, since I have essentially the same--
I have two different places where
I'm calculating the Org doc file in two different ways.
Have I encouraged you to write your own yet?
Or send patches.
All right, so that's pretty much the nuts and bolts
of this program.
Let's go back to just seeing if we can't make it run.
All right.
All right, well, I apologize for not
having taken the time to stage my demo this morning.
I'm going to try to make it better for you.
But apparently, it's going to be non-trivial
to make the program work.
Let's just-- before I completely give up,
let's go ahead and try our Babel execute.
And that, too, is failing.
So there's something unhappy in my local world.
There it goes.
But in any case, let's go ahead and just take a look at
that.
Let's see.
Control Enter.
Let's take a look at that generated .el
and compare it to-- whoa--
and compare it to--
I'm just going to format this manually,
because I've forgotten my key bindings to auto-format it.
There we go.
All right.
So now we can see, as promised, there's really
nothing going on here other than the interpolation
of the variables in.
We're inserting-- we're using an insert and write file
method, which is, again, rather sloppy,
to generate the text file.
All right.
Let's come back to our documentation
and see if we can put a bow on the project.
So I hope I've convinced you that this was actually
rather easy to do.
The entirety of my index.js file is 262 lines,
and that includes a good 40 of whitespace and configuration.
It has only one dependency, the Express,
which really builds the web server.
Any language you'd rather implement this in
will have a similar capability for building
some type of trivial web server.
And I think you may find--
I certainly found that a large portion of the code base
is really making the errors meaningful,
in that, in some cases, sending an appropriate HTTP status
based on what happened.
In other cases-- let's see if
I've got an explicit `throw` left in here--
in other cases, just trapping different types
of failure conditions.
I'm going to look at my pad, and I do see a question here.
So let me jump in here.
[Leo]: Corwin, just to make sure, are you switching to Q&A?
Are you finished with your presentation?
[Corwin]: Well, as I said, I'm happy to take Q&A throughout.
But yes, let's say yes to that.
[Leo]: Okay, so Corwin, what I'm going to need to do now--
you are in charge of the room.
We are going to open up the room so
that if people have questions watching right now on Gen,
feel free to come in.
And there was something else I needed to say.
Yes, Corwin, if there's any problem,
whisper to us on Mumble.
So you might want to unmute Mumble
and be able to listen to us over there.
[Corwin]: I can't do that, Leo.
If I unmute, Mumble is going to bleed through.
[Leo]: Okay, sure.
Well, if you have any problem,
type in #emacsconf-org channel,
and we'll be with you, OK?
[Corwin]: Or I'll PM somebody.
But I don't anticipate having any problems.
I'll put something in -org when I run out of steam here.
How's that?
[Leo]: Amazing, cool.
So I will have to leave the room, though.
I'm leaving the recording going so that we have your Q&A.
And whenever you're available--
[Corwin]: I'll shut off the recording when I close the room.
[Leo]: Okay, great.
Good luck, Corwin.
[Corwin]: Thank you.
All right, and if you're still with me, well, thanks.
I appreciate that.
I did offer to be opposite RMS.
And I'm in no way offended if people do want to jump over,
especially as that starts to shift over to Q&A.
I'm taking Leo's leaving as a pretty good indication
that that's happening now-ish.
So I totally understand
if folks are more excited to do that.
Meanwhile, let me just jump over to the question
that I received.
I'll show the pad here so that I save myself
reading the question out.
But I'll paraphrase it.
Why am I not running the web server in Emacs?
That would be a great way to do it.
I chose to build it in Node.js
because that was trivially easy for me.
And then finally, am I using org-info-js?
No, I learned about this essentially at this conference.
So that's something I'll be learning more about.
And it could well influence this project.
All right, and thanks for the questions.
All right, I'm going to slow my roll just a little bit here
because I think I kind of have all the time in the world.
I will be wrapping up within about 15 or 20 minutes
at the latest just to avoid stressing out
my fellow organizers, especially Leo and Sacha
that have the bulk of the heavy lifting this year,
and Amin, and really, thanks all to everybody.
God, the nicest part of doing my own talk
is that I get to say that.
It's just so much fun to contribute to EmacsConf.
And if you're at all interested, there's
plenty of completely backstage, behind the curtain role.
Behind the curtain roles doesn't mean
you have to be somebody that likes
talking or being on webcam.
Sorry that my camera isn't working this year.
I spent quite a while fussing with that
and lost all my time to get my prerec working.
All right, so trying to think where I can take us
without my demo working.
I was really hoping to show the Org Babel piece.
That's really fun.
So let me just mention briefly how I'm using this at work.
So at work, I'll have some type of Org document.
And usually, it's a project.
So the title of the document is My Project.
And then I'll have a requirements section.
And I'll have a meeting notes section.
That's probably the key thing.
And then as the project goes on, I'll start having--
I'm a solutions architect.
So my job is formalizing design in large part.
So then I'll have a design documents section.
And this is where I'll be doing a lot of my work.
So I'll start out saying--
And maybe Bob is a subject matter expert
whose buy-in I need to have on how we're going
to do the high-level design.
Maybe a lead engineer or a dev manager
or something like that.
All right, as my work goes on,
then this will start getting into more detail.
And things of this nature.
As things get further and further,
I'll actually have documentation
that I'm adding in here.
Oh, I see.
It's a big mess.
All right, well, we'll just reuse this.
So I can insert those all in line.
And now for the fun part,
let's see if the most trivial case
is working here.
No.
All right, completely broken.
Let me drag.
All right, well, apologies again for the poor quality
of my demo today.
And let me just look real quick at my Etherpad once more.
And I'll glance at BBB to see if there's anybody
jumping in with questions.
And then I'll go back to IRC and look for questions there.
OK, and I don't see any additional questions on the pad.
I'm just going to scan IRC real quick.
I suspect that the TreeSitter comment isn't for me.
All right, and I'm not seeing a lot of questions there.
So I'm just going to vamp for just a minute or two.
As I mentioned, I'm a conference volunteer.
This is my third year volunteering
with the conference.
And probably if you take one thing away from my talk,
it should be I really like volunteering
for the conference.
It's fun.
It makes me feel sort of close to the pulse.
And it gives me a chance to just interact with people
that have very different perspectives on Emacs,
which is something that I really value a lot.
Emacs, like anything else sort of in the internet world,
has a real echo chamber factor.
If you do or don't like use-package,
you probably interact with a lot of people
that feel the same way about that.
And so I really recommend volunteering for EmacsConf
as a way to sort of mix it up and get
to know people that may not use Emacs
the same way that you do.
Or perhaps more on topic, though,
the log line for this talk is it's really quite easy
to build a program that uses Emacs in a pipeline capability.
I think there's a ton of opportunity in this space.
This particular example is just a trivial web server
written using Node.js.
But as was pointed out, we could have used elnode
as a web server
and done the entire thing within Emacs Lisp.
Or really, almost any technology
would get us this capability.
From an implementation standpoint,
I had a lot of fun building
this trivial little Elisp parser,
and I'm rather pleased with the fact
that the entirety of that--
the entire algorithm for turning JavaScript or JSON data,
we could say, into Elisp is really a one-liner,
albeit a nasty one-liner. That was pretty cool
to discover how simple that was.
So in my mind, that opens up a lot of possibility.
If it's this easy in JavaScript,
I wouldn't expect it to be hard,
any more difficult in your favorite language.
Glance one more time to see if there
happen to be any other questions.
And not seeing any, I'm going to go ahead and start
wrapping up my chat now.
It will take me a couple of minutes to do that.
So if you do have any other questions that you
want to drop into the pad or any comments,
you're more than welcome to hit me with those
as I coordinate closing this chat, this talk,
with the organizer team.