Good evening again. I think
I have a little time here
to talk about macros.
Is there still room in our schedule for that,
or should I just jump to some of
my thoughts on the day?
([Amin:] Pretty sure we
have some time.)
[Corwin:] All right. Great. ([Amin:] Yeah, go for it.)
Well, I'll just dive into my
prepared thing here then.
(Amin: yeah, actually, you're right on time, so...)
[Corwin:] oh what an amazing thing.
I just... You know, I have been trying to do what I...
I've got a big thank you planned
at the end, but let me just say,
it's been really cool to watch the
way that people work together.
([Amin:] Absolutely. It's... This whole event today has been
nothing but awesome, and
no little part thanks to all of the help
from all of you guys and everyone.
Yeah, it's awesome.
With that, I'll just shut up for now.
Take it away, Corwin.)
[Corwin:] Who knows how to make
make that the default in good old smex?
All right. So I'm gonna try to continue
my theme from the previous talk.
I'm a longtime Emacs user,
but I'm a pretty new person
to trying to really understand
what's going on within Emacs and make
my customizations to it--simple for
what I tend to just think of will work.
And maybe that's a nice bow
to put on that earlier talk.
Let's see here now. It's C-x M-i.
That's right.
And let's try that again. Okay, good.
So demoing is fun,
but I will save most of that
for tomorrow where my
dear friend and co-collaborator
in bringing you the dungeon-mode project,
which is sort of the exciting thing
that we hope you'll be interested in,
gets a little more of a reveal.
Tonight, I'll just close saying
a few things about the process of
making it and continuing my theme of community.
First of all, a specific and upfront shout out
to tv's wasamasa who
absolutely shaped and guided this
this program. I may have taken out
a slide with your name on it,
but thank you.
So when we think about Emacs macros
and the power that they give us,
I think about them as a really
deep rabbit hole. They confuse
people a lot. And so, to try to center
myself on that, I remember first that
they're going to be talking to us about code.
Excuse me, I realize I hadn't set my timer.
Here we are.
So a simple macro syntax is
going to generate
something that is implicitly confusing
to somebody that knows the syntax
of Emacs Lisp well.
We see something like this
and a veteran eye says
"That x isn't quoted. What's going on?"
but it can be hard to miss.
A lot of the functions (as we'll talk
about in a moment) that are built into Emacs
really are macros, so a lot of
Emacs features work this way.
It might be scary, but we have to look at
it closely if we really want to get
friendly with Emacs.
Let's just jump right into defmacro,
which is our key entry point.
The notes from this talk
include the link to that,
which... Definitely
read through a couple of times.
That may take you through
into the cl-def macro, which adds
the Common Lisp extensions.
Definitely challenging.
I've struggled there,
as we'll take a look at
in a moment.
So I haven't played too much with
cl-maclet. Perhaps success in
in that keyword space
and figuring out what the right balance is there
will give me the confidence to try
some more lexical macros.
Let me also briefly introduce
the comma (,) and back quote (`).
If you have allowed your eyes
to cross when you see these,
that's not a shameful, shameful thing.
It's confusing, and we should be
alerting each other
when we stick macros in,
often by putting them in
different library spaces
for complicated projects,
or otherwise warning people
that this is not an interactive function,
even if you get away with using it like one.
Watch your back.
The manual itself talks about macros
as being a way of evaluating,
as being an evaluator that will take
our Emacs Lisp expression
and the set of forms that
will feed to it our code,
but it also provides us with
this concept of an environment.
That's really where the power comes in.
Through that, we can have
lexical variables and
think about--bring in some of the
capabilities that
can be harder to reach with
a pure declarative statement that
doesn't allow for top level
asynchronous... Asynchronicity...
I'm gonna basically
ignore the byte-compilation phase
for this talk
in order to have any prayer
of getting through it in the
remaining 9 or 11 minutes or whatever.
But suffice to say, that's a scary space,
and that's really
the thing that you want to
start learning about
as you think about
taking macros on in earnest.
Coming back to the comma syntax,
then, having given ourselves a
working definition for the Emacs Lisp
runtime environment, then we can say that
macros are going to
inject code back into that stream,
whereas backquote (`)
is going to give code back.
to the stream--or interject, sorry,
it's going to interject
back into the stream.
Sort of an exclamatory "Excuse me,
I'd like to have a value here." We can take that value
from the environment as it exists
when our macro is evaluated.
Backquote, on the other hand,
takes the result from that
and returns it back to the stream
for evaluation at the
processing level that invoked us.
So in other words, perhaps back up to
a top-level eval expression where our
macro is invoked.
I'm going to briefly bring
you back to the game
for just a moment.
I won't linger on this slide,
but briefly: this is a
role-playing, pen and pencil,
physical dice tradition
that dates back a long time
from a technology perspective.
It's old in the same way
that other tools that I like are old.
It's simple to understand.
I can communicate a lot with it
with a simple amount of typing
or scribbling something on a piece of paper.
It has a complicated problem space of its own.
Again, I don't want to
get too much into the game here,
but in this talk,
for the last five minutes,
I'll focus on the process
that we took to
automate getting data out of
the Org Mode tables which eventually
(as we'll talk about more tomorrow)
are used to draw
game maps and other things.
Here I talk about why we did that.
I'm going to skip briefly past that,
and say instead that at a high level,
it's symbolic informatics.
We're giving a symbolic name
to a tile set,
and then assigning that tile set some
some characteristics
like physical speeds, screen space
(a variable that we might
want to swap in), and so forth.
You know, our project rests heavily on
Org Mode and its fundamental capabilities.
The code I'm going to show here
is focused around a sticky
problem space in
information technology.
I'm a professional software engineer
turned technology architect.
I support the websites for
a recognizable financial services brand
that I don't identify
just so I don't accidentally end up
inadvertently misrepresenting my firm
in some financial perspective
if I let some other
companies' name slip, or my own.
It's certainly no representation
of an opinion other than my own.
So ETL has to do with moving data around.
We have the idea of
a pipeline where we'll be able to verify
certain assumptions,
nominally about data quality,
but it could be about anything.
Before the pipeline starts, okay,
we've got a state where we think
it should work if we run it.
We have some extraction where we'll
get our sources, and we may have
the opportunity to
make some assertions there.
In the transform stage, as well as the load,
things get a little dicer,
to the point where we come out of the
load stage and we should have some
really solid assertions again
that we can even go back
and compare to the extract stage.
From this, we have the rudimentaries
of a data quality practice.
In this case, we have a number of
Org Mode files that will all
be distributed across a
number of players' computers,
so we might not want to update every
part of every buffer.
I think it's a complicated problem space.
So we tried to take
a long-term view of
the solution that we needed.
So I'll go ahead
and open up the function that...
Let's actually start with the one
that's pretty easy to read.
I'm gonna go ahead and just crank it up huge,
in case anybody's watching in 480.
This program is not a work of art
It's a simple implementation of the idea that
an alist of functions
that return maybe some data,
maybe some data and an entry back into that alist,
can be done quite extensively with
very few lines of code.
Neither is it an especially tight
or thrifty implementation.
It's just trying to get the job done
with a doc statement for everything.
At the heart, we see a call to
this macro called dm-coalesce-hash,
and that's what I'd like to focus in on.
You can see... I think
that something unpleasant is happening here.
I've got an eval in what is...
I will share a fairly central function
that those implementing this ETL pattern are
welcome to derive from.
That is, this is a default transform
that you can get when loading certain kinds of
org-mode tables that have been
properly adorned. Again, we'll get
into that all tomorrow.
So, keeping an eye on time. Couple minutes left.
Let's look at the macro itself.
I have a slide on this,
but let's go ahead and risk getting off page.
Oh boy. Here we go. So this is my
utilities bucket.
It has such basic features as "give me a
hash table with some defaults, I'll think
about that later,"
and "add to list," a special version
that enables us to be a little cavalier
in experimenting with alist versus
hashes versus plists.
We've made a right mess for ourselves
in the proof of concept area,
and it's ripe for someone to write a
whitepaper about
when to prefer these things.
The merge alist...
Same work here. Let's get down to business.
This function has quite a...
This macro has quite a doc string.
I think I mentioned earlier that I got myself into
trouble with the keyword properties.
You can see that we have
not only quite a number of them,
but a lot of default values,
many of which may be relying on
the values that are passed in here.
This is complicated. As it turns out,
I wasn't brave enough in most cases
to try to write a lambda
that could understand and
replace its own local variable.
I just didn't...
It didn't save me enough time. This was
really easy to read and write and understand
as I thought through my problem,
but now, as I use it,
I've lost a little ground with this.
I'm not even sure
I like what I got from
the many keyword properties when it...
And we can look, perhaps if we have
the time, at what that looks like in
Oh, all right, I have to separately
dismiss and restart that.
So that's just about my time.
Being respectful of that, I want
to invite presenters to just jump in at
any of the many large pauses I leave.
I'll just leave up the doc string
for a moment and maybe split the screen
and pull open an item.
([Amin]: Thank you very
much for your talk, Corwin.
I think you still have
maybe three or four more minutes,
if you want to quickly wrap up.)
[Corwin:] Okay, so three or four more minutes
I can easily spend on thank yous.
I might switch to that if there aren't
questions on the pad.
([Amin:] Would you like me to pull up the pad,
or are you looking at it?)
[Corwin:] I am. I bookmarked it.
I am pulling the tab and I'll bring it in.
Okay. All right. This is the wrong Etherpad.
Thanks for the link.
All right. So I think I'm looking for macros.
Okay. Key message. Sure.
So, the key message is that it's
a jungle out there.
Macros, along with any other design,
can leave you in a position
where you have a nice API. I can show
you other examples (you can find them in
the dungeon-mode source)
of many, many other places where I use
this exact same formula,
quickly sketching out how a character sheet
or another big data set
needs to figure out what tables are
going to be interesting
from the collection of files,
and then load up the tile set,
and the layout file from that.
And I mean, it works.
The project is moving forward with this.
I have the flexibility that I need.
But here I am evaling my own code
to make darn sure even if I get
byte-compiled, this macro
does get evaluated in the user's real
run time. Clearly a design fail.
So that would be...
The key point of my talk is to
present this design fail and
thank the community, but especially
wasamasa for some patience.
Let me add at this moment that
he was so frustrated with me.
They were sort of frustrated with me
(I think I didn't qualify pronouns)
with doing this. The first...
This was one of our first interactions,
and the feedback was,
"Why is this a macro. Full stop."
And that's a great message, actually.
I hope that maybe this can encourage
further talks across the subject about,
you know, "Hey, wait a minute,
macros are really fantastic,"
as I hope I made clear.
You can do a tremendous amount
with them, and we rely on them
for almost all the fun goodies,
from defun, setq...
I want to get to my thank yous.
Let me just peek back at the pad.
Well, that was actually a scratch buffer,
so I'll have to read it cold off my notes.
But I'll switch to... I'll also...
I'll say a couple of thank-yous if you
don't mind, Amin.
In addition to the big thank you that I
hope was implied by my shout out to wasamasa,
I also want to thank you, Amin, for
your kindness in extending
to the project as well as to me, the
the chance to present here.
You've also done a lot of great stuff
for our project. Thank you very much for that.
Sacha Chua (I'll get there),
thank you so much for
the inspiration that you are to our
whole community.
I also want to thank the presenters
for just being so flexible and
nagging back through the whole thing,
and especially to Leo
who has done so much to drive the show today.
This is a fractious tent at times,
and sometimes it is indeed
a little bit of a circus,
but I am learning so much so fast.
I'm just inspired by how much
Emacs can teach us.
([Amin:] thank you, Corwin, for your kind words
about me, of course,
about all of us
and the conference...
Indeed, thanks to everyone who's helped,
including the speakers, of course,
without whom EmacsConf really
wouldn't have been a EmacsConf.
It's been a pleasure
knowing you and working with you,
from afar for the most part
on dungeon-mode,
helping with small things here and there
but yeah, it's been my pleasure,
and it's great to have you and
everyone else part of the community,
and for me to be part of the community.
It's been a lot of fun.
Thank you.)
[Corwin]: It's an honor.
I don't use that word an awful lot
because I sort of smirk at it.
It gets us in a lot of trouble, honor does,
but this will be a sure time to use it.
([Amin:] Thank you.)
[Corwin:] Likewise.