Hi, this is Dave O'Toole, and today
I'll be giving a presentation on tracking health data
with Emacs, Org Mode, and Gnuplot.
So Gnuplot is the well-known scientific
and mathematical plotting application.
You feed it text files full of names, dates, numbers,
data points, and you get out a nice graph.
You can spit out SVG. You can spit out PNG graphics.
In this case, we're using an SVG.
What I'm going to show you today
is how to take daily health journal items:
in other words, things like I exercised
such and such number of minutes today,
I got X hours of sleep last night,
I used such and such number of pieces of nicotine gum,
say five pieces. So let's see,
we've got this whole picture here, all right,
and I've tracked here...
This is a month of data from my life.
This is... I'm not showing all the variables,
but this is what I felt comfortable sharing
in order to help people who might have a need to track,
either because of a chronic condition,
or just because of a health improvement goal
or what have you, people who might need to
track health data in a way
that's a little bit more robust
than just one or two variables
and just weight or just blood pressure.
So in this case, I've got exercise,
I've got the number of hours of sleep,
the number of doses of nicotine,
(that's the yellow line here),
and this is referring to nicotine gum.
What we're going to be talking about
is looking at connections, the idea
that plotting your data can actually
help you figure out what's going on.
This is just one month.
I've been doing this for a couple of months now,
but I felt comfortable showing one month
with a limited subset of the variables.
What I'm going to be doing in this presentation
is showing you how to set up your org templates
so that you can, you know, hit a hotkey
to capture today's data with an org template--
or in this case yesterday's. Usually I'm saying, okay,
yesterday this happened,
because you don't know until the day's over
how many pieces of nicotine gum you ate
or how many hours you slept.
So usually we're recording data for the previous day.
We can set up a capture template
so that it fills a little org entry. One for exercise,
one for sleep, one for nicotine, one for distress.
Here distress is just 1 to 10:
how bad do you feel today?
It's not a scientific measure, but you know,
many, many things ask you to rate
on a scale of 1 to 10, how bad is the anxiety,
how bad is the general level of stress,
and so without a lot of complication,
I just rate that one to ten.
Pain, okay, we won't have to get into any details,
but if there is a level of chronic pain, well,
I put that between 1 and 10. As we can see here,
during the period that I've shown you, it's pretty low.
There's some. If you miss a dose of medication,
you can track that, in this case
with a big ugly red triangle, you know.
You can see, I can see here that in mid-, in late September,
sorry, in early to mid-October,
I stopped using the nicotine gum
and probably should have cut down more gradually
because my sleep suffered. Look at this.
The sleep line is down here, okay?
What I'm going to do now,
now that I've shown you the graph
and some of the things that are useful about it,
I'm going to actually take a step back
and show you from start to finish how you can do this
in GNU Emacs, and I have a little template generator
that you can use if you'd like.
All right, so let's go back.
Let's step back from this file.
We're going to split the screen, and on the left side,
I'm going to put the underlying Org file
that generates this graph.
Let me shrink that a little bit.
All right, I'm going to work my way backwards
from the template to the template generator,
meaning that you'll be able to spit out,
given your own specification of health variables,
you'll be able to have it spit out
a custom Gnuplot script like this
that's preset up with the definitions
for the column view in Org mode.
I'm assuming a little bit of familiarity
with Org mode and Gnuplotting,
but I'll try to explain as much as I can as I go along.
The journal here is where... okay, okay, one moment.
So as you can see, there's a sub-entry here
for each day that I've included from my data set
starting on September 13th of this year
and ending on October 17th.
And there's an Org property drawer with
the corresponding names of each field and the value.
Now the idea here is that the columns specify...
if you know a little bit about Org mode,
what happens is that you...
let's say that I hit the key for my journal template,
which... Mine is very similar.
This is the capture buffer for today's date,
and if you're recording yesterday's date,
you can just flip it like that if you need to.
Then I say, yesterday, I remember
I went for about a one-mile walk,
so that's probably about 20 minutes,
and that I had such and such,
I had eight and a half hours of sleep, let's say.
I estimate how many pieces of nicotine gum I have.
I try to count as closely as I can, how much distress,
you know what I mean,
whether or not I missed a dose of medication.
Then when you hit C-c C-c,
it captures that to the end of your Org file.
Now what this shows is that... I cut and paste it in.
I've been keeping these entries every day for months,
and that I cut and pasted in a month of data.
Now I'm going to dig in a little bit to the Gnuplot script.
This here, all this stuff, is one component of the graph,
and I'll go over how it works.
First, the items through this column declaration here,
and the id:myid, this columnview table here,
#+BEGIN: columnview, this whole bit here,
is going to get filled in with the corresponding columns,
exercise minutes, sleep hours, nicotine doses.
And then it gets pumped out by Org mode into a file
that looks like this: tab-separated values
with an ISO-style date at the beginning.
So what we're going to do is we're going to go through
the Gnuplot portion of this,
and I'm going to enlarge the font a little.
I'm going to go line by line through the Gnuplot portion.
Now, my template generator will give you one like this.
You don't have to write this from scratch.
But I'm going to go through it line by line
because if you do use the template,
then it'll help to have gone through it line by line,
because you're probably going to have to modify it.
So first, we're going to clear the graphics
from any previous runs
so that if we reuse the same Gnuplot process,
we're not overwriting the old--
that we are completely overwriting the old image.
So that's the purpose of this line here.
The output parameters: we want to put out an SVG file.
Font Arial, that's funny,
but I don't know what font it's actually ending up choosing,
but it looks fine.
Then we want it to be square,
so I'm giving it 900 by 900 pixels,
even though it is a scalable vector graphic.
We're putting it in the same folder as the org file,
example.svg.
These lines here set it up to use the Org mode format
that we showed in the other file over here.
The time format is four-digit year, two-digit month,
two-digit day.
The time format doesn't specify here the time,
but that doesn't seem to mess it up.
This line "set datafile separator" means that
the separators between that
and between all the other fields are tabs,
which is what Org mode does
when it spits out a table by default.
Okay, along to the next lines.
We're going to set up for time series data,
meaning that the x-axis is going to be time,
x2tics 1 format.
I believe this means that every day has one tick
and that this tells it that the first--
unfortunately, I forget the exact meaning of this one line.
I'm just going to move on. We want one X tick per day,
and because X is in seconds,
it's 24 hours times 60 minutes times 60 seconds.
This line "set grid xtics" gives us
a vertical line on each day of the graph.
I'll pull up the graph
just so that it's a little easier to see.
All these vertical lines, one on each day,
that's given to you by "set grid xtics".
One Y tick every five points.
So here at five pieces of nicotine,
we've got a five, at ten pieces – well,
we don't want to eat ten pieces, but ten, fifteen, twenty.
Rotating the labels to make them fit a little bit better,
that's this part here where the labels are sideways,
and even with just one month of data,
they're getting a little crowded.
This "set key box lc" just makes the line around the key,
the legend here, a little bit less severe.
set xtics format: this makes it so that, for example,
I've done a United-States-style date here
with the month and then the day.
You don't necessarily have to do that.
You can have whatever you want.
This xtics format,
that relates to how the dates are printed.
Remember that over here, this set timefmt,
that relates to how the dates are formatted
in the Org mode output.
So remember, those are two...
You don't want to mix those up.
All right, "yrange [0:40]".
Thus far, my exercise sessions have all been
less than 30 minutes, and nothing's gone over 30.
If you have a health variable
that is in a significantly different range,
you may need to get a slightly more complicated
Gnuplot script because it is possible to plot
multiple yranges in one plot
if you have a variable that uses a different range.
It's just a little trickier.
These parts here, aside from the fact
that you might make some changes that relate to
the date and your country format,
are going to be the same.
This is like boilerplate for almost anything.
Now here are the parts that are going to vary
depending on what health variables you want to store.
There are three main sections here.
One is setting the different line types that are used.
Setting linetype 1 with line width 2, line color RGB.
Unfortunately, Gnuplot is a little bit cryptic,
which is why I've made this template generator
that I'll show you in a moment.
I pick a color. So this is exercise, forest green.
Point size 1, meaning you get
these little green triangles about that size.
But the point type 9 is the pointing up triangle.
Line type 2, purple. So that's the sleep line.
So we're just establishing these different line types
that we've given arbitrary numbers.
Now onto the next section.
Oh, before I move on here,
you can see point type 11 for line 5, which is red.
And that's the missed medications line,
so you get a triangle that's upside down
because that's point shape 11.
All right. The next section here is the goal lines.
There are horizontal dashed lines here
at 8 purple hours of sleep, because 8 hours is the goal.
So there's a horizontal line at Y = 8.
For pieces of nicotine gum,
I'm trying to keep it to around 5 right now.
So my goal line is at 5. So these...
Here, a goal of at least 20 minutes of exercise.
Sometimes I get more, sometimes I get less.
There's a green line and a 20, showing that that's the goal.
These lines here are actually the goal lines.
You can specify the goal for each one
in the template generator that I'll show you.
The last part is the actual plot command.
So the dependent... So okay,
these all start with 1, "using 1" against this variable.
So $2... This is a ternary operator here
that says if the value of the second column is zero,
then don't plot a point. In other words,
not a number means it won't plot a point.
The template generator lets you skip over
the details of that. It sticks this in there.
I'll show you.
So we only want to plot a point when the value is non-zero.
If there was no exercise, we're not plotting a point.
The with construct means we'll plot data
using date against exercise with points,
the title is "exercise (minutes)", line type 1.
Remember, we established line type 1 up here
as being forest green, point style 1,
point type 9, green triangles.
Now I'm going to show 1 against column 3,
which is "hours of sleep".
This one is plotted with lines,
so we don't specify a point type or point size,
just a line type 2. And remember, you can see
that line type 2 is defined as purple
with point type 1, point size 1.
Okay, so I did specify point size and point type,
but because I'm not plotting with points,
those are ignored.
Here we come to the line with nicotine.
The fourth column is the nicotine number,
the fourth column from the Org mode file.
So here you can see how we're telling Gnuplot
to take each column of the tab-separated Org mode file
and put it into the graph.
The line types are set up here.
The goal lines are set up here.
And then the actual plot command is set up here.
So now we're going to work further backwards
from this Gnuplot template
to the template generator that I used to make it.
Now I'm not going to go into
all of the details of the code,
but what I am going to show you is that
there's a variable called `health-factors`.
And what this does, this `health-factors-from-list`
lets you specify, with a property list
of keyword and value pairs
(here's the keyword name and the value is exercise),
the goal that I want 20 minutes of exercise,
that the unit is minutes,
that the color is forest green, and so on.
The aspects of the Gnuplot setup
have been abstracted here.
Eight hours of sleep is the goal here.
The hours are units. What color,
what thickness of the line.
Here we specify the number of points.
There's references online
that show you what point types are what shapes in Gnuplot,
and so on and so forth.
I'll walk through the code a little bit that does this,
that actually takes these pieces,
that takes this specification of what your variables are
and turns it into a template.
First, I'm using EIEIO,
the object system that's included with GNU Emacs.
It's a reasonable facsimile
of the Common Lisp Object System.
What I'm going to be doing here
is defining a class with each of those items,
those properties that we talked about in that list
that lets you specify name, what the goal is,
what the units are, and the Gnuplot things
(the Gnuplot parameters like thickness,
plot type, and all that) into a class that will then
spit out the template once you feed it
some of these health factor objects. So just a moment.
For example, you can see that this template
originally came from being generated by this code here.
To use the template,
to use this little template generator...
See, here's where it spits out the line type
given the pieces.
This is all just text formatting.
This is one of the things that Emacs Lisp
just really excels at.
I need to take a piece of data
like a list of health information,
a list of health variables, what their units are,
and how they're supposed to be formatted in Gnuplot,
and go from that to the nice template.
So that's pretty much the whole thing.
I want to see if there's anything I missed.
Bring up the chart.
This has been really useful
for communicating with healthcare professionals
because you are both on the same page
about exactly what is happening,
what's been happening because if... Let's say
that you're tired when you talk to your care provider.
Well, if you have objective information
that you've been recording every day,
that you're ahead of the game, really,
because you don't need, necessarily, the presence of mind
to be able to give your care provider
a complete picture of what's going on in your world.
If you can find those few minutes a day to enter--
not even a few minutes,
really just a minute to enter the data
and say what happened yesterday...
I'm finding over these months
that I've been more in touch with my health when I can--
not forced, but when I have the habit,
the consistent habit every single day
of recording that data--I'm accountable to myself.
It's interesting.
I guess it gets into a little bit of ideas
about the Quantified Self
and how holding yourself accountable
can change what you do and what the outcomes are.
Just look at this here.
Without getting into too much detail,
one of the reasons I track my sleep is because,
as you can see, my sleep
is not as well-regulated as most people,
and that's why I need to do that.
This was a time... 10, 12,
here's 14 hours of sleep, that's depression.
It oscillates a little bit. But then below the goal line,
the things are a little more normal here.
This is a little more normal.
But then, really, without thinking about it too much,
I cut out the nicotine, and my sleep suffered.
Just the fact that I'm able to look and see that connection
is really amazing to me.
Maybe I would have anyway,
but looking at the whole months of data,
there have been many things to discuss
and many things to think about.
Because this is a short presentation,
I probably should wrap up.
I just want to thank the whole Emacs community
for being there and for including me in the conference
and I hope to participate next year as well.
Thank you so much.