WEBVTT captioned by sachac
NOTE Introduction
00:00:00.000 --> 00:00:06.839
Hello everyone and thanks for tuning in. I'm Timothy,
00:00:06.840 --> 00:00:08.559
and in this talk, we'll be going over
00:00:08.560 --> 00:00:11.342
the 2022 Emacs User Survey.
00:00:11.970 --> 00:00:15.078
Since this is the first time we're discussing this,
00:00:15.079 --> 00:00:18.399
we'll be going over the survey itself a bit,
00:00:18.400 --> 00:00:21.199
how it's being put together and run,
00:00:21.200 --> 00:00:24.199
and then we'll have a little taste of the results
00:00:24.200 --> 00:00:26.039
with more analysis to be published in the future.
NOTE The 2020 Emacs User Survey
00:00:26.040 --> 00:00:32.399
To start with though, a bit of background.
00:00:32.400 --> 00:00:36.679
So in 2020, we had an Emacs User Survey
00:00:36.680 --> 00:00:38.839
run by Adrien Brochard.
00:00:38.840 --> 00:00:41.359
Now this is, to the best of my knowledge,
00:00:41.360 --> 00:00:45.559
the first time that a large-scale Emacs User Survey
00:00:45.560 --> 00:00:48.039
has actually been run.
00:00:48.040 --> 00:00:50.439
About 7,000 people responded to the survey,
00:00:50.440 --> 00:00:53.239
so in many respects, it was quite successful.
00:00:53.240 --> 00:00:56.519
And what's significant about this is that
00:00:56.520 --> 00:00:57.679
with this being the first time
00:00:57.680 --> 00:00:59.999
that a large-scale survey has been run,
00:01:00.000 --> 00:01:01.719
it actually provided some insight
00:01:01.720 --> 00:01:06.719
into questions about how the community is using Emacs
00:01:06.720 --> 00:01:09.959
that allow for much better guesses
00:01:09.960 --> 00:01:15.359
than just speculation based on the small number of people
00:01:15.360 --> 00:01:16.919
who respond on the mailing list usually.
00:01:16.920 --> 00:01:24.879
So, why are we doing another survey? Well, to start with,
00:01:24.880 --> 00:01:28.799
in order to get the most value out of an Emacs User Survey,
00:01:28.800 --> 00:01:32.519
it's quite helpful if the information in it is recent.
00:01:32.520 --> 00:01:35.439
Furthermore, we can actually get some more value
00:01:35.440 --> 00:01:38.039
if we can examine trends,
00:01:38.040 --> 00:01:41.199
shifts in the way that people are using Emacs,
00:01:41.200 --> 00:01:42.919
where the pain points lie,
00:01:42.920 --> 00:01:45.479
what people are enjoying the most, etc.
00:01:45.480 --> 00:01:46.520
So in both of these respects,
00:01:46.521 --> 00:01:49.599
it's to our benefit if the survey
00:01:49.600 --> 00:01:51.519
is actually a regular event,
00:01:51.520 --> 00:01:54.359
instead of just something that's run once.
NOTE The design of the survey
00:01:54.360 --> 00:01:57.159
Now, with this in mind,
00:01:57.160 --> 00:02:00.959
we ran the 2022 Emacs User Survey with the plan
00:02:00.960 --> 00:02:05.079
that this will actually become an annual event.
00:02:05.080 --> 00:02:08.999
In the design of the survey, there are a few goals here.
00:02:09.000 --> 00:02:11.520
The main one is of the user community.
00:02:11.521 --> 00:02:14.520
Now, user community is a rather nebulous phrase.
00:02:14.521 --> 00:02:17.520
In this case, what's meant in particular
00:02:17.521 --> 00:02:21.020
is value in questions, for example,
00:02:21.021 --> 00:02:23.839
things like pain points with Emacs,
00:02:23.840 --> 00:02:27.119
which versions people are using,
00:02:27.120 --> 00:02:30.239
which capabilities people are making the most use of,
00:02:30.240 --> 00:02:34.519
which could potentially be helpful to both emacs-devel
00:02:34.520 --> 00:02:36.520
but also our collection of Emacs package maintainers
00:02:36.521 --> 00:02:38.020
and the whole community.
00:02:38.021 --> 00:02:40.799
Actually, I think going beyond just the packages,
00:02:40.800 --> 00:02:46.039
we've also got the people who develop tutorials, guides,
00:02:46.040 --> 00:02:49.279
and all of that sort of surrounding activity,
00:02:49.280 --> 00:02:51.020
which can benefit from a clear understanding
00:02:51.021 --> 00:02:56.020
of how Emacs users use Emacs.
00:02:56.021 --> 00:02:58.519
Separately to that,
00:02:58.520 --> 00:03:01.639
I think as an Emacs user myself,
00:03:01.640 --> 00:03:02.839
that it's rather interesting to see
00:03:02.840 --> 00:03:04.479
how other people are using Emacs
00:03:04.480 --> 00:03:07.079
and what their experience is. So yes, basically,
00:03:07.080 --> 00:03:08.559
you've got utility and interest
00:03:08.560 --> 00:03:10.719
as the two separate driving factors
00:03:10.720 --> 00:03:14.020
as we try to pick questions, which actually can give us
00:03:14.021 --> 00:03:16.520
all of this without taking up too much
00:03:16.521 --> 00:03:18.559
of the respondents time.
NOTE Survey frameworks
00:03:18.560 --> 00:03:24.399
Now, last time in 2020, the Emacs survey that Adrien ran
00:03:24.400 --> 00:03:27.079
used, I think Google Forms, if I recall correctly,
00:03:27.080 --> 00:03:28.799
with an option to send in responses manually.
00:03:28.800 --> 00:03:33.159
This worked, but it's not great,
00:03:33.160 --> 00:03:35.079
particularly given that this is for a survey
00:03:35.080 --> 00:03:37.199
being run in an ardently FOSS community.
00:03:37.200 --> 00:03:38.959
Ideally, we actually want
00:03:38.960 --> 00:03:40.799
to find a survey framework
00:03:40.800 --> 00:03:44.319
that respects the priorities of users, is open source,
00:03:44.320 --> 00:03:46.359
ideally free and open source,
00:03:46.360 --> 00:03:49.999
and is a relatively pleasant experience.
00:03:50.000 --> 00:03:53.079
Unfortunately, looking at available options,
00:03:53.080 --> 00:03:56.879
it seems that one always has to compromise on at least one,
00:03:56.880 --> 00:03:58.020
if not all of those criteria,
00:03:58.021 --> 00:04:01.020
which is quite far from ideal.
NOTE Writing a new survey framework in Julia
00:04:01.021 --> 00:04:04.359
So what's the obvious solution?
00:04:04.360 --> 00:04:06.639
Okay, we should just write a new survey framework.
00:04:06.640 --> 00:04:10.679
Obviously, this is easier said than done.
00:04:10.680 --> 00:04:12.239
But around a year ago,
00:04:12.240 --> 00:04:13.639
I actually started doing exactly this.
00:04:13.640 --> 00:04:17.679
I've used the programming language Julia quite a bit
00:04:17.680 --> 00:04:21.020
on a day to day basis. And there just so happens to be
00:04:21.021 --> 00:04:23.199
a web framework for that called Genie.
00:04:23.200 --> 00:04:24.719
So I thought I'd give it a shot.
00:04:24.720 --> 00:04:26.559
And well, here we are today.
00:04:26.560 --> 00:04:28.479
I ended up putting something together,
00:04:28.480 --> 00:04:34.279
which could take a set of questions written in Julia
00:04:34.280 --> 00:04:35.839
and using a survey library,
00:04:35.840 --> 00:04:38.799
actually pass that into this helpful structure
00:04:38.800 --> 00:04:44.119
and then construct HTML forms based on that,
00:04:44.120 --> 00:04:47.020
and ingest results from the HTML forms,
00:04:47.021 --> 00:04:48.520
and just sort of handle that altogether.
00:04:48.521 --> 00:04:52.439
Now, all of this ends up being fed into an SQLite DB.
00:04:52.440 --> 00:04:55.159
So everything's there, even part responses.
00:04:55.160 --> 00:04:57.599
One of the goals with the actual design of this has been
00:04:57.600 --> 00:05:01.119
to just minimize what's actually done on the client side.
00:05:01.120 --> 00:05:05.559
So that means JavaScript, cookies, the whole lot.
00:05:05.560 --> 00:05:08.759
Basically, as far as this could reasonably be taken,
00:05:08.760 --> 00:05:14.599
we've just got static HTML being shoved to the user,
00:05:14.600 --> 00:05:16.719
or respondent rather. And then we just
00:05:16.720 --> 00:05:18.519
take an HTTP post request back
00:05:18.520 --> 00:05:20.919
and update the results that way.
00:05:20.920 --> 00:05:24.239
Now by doing things like actually paging the survey,
00:05:24.240 --> 00:05:26.559
we can allow for incremental saving of results
00:05:26.560 --> 00:05:30.559
and a few other niceties while essentially preserving
00:05:30.560 --> 00:05:36.319
an experience that doesn't really require any data
00:05:36.320 --> 00:05:37.319
of any particular capabilities, which is sort of
00:05:37.320 --> 00:05:40.199
a nice, clean, minimal experience as far as I'm concerned.
NOTE In practice
00:05:40.200 --> 00:05:45.679
So how does this actually look like in practice?
00:05:45.680 --> 00:05:48.119
Well, one of the nice things about this is
00:05:48.120 --> 00:05:51.479
because the question itself is written in Julia,
00:05:51.480 --> 00:05:54.279
we can get some nice features like custom validators
00:05:54.280 --> 00:05:57.919
and other fancy behavior and directly specify
00:05:57.920 --> 00:06:01.119
how we actually want questions to be registered
00:06:01.120 --> 00:06:04.439
in the database. So here we have, for example,
00:06:04.440 --> 00:06:06.679
two questions we had from this email survey.
00:06:06.680 --> 00:06:09.959
One is a multi-select. Another one is just putting in
00:06:09.960 --> 00:06:14.399
the number of years people have used Emacs for.
00:06:14.400 --> 00:06:16.159
I think this gives a brief overview of the capabilities.
00:06:16.160 --> 00:06:19.599
One of the things I'd like to draw particular attention
00:06:19.600 --> 00:06:20.759
to here is in the multi-select,
00:06:20.760 --> 00:06:22.199
you'll see an array of options,
00:06:22.200 --> 00:06:24.319
the first one of which actually maps for different value
00:06:24.320 --> 00:06:25.879
to be stored for convenience.
00:06:25.880 --> 00:06:29.119
And then the final one is a special one, :other,
00:06:29.120 --> 00:06:30.359
and you can see that's a bit different to the rest
00:06:30.360 --> 00:06:32.599
where it's got that colon function,
00:06:32.600 --> 00:06:33.719
it's a symbol, not a string.
00:06:33.720 --> 00:06:37.639
And this is quite a nice one because the way
00:06:37.640 --> 00:06:39.279
that this framework's been designed,
00:06:39.280 --> 00:06:41.759
when we have an :other value like that,
00:06:41.760 --> 00:06:44.199
instead of it just being a sort of tick box "Other",
00:06:44.200 --> 00:06:47.199
it actually provides the option to write
00:06:47.200 --> 00:06:50.559
your own different response to all of the above.
NOTE Results
00:06:50.560 --> 00:06:55.319
Okay, so at the very end, we've now got
00:06:55.320 --> 00:06:58.519
a completely FOSS survey framework, rather nice.
00:06:58.520 --> 00:07:00.020
So the set of what were these...
00:07:00.021 --> 00:07:01.119
Decent array of input types.
00:07:01.120 --> 00:07:02.639
It would be nice to expand, but at the moment
00:07:02.640 --> 00:07:04.599
I think we could just about describe it as a rich set.
00:07:04.600 --> 00:07:07.159
Zero JavaScript required, but a little bit useful
00:07:07.160 --> 00:07:08.079
for progressive enhancement.
00:07:08.080 --> 00:07:12.759
As demonstrated, we can get some fancy validation going on.
00:07:12.760 --> 00:07:16.679
And then because we've got the results
00:07:16.680 --> 00:07:18.559
tied into this quite nicely,
00:07:18.560 --> 00:07:20.999
we can actually have them available live
00:07:21.000 --> 00:07:22.999
and in quite a number of formats.
00:07:23.000 --> 00:07:25.439
I'm not sure how much you saw in the architecture diagram,
00:07:25.440 --> 00:07:27.079
but we've got all sorts of things here.
00:07:27.080 --> 00:07:29.679
CSV, TSV, plain text, JSON,
00:07:29.680 --> 00:07:32.119
just grab a copy of the SQLite database,
00:07:32.120 --> 00:07:33.319
but only the relevant bits.
00:07:33.320 --> 00:07:35.879
Or something called JLD2,
00:07:35.880 --> 00:07:37.999
which preserves a lot of type information
00:07:38.000 --> 00:07:39.599
and a few other nice things.
NOTE Going forward
00:07:39.600 --> 00:07:43.799
Now, what are we going to do going forward from here?
00:07:43.800 --> 00:07:46.159
Well, there are a few minor issues here.
00:07:46.160 --> 00:07:48.599
For example, there's a memory leak issue which is going on,
00:07:48.600 --> 00:07:51.839
resulting in the service being restarted,
00:07:51.840 --> 00:07:54.519
I think every day or two, while the survey was running.
00:07:54.520 --> 00:07:56.159
I actually have the suspicion
00:07:56.160 --> 00:07:57.639
that that's largely responsible for
00:07:57.640 --> 00:08:01.479
about 1% of respondents, which is about 75 people,
00:08:01.480 --> 00:08:04.399
who described the survey experience as not great.
00:08:04.400 --> 00:08:08.199
Overall though, the feedback has been quite positive.
00:08:08.200 --> 00:08:09.919
There's been some detailed written feedback,
00:08:09.920 --> 00:08:12.799
but just from the quick great/okay/not great options,
00:08:12.800 --> 00:08:14.839
we had about two-thirds of people saying
00:08:14.840 --> 00:08:16.839
that the user experience was great,
00:08:16.840 --> 00:08:19.199
which is really nice to hear the first time being run.
00:08:19.200 --> 00:08:22.839
A few other things would be nice to add, for example,
00:08:22.840 --> 00:08:25.759
in future control flow. By this, I mean
00:08:25.760 --> 00:08:27.879
the option to present different questions
00:08:27.880 --> 00:08:28.999
based on previous answers
00:08:29.000 --> 00:08:31.199
would be quite nice to streamline the experience.
00:08:31.200 --> 00:08:33.519
For example, having a set of questions
00:08:33.520 --> 00:08:37.239
for first-time respondents or people who are involved
00:08:37.240 --> 00:08:42.239
in the packaging side of things
00:08:42.240 --> 00:08:45.079
without actually cluttering the experience
00:08:45.080 --> 00:08:46.039
for everybody else. That'd be quite nice.
00:08:46.040 --> 00:08:48.599
Further to this, all of this,
00:08:48.600 --> 00:08:51.879
I think on top of the standard web interface,
00:08:51.880 --> 00:08:53.599
it'd be quite nice to actually write a server API.
00:08:53.600 --> 00:08:55.520
And the particular reason why I mentioned this
00:08:55.521 --> 00:08:58.020
is because this could potentially allow for
00:08:58.021 --> 00:09:00.359
basically an Emacs survey package.
00:09:00.360 --> 00:09:03.039
I mean, we already use Emacs for so many things,
00:09:03.040 --> 00:09:05.519
might as well fill the survey out from within it as well.
00:09:05.520 --> 00:09:11.159
Okay, so this is how the survey has been conducted.
NOTE Responses
00:09:11.160 --> 00:09:13.679
Now, what are the responses look like?
00:09:13.680 --> 00:09:16.039
Now, at this stage, I was actually hoping
00:09:16.040 --> 00:09:18.919
to get into some somewhat sophisticated analysis
00:09:18.920 --> 00:09:22.599
because there's quite a bit that you can dig out
00:09:22.600 --> 00:09:24.239
of the data responses that we've received.
00:09:24.240 --> 00:09:27.879
However, unfortunately, I've been much more limited on time
00:09:27.880 --> 00:09:30.039
than I'd hoped for, so that's going to have to come later.
00:09:30.040 --> 00:09:33.559
For now, we're just going to take a bit of a peek
00:09:33.560 --> 00:09:35.959
at some of the really basic answers.
00:09:35.960 --> 00:09:38.239
Well, it's not even really analysis.
00:09:38.240 --> 00:09:40.239
Expect to see lots of pie charts, basically.
00:09:40.240 --> 00:09:42.999
But there's still a bit of interest there,
00:09:43.000 --> 00:09:44.359
so we'll go through a bit of that
00:09:44.360 --> 00:09:47.119
and just give a bit of a tease
00:09:47.120 --> 00:09:50.319
as to what might come in the future.
00:09:50.320 --> 00:09:51.919
So to sum up for starters,
00:09:51.920 --> 00:09:55.079
we've had about 6,500 responses.
00:09:55.080 --> 00:09:58.359
It is worth noting that a thousand of those are partials,
00:09:58.360 --> 00:10:02.199
so people who gave up on the survey partway through.
00:10:02.200 --> 00:10:05.399
Given that the 2020 survey had about 7000 responses,
00:10:05.400 --> 00:10:06.999
I'll tell you we're basically on par here.
00:10:07.000 --> 00:10:10.399
This ran over a month and interestingly,
00:10:10.400 --> 00:10:12.239
about half of these respondents
00:10:12.240 --> 00:10:13.799
did not participate in the 2020 survey.
00:10:13.800 --> 00:10:16.199
I think at this point,
00:10:16.200 --> 00:10:17.679
it's not really clear what to make of that.
00:10:17.680 --> 00:10:21.359
There's been a two-year gap between the surveys.
00:10:21.360 --> 00:10:25.159
It's been done, well, it's been done quite differently,
00:10:25.160 --> 00:10:29.639
and yes, there's not enough, really, to say.
00:10:29.640 --> 00:10:31.999
What could be interesting though is actually,
00:10:32.000 --> 00:10:33.839
once this starts running regularly,
00:10:33.840 --> 00:10:36.799
we can see whether there's regular churn
00:10:36.800 --> 00:10:38.520
in the survey respondents,
00:10:38.521 --> 00:10:40.020
or if we have a consistent core
00:10:40.021 --> 00:10:42.020
with people who respond each year,
00:10:42.021 --> 00:10:46.159
and then just people who come by every now and then and go,
00:10:46.160 --> 00:10:47.759
"Oh, why not respond to this year's survey?"
00:10:47.760 --> 00:10:51.479
But we're going to have to wait a bit to actually see
00:10:51.480 --> 00:10:52.759
how people treat the survey.
00:10:52.760 --> 00:10:57.519
Now these responses came from quite a wide range of places
00:10:57.520 --> 00:11:02.519
we've got 115 nations represented here. Collectively,
00:11:02.520 --> 00:11:04.039
these ones have spent about a thousand hours
00:11:04.040 --> 00:11:06.959
giving us information. So I think, if nothing else,
00:11:06.960 --> 00:11:10.479
just from the effort that people have put into
00:11:10.480 --> 00:11:12.879
actually giving us useful data to work with,
00:11:12.880 --> 00:11:13.599
it's worth giving at least a good effort
00:11:13.600 --> 00:11:15.999
to actually trying to extract some value
00:11:16.000 --> 00:11:16.999
out of these responses.
NOTE Geography
00:11:17.000 --> 00:11:20.879
Now, overall we found a lot of responses came from America,
00:11:20.880 --> 00:11:23.199
no surprises there, but as mentioned,
00:11:23.200 --> 00:11:24.020
we've got a good mix around the globe.
00:11:24.021 --> 00:11:29.159
The usual suspects for the rest of the responses,
00:11:29.160 --> 00:11:33.279
a whole bunch in Europe, a whole bunch around Asia,
00:11:33.280 --> 00:11:36.799
a bit in Australasia as well and yes,
00:11:36.800 --> 00:11:38.959
there's nothing particularly surprising here,
00:11:38.960 --> 00:11:41.399
there's a lot of inline expectations.
00:11:41.400 --> 00:11:42.839
What I find a bit more interesting, though,
00:11:42.840 --> 00:11:45.359
is if we actually normalise
00:11:45.360 --> 00:11:48.079
the number of responses from each nation
00:11:48.080 --> 00:11:50.079
by the population of said nations,
00:11:50.080 --> 00:11:54.239
essentially giving a popularity of Emacs
00:11:54.240 --> 00:11:57.359
or at least of Emacs early respondents for each nation,
00:11:57.360 --> 00:12:00.919
we end up finding that Europe, particularly Scandinavia,
00:12:00.920 --> 00:12:02.199
becomes a bit of a hotspot.
00:12:02.200 --> 00:12:04.519
So I'm not sure what's going on
00:12:04.520 --> 00:12:07.319
in Sweden, Finland and Norway,
00:12:07.320 --> 00:12:10.919
but it seems to be particularly popular around there.
00:12:10.920 --> 00:12:14.199
It's also worth noting that we now find
00:12:14.200 --> 00:12:18.319
that the proportion of respondents
00:12:18.320 --> 00:12:21.799
in countries like America, Canada, Australia
00:12:21.800 --> 00:12:24.039
and most of Europe actually becomes
00:12:24.040 --> 00:12:26.399
quite comparable with each other,
00:12:26.400 --> 00:12:30.239
which yes, once again, sort of lines up
00:12:30.240 --> 00:12:32.279
with these responses, expectations from the last slide.
NOTE Gender
00:12:32.280 --> 00:12:36.279
Okay, getting into some of the other
00:12:36.280 --> 00:12:38.599
demographic information.
00:12:38.600 --> 00:12:40.319
The demographic information was new to this survey.
00:12:40.320 --> 00:12:44.479
In the 2020 survey, people were asked what they think
00:12:44.480 --> 00:12:47.199
of being asked about some demographic information
00:12:47.200 --> 00:12:50.199
in a future survey, and the overwhelming response is, "Sure,
00:12:50.200 --> 00:12:52.759
I don't really mind." And so that's what we've done here.
00:12:52.760 --> 00:12:56.279
One of the ones of somewhat interest
00:12:56.280 --> 00:12:59.759
is the age gender breakdown. So we expect Emacs
00:12:59.760 --> 00:13:03.119
to be used predominantly among people in software
00:13:03.120 --> 00:13:05.839
and programming and within the industry,
00:13:05.840 --> 00:13:08.599
I think it's quite widely documented
00:13:08.600 --> 00:13:14.520
to have about a sort of 75-25%, roughly, split
00:13:14.521 --> 00:13:14.759
between male and female.
00:13:14.760 --> 00:13:19.359
Interestingly, in Emacs,
00:13:19.360 --> 00:13:22.879
it's a much more aggressively-biased result.
00:13:22.880 --> 00:13:28.679
So we had about 96% of respondents are male
00:13:28.680 --> 00:13:34.559
with just 4% for the rest. Interestingly, though,
00:13:34.560 --> 00:13:35.359
if we look at the young respondents,
00:13:35.360 --> 00:13:41.719
say for example, under 25, we go from 96% male to 88%.
00:13:41.720 --> 00:13:46.119
So it's fair to say that the young respondents are
00:13:46.120 --> 00:13:49.199
in this respect, a somewhat more diverse group.
00:13:49.200 --> 00:13:52.399
Hopefully, as future surveys go on,
00:13:52.400 --> 00:13:54.399
we'll see this continue not die off
00:13:54.400 --> 00:13:58.719
to the sort of well, at this point,
00:13:58.720 --> 00:14:02.919
it's more like 99% if you look at the older ages.
00:14:02.920 --> 00:14:04.439
But we'll see.
NOTE Occupations
00:14:04.440 --> 00:14:07.919
Occupations was an interesting slide as well.
00:14:07.920 --> 00:14:09.399
Interesting question as well.
00:14:09.400 --> 00:14:11.559
We've got the usual suspects here. I mean,
00:14:11.560 --> 00:14:15.079
it's a text editor, well, Lisp machine
00:14:15.080 --> 00:14:17.639
masquerading as a text editor, mainly used for programming,
00:14:17.640 --> 00:14:20.639
and so we expect lots of software development
00:14:20.640 --> 00:14:23.519
and that sort of thing. But that's only about
00:14:23.520 --> 00:14:25.399
just over half of the responses.
00:14:25.400 --> 00:14:28.679
We've got a huge chunk from academia,
00:14:28.680 --> 00:14:29.999
and then really just an odd bag
00:14:30.000 --> 00:14:30.879
of all sorts of other things,
00:14:30.880 --> 00:14:33.079
including things which you wouldn't really associate
00:14:33.080 --> 00:14:35.359
with programming and software at all.
00:14:35.360 --> 00:14:39.639
Things like creative writing, publishing, legal, yes.
00:14:39.640 --> 00:14:41.719
And then you've got this chunk of Other,
00:14:41.720 --> 00:14:43.239
which is I think here is
00:14:43.240 --> 00:14:46.679
the fourth most popular option here.
00:14:46.680 --> 00:14:49.399
And what we have here is about 500 different responses
00:14:49.400 --> 00:14:51.839
from a huge range of activities.
00:14:51.840 --> 00:14:54.359
It's really quite interesting to read things like
00:14:54.360 --> 00:14:56.919
I think, things like "naval officer",
00:14:56.920 --> 00:15:01.319
and just... All sorts of surprising occupations for Emacs.
00:15:01.320 --> 00:15:04.799
And I think this is a particular area
00:15:04.800 --> 00:15:10.199
because I imagine compared to other code editors,
00:15:10.200 --> 00:15:13.879
sort of your VS Code, remember like
00:15:13.880 --> 00:15:18.959
that Emacs may have a particularly diverse set
00:15:18.960 --> 00:15:23.599
of industry occupations represented in its users.
00:15:23.600 --> 00:15:28.359
Now, if you look at where the response actually came from,
00:15:28.360 --> 00:15:31.039
we've got the usual suspects up top,
00:15:31.040 --> 00:15:33.959
Hacker News and r/emacs.
00:15:33.960 --> 00:15:40.119
But then we actually get a much more graduated breakdown
00:15:40.120 --> 00:15:43.679
than in the 2020 survey.
00:15:43.680 --> 00:15:46.279
We do think familiar results here like IRC, Telegram,
00:15:46.280 --> 00:15:48.639
Emacs China, and Twitter.
00:15:48.640 --> 00:15:50.839
But now you've got a few new entries,
00:15:50.840 --> 00:15:53.519
things like the Fediverse, Discourse, Matrix,
00:15:53.520 --> 00:15:56.119
which didn't pop up previously.
00:15:56.120 --> 00:15:59.079
So I think this is yes, quite a nice sign in terms of
00:15:59.080 --> 00:16:02.520
actually hitting a wide range
00:16:02.521 --> 00:16:05.999
of pockets of Emacs users across different platforms,
00:16:06.000 --> 00:16:10.319
which bodes well for the potential representiveness
00:16:10.320 --> 00:16:11.319
of this survey.
NOTE Free and open source software
00:16:11.320 --> 00:16:15.119
Unsurprisingly, if we're talking about Emacs
00:16:15.120 --> 00:16:17.919
and particularly people who are quite engaged in it,
00:16:17.920 --> 00:16:19.679
which are the respondents to this survey,
00:16:19.680 --> 00:16:25.359
we find that we also get quite a high degree of care
00:16:25.360 --> 00:16:27.479
for free and open source software.
00:16:27.480 --> 00:16:30.519
So if you have a look here,
00:16:30.520 --> 00:16:35.279
only about a quarter of users
00:16:35.280 --> 00:16:39.799
didn't express a strong preference towards FOSS software.
00:16:39.800 --> 00:16:43.759
In fact, we had over a quarter saying that
00:16:43.760 --> 00:16:49.239
they would accept significant or even any compromise
00:16:49.240 --> 00:16:52.199
to use a FOSS user software
00:16:52.200 --> 00:16:55.759
over a proprietary alternative,
00:16:55.760 --> 00:16:59.679
which given the nature of Emacs,
00:16:59.680 --> 00:17:00.639
not terribly surprising,
00:17:00.640 --> 00:17:02.439
but a strong showing nonetheless.
NOTE Emacs versions
00:17:02.440 --> 00:17:05.599
Now, let's start getting to things
00:17:05.600 --> 00:17:07.719
which are actually useful for
00:17:07.720 --> 00:17:11.479
potential Emacs development and packaging.
00:17:11.480 --> 00:17:13.599
If you're thinking about supporting Emacs versions,
00:17:13.600 --> 00:17:16.599
it looks like you can do fantastically well
00:17:16.600 --> 00:17:20.639
in terms of hitting most users if you support Emacs 27+.
00:17:20.640 --> 00:17:23.159
That hits about 96% of respondents.
00:17:23.160 --> 00:17:26.199
Interestingly though, you can actually make an argument
00:17:26.200 --> 00:17:27.119
for being even more aggressive.
00:17:27.120 --> 00:17:30.319
I mean, if you have a look at Emacs 28+,
00:17:30.320 --> 00:17:32.359
that's still over three quarters of respondents.
00:17:32.360 --> 00:17:35.799
We've got, at this point, a quarter
00:17:35.800 --> 00:17:37.279
using the unreleased HEAD version,
00:17:37.280 --> 00:17:40.159
even though it's getting close to release.
00:17:40.160 --> 00:17:43.039
Obviously here, as stated, we're hitting
00:17:43.040 --> 00:17:44.599
a sort of more engaged with the community
00:17:44.600 --> 00:17:47.799
subset of Emacs users, but still,
00:17:47.800 --> 00:17:49.879
I think it's interesting to see that
00:17:49.880 --> 00:17:52.639
with Emacs's increasingly frequent update schedule,
00:17:52.640 --> 00:17:54.999
that users are actually picking up those updates
00:17:55.000 --> 00:17:56.359
quite promptly as they roll out.
NOTE Languages
00:17:56.360 --> 00:18:02.079
Continuing on with how people actually use Emacs: languages.
00:18:02.080 --> 00:18:05.199
We've got the usual suspects here: lots of Python,
00:18:05.200 --> 00:18:08.959
quite a bit of JavaScript and C, lots of shell.
00:18:08.960 --> 00:18:11.879
What I find quite interesting though is
00:18:11.880 --> 00:18:12.799
if we actually bring in
00:18:12.800 --> 00:18:16.719
the 2020 Stack Overflow language usage survey data,
00:18:16.720 --> 00:18:19.239
and that maps quite well
00:18:19.240 --> 00:18:20.079
to the array of language options we provided here.
00:18:20.080 --> 00:18:21.199
They had a general Lisp option,
00:18:21.200 --> 00:18:23.919
which I've folded into Common Lisp
00:18:23.920 --> 00:18:26.919
since they listed Clojure separately.
00:18:26.920 --> 00:18:29.679
I think that seems like a fairly safe bet.
00:18:29.680 --> 00:18:31.919
But other than that, the only languages that we missed
00:18:31.920 --> 00:18:35.839
are Scheme and Elisp.
00:18:35.840 --> 00:18:37.879
What we can do is we can look at
00:18:37.880 --> 00:18:41.199
the relative popularity of different languages
00:18:41.200 --> 00:18:44.519
from our Emacs user survey compared to Stack Overflows.
00:18:44.520 --> 00:18:48.319
What do we find? Well, Clojure and Common Lisp
00:18:48.320 --> 00:18:51.639
far above the rest, I imagine in no small part due to
00:18:51.640 --> 00:18:54.959
the fantastic SLIME and Cider packages.
00:18:54.960 --> 00:18:59.559
Following that, we see Haskell being particularly prominent,
00:18:59.560 --> 00:19:00.639
and then a collection of other languages,
00:19:00.640 --> 00:19:06.199
your Erlang, Elixir, Julia, Perl and the rest.
00:19:06.200 --> 00:19:10.959
And then lastly, if we have a look at the ones
00:19:10.960 --> 00:19:13.439
which have significantly diminished popularity
00:19:13.440 --> 00:19:17.719
compared to Stack Overflow, we end up with, I think,
00:19:17.720 --> 00:19:20.159
what I could probably cast as more enterprising languages.
00:19:20.160 --> 00:19:25.799
Things like C#, Java, Typescript and the like.
NOTE Prose
00:19:25.800 --> 00:19:31.559
So, that's interesting. Now, earlier
00:19:31.560 --> 00:19:33.239
when we were looking at the split of Emacs users,
00:19:33.240 --> 00:19:37.239
we found that we actually had a fair few
00:19:37.240 --> 00:19:42.199
in more creative areas, like writing and publishing.
00:19:42.200 --> 00:19:44.479
So if looking at prose, we'd expect a decent chunk
00:19:44.480 --> 00:19:47.039
to be using Emacs for prose, but it's actually more
00:19:47.040 --> 00:19:48.719
than just a little bit, it's a little slice.
00:19:48.720 --> 00:19:50.599
We've got a whopping about a third of users
00:19:50.600 --> 00:19:54.719
saying they frequently use Emacs for writing prose.
00:19:54.720 --> 00:19:55.999
I'd imagine that the availability
00:19:56.000 --> 00:19:57.799
of things like Org mode and AUCTeX
00:19:57.800 --> 00:20:03.399
probably help like this.
NOTE Packages
00:20:03.400 --> 00:20:05.119
Moving on to other packages, or more packages,
00:20:05.120 --> 00:20:08.879
we've actually got a very similar split here
00:20:08.880 --> 00:20:13.199
to the 2020 survey. Org has seen a bit of a growth
00:20:13.200 --> 00:20:16.039
in popularity. We've got some new arrivals here as well.
00:20:16.040 --> 00:20:18.479
For example, Vertico has popped onto the scene
00:20:18.480 --> 00:20:21.279
and overtaken Ivy here, along with
00:20:21.280 --> 00:20:24.519
a few other new packages like Consult.
00:20:24.520 --> 00:20:27.599
Other than that, quite comparable.
00:20:27.600 --> 00:20:29.999
What's rather interesting, though, I find here is that
00:20:30.000 --> 00:20:33.719
when you have people who listed a small number of packages,
00:20:33.720 --> 00:20:39.439
they actually predominantly listed packages
00:20:39.440 --> 00:20:41.319
other than the most common set.
00:20:41.320 --> 00:20:43.959
So if we have a lot of people who only listed one package,
00:20:43.960 --> 00:20:48.959
basically two-thirds of that,
00:20:48.960 --> 00:20:51.479
or actually three-quarters of those responses
00:20:51.480 --> 00:20:53.879
were saying other packages,
00:20:53.880 --> 00:20:56.279
despite the fact that overall packages
00:20:56.280 --> 00:20:58.599
other than the highlighted selection here
00:20:58.600 --> 00:21:01.399
only constitute a quarter of responses.
00:21:01.400 --> 00:21:04.919
So there might be something a bit more to look at there.
NOTE Documentation
00:21:04.920 --> 00:21:07.799
Now when people are using packages,
00:21:07.800 --> 00:21:11.039
we also asked what types of documentation
00:21:11.040 --> 00:21:14.399
people would like to see more of on package READMEs.
00:21:14.400 --> 00:21:17.159
Basically we've got a big mix here.
00:21:17.160 --> 00:21:20.079
It seems like generally people are interested in
00:21:20.080 --> 00:21:23.839
seeing more in various forms, whether it be tutorials,
00:21:23.840 --> 00:21:29.479
overviews, screenshots, comparisons, or clips and videos.
00:21:29.480 --> 00:21:32.919
So full READMEs with a lot of context
00:21:32.920 --> 00:21:38.439
seem to be quite desirable from this.
NOTE Moving forward
00:21:38.440 --> 00:21:42.359
Now moving forward, what are we going to do?
00:21:42.360 --> 00:21:45.039
So 800 people gave some detailed feedback on the survey.
00:21:45.040 --> 00:21:47.759
That's quite nice. I'm going to be taking a good read
00:21:47.760 --> 00:21:50.799
of all of those responses and use that
00:21:50.800 --> 00:21:55.639
to improve the process and also the set of questions.
00:21:55.640 --> 00:22:00.759
Now all of you can also give some feedback on the questions,
00:22:00.760 --> 00:22:02.679
both that you found most useful in this survey,
00:22:02.680 --> 00:22:04.799
ones that you think might not add much value,
00:22:04.800 --> 00:22:07.039
and/or new questions
00:22:07.040 --> 00:22:08.359
that you think might be a good addition.
00:22:08.360 --> 00:22:11.119
Once I've done a bit more analysis,
00:22:11.120 --> 00:22:13.119
particularly the more sophisticated analysis
00:22:13.120 --> 00:22:17.159
which I'm planning, which will probably come out actually
00:22:17.160 --> 00:22:18.719
maybe in the first quarter of next year,
00:22:18.720 --> 00:22:22.919
we can see which questions there seem to have provided
00:22:22.920 --> 00:22:25.039
the most interesting or surprising results
00:22:25.040 --> 00:22:26.559
and those are probably worth keeping.
00:22:26.560 --> 00:22:31.959
Lastly, once we actually have an API
00:22:31.960 --> 00:22:33.279
and potentially even an Emacs package,
00:22:33.280 --> 00:22:36.159
we could automate a large number of the questions,
00:22:36.160 --> 00:22:38.999
things like Emacs version, set of packages used,
00:22:39.000 --> 00:22:41.039
and that could just streamline the experience
00:22:41.040 --> 00:22:42.279
of actually filling out the survey,
00:22:42.280 --> 00:22:44.199
make it a bit more frictionless.
NOTE Time
00:22:44.200 --> 00:22:47.319
Now talking of the question of questions,
00:22:47.320 --> 00:22:49.319
a quick survey is a good survey.
00:22:49.320 --> 00:22:52.959
If we're asking people to dedicate their time
00:22:52.960 --> 00:22:56.279
to fill out this, it's good to try to get as much value
00:22:56.280 --> 00:22:59.759
without asking them to donate much of their time.
00:22:59.760 --> 00:23:02.399
How has the survey done in this respect?
00:23:02.400 --> 00:23:04.119
I'm actually very happy with how it's done.
00:23:04.120 --> 00:23:06.639
We get a few comments from the feedback saying
00:23:06.640 --> 00:23:07.759
that it was a bit of a long side,
00:23:07.760 --> 00:23:10.759
but the median time was about 12 minutes,
00:23:10.760 --> 00:23:13.759
which doesn't seem too bad, and most commonly
00:23:13.760 --> 00:23:16.399
we saw people completing it in about 8 minutes.
00:23:16.400 --> 00:23:18.879
For a once-per-year survey,
00:23:18.880 --> 00:23:20.519
I think this seems fairly reasonable.
00:23:20.520 --> 00:23:24.279
Getting closer to a 5-10 minute range would be nice,
00:23:24.280 --> 00:23:26.199
but this isn't far off.
NOTE How long the survey is open for
00:23:26.200 --> 00:23:30.879
Lastly, we're also going to be considering
00:23:30.880 --> 00:23:32.719
how long the survey is open for.
00:23:32.720 --> 00:23:36.719
So from the initial opening date,
00:23:36.720 --> 00:23:38.479
what we have here is a plot of
00:23:38.480 --> 00:23:41.919
the page which people ended up on
00:23:41.920 --> 00:23:43.399
and when they started the survey.
00:23:43.400 --> 00:23:46.759
So what we can see is a huge spike in the first few days.
00:23:46.760 --> 00:23:50.239
I've just realised that this plot
00:23:50.240 --> 00:23:53.399
is actually labelled incorrectly.
00:23:53.400 --> 00:23:55.679
Please disregard the minutes to complete the survey.
00:23:55.680 --> 00:23:58.839
This should be days after survey opening
00:23:58.840 --> 00:24:01.519
that a response is actually submitted.
00:24:01.520 --> 00:24:05.399
And what we have here is a big spike
00:24:05.400 --> 00:24:08.679
in popularity in the first week basically,
00:24:08.680 --> 00:24:10.599
and then it trickles down
00:24:10.600 --> 00:24:11.959
to a fairly consistent level after that.
00:24:11.960 --> 00:24:15.839
I'm about to publish a last call for survey responses,
00:24:15.840 --> 00:24:18.279
so I'll see if any final bump happens,
00:24:18.280 --> 00:24:20.039
but this indicates that we can probably just
00:24:20.040 --> 00:24:23.079
have the survey open for a week or two
00:24:23.080 --> 00:24:25.199
and that should be sufficient.
NOTE Plan going forward
00:24:25.200 --> 00:24:30.839
Alright, so what's the general plan going forwards?
00:24:30.840 --> 00:24:35.639
Well, as stated earlier, the idea is to run this annually
00:24:35.640 --> 00:24:38.399
and then consistently improve the questions,
00:24:38.400 --> 00:24:41.039
the experience, and the analysis that's done.
00:24:41.040 --> 00:24:43.559
This year has been the hardest by far
00:24:43.560 --> 00:24:45.839
because a lot had to be set up from scratch.
00:24:45.840 --> 00:24:50.159
The hope is that moving on from here,
00:24:50.160 --> 00:24:51.799
a lot of it can be reused.
00:24:51.800 --> 00:24:54.039
For example, with my comments about
00:24:54.040 --> 00:24:56.439
more sophisticated analysis being down the line,
00:24:56.440 --> 00:24:58.439
once that's all worked out,
00:24:58.440 --> 00:25:00.719
as long as nothing changes too drastically,
00:25:00.720 --> 00:25:03.559
we should be able to reuse a lot of that work
00:25:03.560 --> 00:25:05.759
quite easily in future years.
00:25:05.760 --> 00:25:08.599
Alright, that's it for now.
00:25:08.600 --> 00:25:11.879
Hopefully, you've found this an interesting peek
00:25:11.880 --> 00:25:13.359
into how the survey is operated
00:25:13.360 --> 00:25:15.319
and some of the initial results,
00:25:15.320 --> 00:25:18.919
and hopefully, I'll see you around next year
00:25:18.920 --> 00:25:36.960
for the 2023 survey. Thanks for listening.
