hello everyone and welcome to this short
lightning talk
traverse complex json structures with
live feedback
this is a pre-recorded talk and part of
the e-max conf 2020 schedule
this is what we're going to do I'll make
a quick introduction to the topic at
hand
I'll give you a demonstration of some
tools and then we'll leave you
with the links to set tools
before that just a little bit about me I
am the
ceo and co-founder of a company
based in the swiss mountains called 200
okay
we are a product incubator and
service consultancy but we like to spend
most or at least
as much time as we can building free
software
I'm also an ordained zen monk and abbott
of the lambda zen temple
you can reach me anytime on questions
regarding Emacs
for example at ala at 200ok.ch
[Music]
but back to the topic at hand the
proposition is as following
most work on the computer is based on
either text processing or text
consumption
and very often the text which you need
to process
is in a structured format for example in
json
that might even be if your job is not
programming per se
and reading through such a bigger chunk
of json can be non-trivial however
while just reading and understanding it
will be essential to getting your job
done
so let's quickly check out an example
json file this is from the github api
where which is a request sorry the
response to a request for a specific
issue on the github api so let's
quickly check that one out
okay so here it is open and we can
already see that there is lots of stuff
going on here
it's uh 200 lines it's not going to be
very easy just just to find out what are
the top level things in here what are
the top level attributes of course I can
do this
and maybe it would by hand but that
doesn't scale
I can use cool Emacs facilities like the
height show mode and try to
fold all the things that are top level
but that also doesn't really scale
there must be a better way and of course
there is there is prior art
there is a tool called jq I'm going to
quote the usp
from their website jq is like
sed said for json data you can use it to
slice and filter and map
and transform structured data with the
same ease that
z awk grep and friends let you play with
text
let me give you a quick demonstration of
it by the way
it's written in portable c it has zero
runtime dependency so it's very easy
to get started with it and use it on
pretty much any unix-based
computer sorry no linux based computer
apologies okay so let's explore a
json file with it it's a command line
tool and it has a very
simple command line syntax so you
call the binary and then you give it a
query
and a file and then we'll return
its answer so for example if I want the
top level keys
I will just say jq keys the file and it
will return the keys
simple as that so let's check this out
in a real
shell here I am in eshel
let's run jq keys on the github
issue comment and we can see
that we have actually received a list
back here with the top level
things okay so this issue it looks very
very interesting so let's uh ask it to
give me more information on this
issue then it's hairy again that's a lot
of stuff
I mean lucky for for us we are in Emacs
here so we can use nice shortcuts we can
copy this we go can go in here just
select that
get that out or something like this but
still
this is not really it's not really the
best way
to do that right it gets kind of tedious
at this point the output can be
humongous
the shell is not really the best place
to read through such big output I mean e
shell is
probably one of the better shells for
this because it's just a regular Emacs
buffer but still
it's not really the best tool and I need
to repeat the command
all the time until I finally build the
right query
and all the time I use I lose my
focus I lose what I'm currently looking
at
I'm seeing the new result it would be so
much nicer to have live feedback
and when working with Emacs we're quite
used to that so there should be an
option
and of course there is it's Emacs right
so you can do anything
there is various good tools for
completion in Emacs
I used ivy for this
I'm going to code the usp for ivy
iv is a generic completion mechanism for
Emacs
while it operates similarly to other
completion schemes
such as icomplete mode iv aims to be
more efficient smaller
simpler and smoother to use yet highly
customizable
and that's true one of the cool things
of iv
compared to other completion mechanisms
in Emacs
is that it can be used on dynamic
data so usually completion works on a
static input for example you're in a
buffer a text buffer
and you use I search maybe with idle
mode
and you find your results that's all
nice
however if I
want to search on dynamic data that
doesn't work
so whenever I type in my query
for jq I actually need to call the jq
binary
and it will give a different result set
back so it's a really
dynamic mechanism that we need here it's
much more like a search engine
and ivy luckily has something built in
and it's called
console so I used council and jq
and combined them and build a new
package
with which we can use Emacs and jq
to have live feedback
it's very easy to use so you just call
console.jq
on a buffer containing json for example
the one we have here
let's call console.jq on it and we
already
get a default query the dot query which
just gives us
the same file but now we can change it
and for example
find all the keys in here and then we
see I had this issue this was the one
that we were interested in
so let's find more information on the
issue what
keys does it have actually have uh it
has assignees that that interests me
so let's check out the assignees in here
there's two of them but I'm only
interested in the first one
I'm making stuff up as I go here of
course
and whenever I hit enter I get a new
buffer
which just shows me this
particular result for the particular
query that I entered
so let me do that again we are in here
we are looking at a
json file this can be very very big
doesn't also need to be a file just
needs to be a buffer
you call console.jq on it and you can do
any kind of
query on it for example let's see if
there is a url here
yes there's a url let's see if there's a
repository here repository
no there isn't what was it called issue
keys repository url it was called okay
so let's see issue pause
url and then we see so
apparently this issue command is for a
repository called
organize I wonder what that might be
okay so that was a very short
introduction to console jq
you can see the timer here I only have
one minute left to go so I'm going to
leave
with a very very short introduction to
the console gq
code it's not even 60 lines
of e-lisp so building something like
this is very very easy
I would encourage you to go and read
through the code in your own time
if you're interested in building
something like this if you're interested
in just using
jq or you're done
these are the links to all the tools
console.jq of course
is readily available on melpah also
developed
under the agpl license on github
and disorganized thing by the way it's
orig mode for mobile and desktop
browsers also a great
free software tool maybe that interests
you
thank you for listening have great time
10 seconds left I am going to stop this
now
enjoy EmacsConf have a great day