diff options
Diffstat (limited to '2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv')
-rw-r--r-- | 2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv | 756 |
1 files changed, 756 insertions, 0 deletions
diff --git a/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv b/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv new file mode 100644 index 00000000..95bdfb5e --- /dev/null +++ b/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv @@ -0,0 +1,756 @@ +0:00:00.799,0:00:05.520 +hello everyone and welcome to this short + +0:00:04.000,0:00:08.160 +lightning talk + +0:00:05.520,0:00:09.519 +traverse complex json structures with + +0:00:08.160,0:00:13.040 +live feedback + +0:00:09.519,0:00:18.000 +this is a pre-recorded talk and part of + +0:00:13.040,0:00:19.920 +the e-max conf 2020 schedule + +0:00:18.000,0:00:21.840 +this is what we're going to do i'll make + +0:00:19.920,0:00:22.320 +a quick introduction to the topic at + +0:00:21.840,0:00:23.920 +hand + +0:00:22.320,0:00:25.760 +i'll give you a demonstration of some + +0:00:23.920,0:00:29.199 +tools and then we'll leave you + +0:00:25.760,0:00:31.920 +with the links to set tools + +0:00:29.199,0:00:32.399 +before that just a little bit about me i + +0:00:31.920,0:00:36.079 +am the + +0:00:32.399,0:00:39.520 +ceo and co-founder of a company + +0:00:36.079,0:00:40.399 +based in the swiss mountains called 200 + +0:00:39.520,0:00:43.600 +okay + +0:00:40.399,0:00:46.160 +we are a product incubator and + +0:00:43.600,0:00:47.200 +service consultancy but we like to spend + +0:00:46.160,0:00:50.719 +most or at least + +0:00:47.200,0:00:52.719 +as much time as we can building free + +0:00:50.719,0:00:55.520 +software + +0:00:52.719,0:00:56.879 +i'm also an ordained zen monk and abbott + +0:00:55.520,0:01:00.160 +of the lambda zen temple + +0:00:56.879,0:01:01.359 +you can reach me anytime on questions + +0:01:00.160,0:01:05.860 +regarding emacs + +0:01:01.359,0:01:07.200 +for example at ala at 200ok.ch + +0:01:05.860,0:01:09.600 +[Music] + +0:01:07.200,0:01:11.760 +but back to the topic at hand the + +0:01:09.600,0:01:13.680 +proposition is as following + +0:01:11.760,0:01:15.520 +most work on the computer is based on + +0:01:13.680,0:01:16.479 +either text processing or text + +0:01:15.520,0:01:19.920 +consumption + +0:01:16.479,0:01:20.880 +and very often the text which you need + +0:01:19.920,0:01:23.520 +to process + +0:01:20.880,0:01:24.560 +is in a structured format for example in + +0:01:23.520,0:01:26.640 +json + +0:01:24.560,0:01:28.560 +that might even be if your job is not + +0:01:26.640,0:01:30.400 +programming per se + +0:01:28.560,0:01:33.119 +and reading through such a bigger chunk + +0:01:30.400,0:01:36.479 +of json can be non-trivial however + +0:01:33.119,0:01:39.119 +while just reading and understanding it + +0:01:36.479,0:01:40.320 +will be essential to getting your job + +0:01:39.119,0:01:43.680 +done + +0:01:40.320,0:01:47.200 +so let's quickly check out an example + +0:01:43.680,0:01:50.560 +json file this is from the github api + +0:01:47.200,0:01:53.119 +where which is a request sorry the + +0:01:50.560,0:01:55.280 +response to a request for a specific + +0:01:53.119,0:01:58.560 +issue on the github api so let's + +0:01:55.280,0:01:58.560 +quickly check that one out + +0:01:58.799,0:02:03.280 +okay so here it is open and we can + +0:02:01.280,0:02:05.439 +already see that there is lots of stuff + +0:02:03.280,0:02:08.319 +going on here + +0:02:05.439,0:02:10.319 +it's uh 200 lines it's not going to be + +0:02:08.319,0:02:12.160 +very easy just just to find out what are + +0:02:10.319,0:02:13.920 +the top level things in here what are + +0:02:12.160,0:02:14.560 +the top level attributes of course i can + +0:02:13.920,0:02:16.480 +do this + +0:02:14.560,0:02:17.840 +and maybe it would by hand but that + +0:02:16.480,0:02:20.560 +doesn't scale + +0:02:17.840,0:02:22.560 +i can use cool emacs facilities like the + +0:02:20.560,0:02:24.720 +height show mode and try to + +0:02:22.560,0:02:27.200 +fold all the things that are top level + +0:02:24.720,0:02:29.760 +but that also doesn't really scale + +0:02:27.200,0:02:32.000 +there must be a better way and of course + +0:02:29.760,0:02:34.959 +there is there is prior art + +0:02:32.000,0:02:36.080 +there is a tool called jq i'm going to + +0:02:34.959,0:02:39.120 +quote the usp + +0:02:36.080,0:02:42.959 +from their website jq is like + +0:02:39.120,0:02:44.720 +sed said for json data you can use it to + +0:02:42.959,0:02:46.720 +slice and filter and map + +0:02:44.720,0:02:47.840 +and transform structured data with the + +0:02:46.720,0:02:51.519 +same ease that + +0:02:47.840,0:02:53.920 +z awk grep and friends let you play with + +0:02:51.519,0:02:53.920 +text + +0:02:54.000,0:02:57.519 +let me give you a quick demonstration of + +0:02:56.160,0:02:59.680 +it by the way + +0:02:57.519,0:03:01.920 +it's written in portable c it has zero + +0:02:59.680,0:03:04.959 +runtime dependency so it's very easy + +0:03:01.920,0:03:09.040 +to get started with it and use it on + +0:03:04.959,0:03:12.800 +pretty much any unix-based + +0:03:09.040,0:03:16.159 +computer sorry no linux based computer + +0:03:12.800,0:03:19.599 +apologies okay so let's explore a + +0:03:16.159,0:03:21.120 +json file with it it's a command line + +0:03:19.599,0:03:24.640 +tool and it has a very + +0:03:21.120,0:03:26.799 +simple command line syntax so you + +0:03:24.640,0:03:27.840 +call the binary and then you give it a + +0:03:26.799,0:03:31.680 +query + +0:03:27.840,0:03:33.840 +and a file and then we'll return + +0:03:31.680,0:03:35.440 +its answer so for example if i want the + +0:03:33.840,0:03:38.319 +top level keys + +0:03:35.440,0:03:39.840 +i will just say jq keys the file and it + +0:03:38.319,0:03:42.319 +will return the keys + +0:03:39.840,0:03:43.519 +simple as that so let's check this out + +0:03:42.319,0:03:46.879 +in a real + +0:03:43.519,0:03:50.080 +shell here i am in eshel + +0:03:46.879,0:03:53.200 +let's run jq keys on the github + +0:03:50.080,0:03:56.959 +issue comment and we can see + +0:03:53.200,0:04:00.000 +that we have actually received a list + +0:03:56.959,0:04:02.000 +back here with the top level + +0:04:00.000,0:04:04.319 +things okay so this issue it looks very + +0:04:02.000,0:04:06.239 +very interesting so let's uh ask it to + +0:04:04.319,0:04:10.080 +give me more information on this + +0:04:06.239,0:04:11.360 +issue then it's hairy again that's a lot + +0:04:10.080,0:04:14.319 +of stuff + +0:04:11.360,0:04:17.759 +i mean lucky for for us we are in emacs + +0:04:14.319,0:04:21.040 +here so we can use nice shortcuts we can + +0:04:17.759,0:04:22.000 +copy this we go can go in here just + +0:04:21.040,0:04:24.400 +select that + +0:04:22.000,0:04:25.600 +get that out or something like this but + +0:04:24.400,0:04:28.320 +still + +0:04:25.600,0:04:29.600 +this is not really it's not really the + +0:04:28.320,0:04:34.080 +best way + +0:04:29.600,0:04:36.639 +to do that right it gets kind of tedious + +0:04:34.080,0:04:37.680 +at this point the output can be + +0:04:36.639,0:04:39.360 +humongous + +0:04:37.680,0:04:42.479 +the shell is not really the best place + +0:04:39.360,0:04:43.600 +to read through such big output i mean e + +0:04:42.479,0:04:45.440 +shell is + +0:04:43.600,0:04:47.440 +probably one of the better shells for + +0:04:45.440,0:04:48.560 +this because it's just a regular emacs + +0:04:47.440,0:04:51.199 +buffer but still + +0:04:48.560,0:04:52.720 +it's not really the best tool and i need + +0:04:51.199,0:04:55.280 +to repeat the command + +0:04:52.720,0:04:56.000 +all the time until i finally build the + +0:04:55.280,0:04:59.120 +right query + +0:04:56.000,0:05:02.080 +and all the time i use i lose my + +0:04:59.120,0:05:02.800 +focus i lose what i'm currently looking + +0:05:02.080,0:05:06.080 +at + +0:05:02.800,0:05:08.160 +i'm seeing the new result it would be so + +0:05:06.080,0:05:10.080 +much nicer to have live feedback + +0:05:08.160,0:05:11.600 +and when working with emacs we're quite + +0:05:10.080,0:05:12.320 +used to that so there should be an + +0:05:11.600,0:05:15.120 +option + +0:05:12.320,0:05:17.759 +and of course there is it's emacs right + +0:05:15.120,0:05:20.880 +so you can do anything + +0:05:17.759,0:05:22.960 +there is various good tools for + +0:05:20.880,0:05:26.000 +completion in emacs + +0:05:22.960,0:05:29.039 +i used ivy for this + +0:05:26.000,0:05:31.840 +i'm going to code the usp for ivy + +0:05:29.039,0:05:32.639 +iv is a generic completion mechanism for + +0:05:31.840,0:05:34.639 +emacs + +0:05:32.639,0:05:35.919 +while it operates similarly to other + +0:05:34.639,0:05:38.960 +completion schemes + +0:05:35.919,0:05:40.400 +such as icomplete mode iv aims to be + +0:05:38.960,0:05:43.120 +more efficient smaller + +0:05:40.400,0:05:45.199 +simpler and smoother to use yet highly + +0:05:43.120,0:05:47.840 +customizable + +0:05:45.199,0:05:49.440 +and that's true one of the cool things + +0:05:47.840,0:05:52.479 +of iv + +0:05:49.440,0:05:54.320 +compared to other completion mechanisms + +0:05:52.479,0:05:58.160 +in emacs + +0:05:54.320,0:06:01.600 +is that it can be used on dynamic + +0:05:58.160,0:06:03.520 +data so usually completion works on a + +0:06:01.600,0:06:05.360 +static input for example you're in a + +0:06:03.520,0:06:08.800 +buffer a text buffer + +0:06:05.360,0:06:09.600 +and you use i search maybe with idle + +0:06:08.800,0:06:12.400 +mode + +0:06:09.600,0:06:13.360 +and you find your results that's all + +0:06:12.400,0:06:16.960 +nice + +0:06:13.360,0:06:19.840 +however if i + +0:06:16.960,0:06:20.720 +want to search on dynamic data that + +0:06:19.840,0:06:23.759 +doesn't work + +0:06:20.720,0:06:27.039 +so whenever i type in my query + +0:06:23.759,0:06:28.000 +for jq i actually need to call the jq + +0:06:27.039,0:06:30.160 +binary + +0:06:28.000,0:06:32.840 +and it will give a different result set + +0:06:30.160,0:06:36.319 +back so it's a really + +0:06:32.840,0:06:38.240 +dynamic mechanism that we need here it's + +0:06:36.319,0:06:41.440 +much more like a search engine + +0:06:38.240,0:06:42.560 +and ivy luckily has something built in + +0:06:41.440,0:06:46.000 +and it's called + +0:06:42.560,0:06:48.160 +console so i used council and jq + +0:06:46.000,0:06:49.199 +and combined them and build a new + +0:06:48.160,0:06:52.960 +package + +0:06:49.199,0:06:56.000 +with which we can use emacs and jq + +0:06:52.960,0:06:58.800 +to have live feedback + +0:06:56.000,0:06:59.840 +it's very easy to use so you just call + +0:06:58.800,0:07:02.800 +console.jq + +0:06:59.840,0:07:04.319 +on a buffer containing json for example + +0:07:02.800,0:07:07.280 +the one we have here + +0:07:04.319,0:07:08.319 +let's call console.jq on it and we + +0:07:07.280,0:07:11.280 +already + +0:07:08.319,0:07:13.039 +get a default query the dot query which + +0:07:11.280,0:07:16.080 +just gives us + +0:07:13.039,0:07:17.280 +the same file but now we can change it + +0:07:16.080,0:07:19.039 +and for example + +0:07:17.280,0:07:21.039 +find all the keys in here and then we + +0:07:19.039,0:07:22.800 +see i had this issue this was the one + +0:07:21.039,0:07:24.639 +that we were interested in + +0:07:22.800,0:07:26.479 +so let's find more information on the + +0:07:24.639,0:07:28.800 +issue what + +0:07:26.479,0:07:31.680 +keys does it have actually have uh it + +0:07:28.800,0:07:34.800 +has assignees that that interests me + +0:07:31.680,0:07:37.039 +so let's check out the assignees in here + +0:07:34.800,0:07:39.759 +there's two of them but i'm only + +0:07:37.039,0:07:42.000 +interested in the first one + +0:07:39.759,0:07:43.599 +i'm making stuff up as i go here of + +0:07:42.000,0:07:46.000 +course + +0:07:43.599,0:07:47.039 +and whenever i hit enter i get a new + +0:07:46.000,0:07:50.160 +buffer + +0:07:47.039,0:07:53.520 +which just shows me this + +0:07:50.160,0:07:55.599 +particular result for the particular + +0:07:53.520,0:07:59.199 +query that i entered + +0:07:55.599,0:08:03.199 +so let me do that again we are in here + +0:07:59.199,0:08:05.840 +we are looking at a + +0:08:03.199,0:08:07.520 +json file this can be very very big + +0:08:05.840,0:08:09.520 +doesn't also need to be a file just + +0:08:07.520,0:08:12.160 +needs to be a buffer + +0:08:09.520,0:08:13.599 +you call console.jq on it and you can do + +0:08:12.160,0:08:16.479 +any kind of + +0:08:13.599,0:08:18.080 +query on it for example let's see if + +0:08:16.479,0:08:20.319 +there is a url here + +0:08:18.080,0:08:23.759 +yes there's a url let's see if there's a + +0:08:20.319,0:08:26.879 +repository here repository + +0:08:23.759,0:08:33.360 +no there isn't what was it called issue + +0:08:26.879,0:08:33.360 +keys repository url it was called okay + +0:08:33.440,0:08:40.240 +so let's see issue pause + +0:08:36.640,0:08:42.959 +url and then we see so + +0:08:40.240,0:08:44.159 +apparently this issue command is for a + +0:08:42.959,0:08:47.839 +repository called + +0:08:44.159,0:08:50.320 +organize i wonder what that might be + +0:08:47.839,0:08:52.640 +okay so that was a very short + +0:08:50.320,0:08:54.800 +introduction to console jq + +0:08:52.640,0:08:56.800 +you can see the timer here i only have + +0:08:54.800,0:08:57.440 +one minute left to go so i'm going to + +0:08:56.800,0:08:59.760 +leave + +0:08:57.440,0:09:01.519 +with a very very short introduction to + +0:08:59.760,0:09:05.040 +the console gq + +0:09:01.519,0:09:07.519 +code it's not even 60 lines + +0:09:05.040,0:09:09.600 +of e-lisp so building something like + +0:09:07.519,0:09:12.800 +this is very very easy + +0:09:09.600,0:09:14.560 +i would encourage you to go and read + +0:09:12.800,0:09:16.160 +through the code in your own time + +0:09:14.560,0:09:18.320 +if you're interested in building + +0:09:16.160,0:09:19.600 +something like this if you're interested + +0:09:18.320,0:09:22.720 +in just using + +0:09:19.600,0:09:24.320 +jq or you're done + +0:09:22.720,0:09:25.839 +these are the links to all the tools + +0:09:24.320,0:09:28.800 +console.jq of course + +0:09:25.839,0:09:29.519 +is readily available on melpah also + +0:09:28.800,0:09:32.959 +developed + +0:09:29.519,0:09:36.080 +under the agpl license on github + +0:09:32.959,0:09:37.839 +and disorganized thing by the way it's + +0:09:36.080,0:09:39.839 +orig mode for mobile and desktop + +0:09:37.839,0:09:42.399 +browsers also a great + +0:09:39.839,0:09:43.120 +free software tool maybe that interests + +0:09:42.399,0:09:46.240 +you + +0:09:43.120,0:09:48.800 +thank you for listening have great time + +0:09:46.240,0:09:49.360 +10 seconds left i am going to stop this + +0:09:48.800,0:09:53.920 +now + +0:09:49.360,0:09:53.920 +enjoy emacs conf have a great day + |