From 8baff7beaf0e052f8369cb3dcf477b454744a567 Mon Sep 17 00:00:00 2001 From: Sacha Chua Date: Thu, 18 Mar 2021 23:52:45 -0400 Subject: Update transcript for #25 --- ...-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt | 757 --------------------- 1 file changed, 757 deletions(-) delete mode 100644 2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt (limited to '2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt') diff --git a/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt b/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt deleted file mode 100644 index e7eada5b..00000000 --- a/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.vtt +++ /dev/null @@ -1,757 +0,0 @@ -WEBVTT - -00:00:00.799 --> 00:00:04.000 -hello everyone and welcome to this short - -00:00:04.000 --> 00:00:05.520 -lightning talk - -00:00:05.520 --> 00:00:08.160 -traverse complex json structures with - -00:00:08.160 --> 00:00:09.519 -live feedback - -00:00:09.519 --> 00:00:13.040 -this is a pre-recorded talk and part of - -00:00:13.040 --> 00:00:18.000 -the e-max conf 2020 schedule - -00:00:18.000 --> 00:00:19.920 -this is what we're going to do I'll make - -00:00:19.920 --> 00:00:21.840 -a quick introduction to the topic at - -00:00:21.840 --> 00:00:22.320 -hand - -00:00:22.320 --> 00:00:23.920 -I'll give you a demonstration of some - -00:00:23.920 --> 00:00:25.760 -tools and then we'll leave you - -00:00:25.760 --> 00:00:29.199 -with the links to set tools - -00:00:29.199 --> 00:00:31.920 -before that just a little bit about me I - -00:00:31.920 --> 00:00:32.399 -am the - -00:00:32.399 --> 00:00:36.079 -ceo and co-founder of a company - -00:00:36.079 --> 00:00:39.520 -based in the swiss mountains called 200 - -00:00:39.520 --> 00:00:40.399 -okay - -00:00:40.399 --> 00:00:43.600 -we are a product incubator and - -00:00:43.600 --> 00:00:46.160 -service consultancy but we like to spend - -00:00:46.160 --> 00:00:47.200 -most or at least - -00:00:47.200 --> 00:00:50.719 -as much time as we can building free - -00:00:50.719 --> 00:00:52.719 -software - -00:00:52.719 --> 00:00:55.520 -I'm also an ordained zen monk and abbott - -00:00:55.520 --> 00:00:56.879 -of the lambda zen temple - -00:00:56.879 --> 00:01:00.160 -you can reach me anytime on questions - -00:01:00.160 --> 00:01:01.359 -regarding Emacs - -00:01:01.359 --> 00:01:05.860 -for example at ala at 200ok.ch - -00:01:05.860 --> 00:01:07.200 -[Music] - -00:01:07.200 --> 00:01:09.600 -but back to the topic at hand the - -00:01:09.600 --> 00:01:11.760 -proposition is as following - -00:01:11.760 --> 00:01:13.680 -most work on the computer is based on - -00:01:13.680 --> 00:01:15.520 -either text processing or text - -00:01:15.520 --> 00:01:16.479 -consumption - -00:01:16.479 --> 00:01:19.920 -and very often the text which you need - -00:01:19.920 --> 00:01:20.880 -to process - -00:01:20.880 --> 00:01:23.520 -is in a structured format for example in - -00:01:23.520 --> 00:01:24.560 -json - -00:01:24.560 --> 00:01:26.640 -that might even be if your job is not - -00:01:26.640 --> 00:01:28.560 -programming per se - -00:01:28.560 --> 00:01:30.400 -and reading through such a bigger chunk - -00:01:30.400 --> 00:01:33.119 -of json can be non-trivial however - -00:01:33.119 --> 00:01:36.479 -while just reading and understanding it - -00:01:36.479 --> 00:01:39.119 -will be essential to getting your job - -00:01:39.119 --> 00:01:40.320 -done - -00:01:40.320 --> 00:01:43.680 -so let's quickly check out an example - -00:01:43.680 --> 00:01:47.200 -json file this is from the github api - -00:01:47.200 --> 00:01:50.560 -where which is a request sorry the - -00:01:50.560 --> 00:01:53.119 -response to a request for a specific - -00:01:53.119 --> 00:01:55.280 -issue on the github api so let's - -00:01:55.280 --> 00:01:58.799 -quickly check that one out - -00:01:58.799 --> 00:02:01.280 -okay so here it is open and we can - -00:02:01.280 --> 00:02:03.280 -already see that there is lots of stuff - -00:02:03.280 --> 00:02:05.439 -going on here - -00:02:05.439 --> 00:02:08.319 -it's uh 200 lines it's not going to be - -00:02:08.319 --> 00:02:10.319 -very easy just just to find out what are - -00:02:10.319 --> 00:02:12.160 -the top level things in here what are - -00:02:12.160 --> 00:02:13.920 -the top level attributes of course I can - -00:02:13.920 --> 00:02:14.560 -do this - -00:02:14.560 --> 00:02:16.480 -and maybe it would by hand but that - -00:02:16.480 --> 00:02:17.840 -doesn't scale - -00:02:17.840 --> 00:02:20.560 -I can use cool Emacs facilities like the - -00:02:20.560 --> 00:02:22.560 -height show mode and try to - -00:02:22.560 --> 00:02:24.720 -fold all the things that are top level - -00:02:24.720 --> 00:02:27.200 -but that also doesn't really scale - -00:02:27.200 --> 00:02:29.760 -there must be a better way and of course - -00:02:29.760 --> 00:02:32.000 -there is there is prior art - -00:02:32.000 --> 00:02:34.959 -there is a tool called jq I'm going to - -00:02:34.959 --> 00:02:36.080 -quote the usp - -00:02:36.080 --> 00:02:39.120 -from their website jq is like - -00:02:39.120 --> 00:02:42.959 -sed said for json data you can use it to - -00:02:42.959 --> 00:02:44.720 -slice and filter and map - -00:02:44.720 --> 00:02:46.720 -and transform structured data with the - -00:02:46.720 --> 00:02:47.840 -same ease that - -00:02:47.840 --> 00:02:51.519 -z awk grep and friends let you play with - -00:02:51.519 --> 00:02:54.000 -text - -00:02:54.000 --> 00:02:56.160 -let me give you a quick demonstration of - -00:02:56.160 --> 00:02:57.519 -it by the way - -00:02:57.519 --> 00:02:59.680 -it's written in portable c it has zero - -00:02:59.680 --> 00:03:01.920 -runtime dependency so it's very easy - -00:03:01.920 --> 00:03:04.959 -to get started with it and use it on - -00:03:04.959 --> 00:03:09.040 -pretty much any unix-based - -00:03:09.040 --> 00:03:12.800 -computer sorry no linux based computer - -00:03:12.800 --> 00:03:16.159 -apologies okay so let's explore a - -00:03:16.159 --> 00:03:19.599 -json file with it it's a command line - -00:03:19.599 --> 00:03:21.120 -tool and it has a very - -00:03:21.120 --> 00:03:24.640 -simple command line syntax so you - -00:03:24.640 --> 00:03:26.799 -call the binary and then you give it a - -00:03:26.799 --> 00:03:27.840 -query - -00:03:27.840 --> 00:03:31.680 -and a file and then we'll return - -00:03:31.680 --> 00:03:33.840 -its answer so for example if I want the - -00:03:33.840 --> 00:03:35.440 -top level keys - -00:03:35.440 --> 00:03:38.319 -I will just say jq keys the file and it - -00:03:38.319 --> 00:03:39.840 -will return the keys - -00:03:39.840 --> 00:03:42.319 -simple as that so let's check this out - -00:03:42.319 --> 00:03:43.519 -in a real - -00:03:43.519 --> 00:03:46.879 -shell here I am in eshel - -00:03:46.879 --> 00:03:50.080 -let's run jq keys on the github - -00:03:50.080 --> 00:03:53.200 -issue comment and we can see - -00:03:53.200 --> 00:03:56.959 -that we have actually received a list - -00:03:56.959 --> 00:04:00.000 -back here with the top level - -00:04:00.000 --> 00:04:02.000 -things okay so this issue it looks very - -00:04:02.000 --> 00:04:04.319 -very interesting so let's uh ask it to - -00:04:04.319 --> 00:04:06.239 -give me more information on this - -00:04:06.239 --> 00:04:10.080 -issue then it's hairy again that's a lot - -00:04:10.080 --> 00:04:11.360 -of stuff - -00:04:11.360 --> 00:04:14.319 -I mean lucky for for us we are in Emacs - -00:04:14.319 --> 00:04:17.759 -here so we can use nice shortcuts we can - -00:04:17.759 --> 00:04:21.040 -copy this we go can go in here just - -00:04:21.040 --> 00:04:22.000 -select that - -00:04:22.000 --> 00:04:24.400 -get that out or something like this but - -00:04:24.400 --> 00:04:25.600 -still - -00:04:25.600 --> 00:04:28.320 -this is not really it's not really the - -00:04:28.320 --> 00:04:29.600 -best way - -00:04:29.600 --> 00:04:34.080 -to do that right it gets kind of tedious - -00:04:34.080 --> 00:04:36.639 -at this point the output can be - -00:04:36.639 --> 00:04:37.680 -humongous - -00:04:37.680 --> 00:04:39.360 -the shell is not really the best place - -00:04:39.360 --> 00:04:42.479 -to read through such big output I mean e - -00:04:42.479 --> 00:04:43.600 -shell is - -00:04:43.600 --> 00:04:45.440 -probably one of the better shells for - -00:04:45.440 --> 00:04:47.440 -this because it's just a regular Emacs - -00:04:47.440 --> 00:04:48.560 -buffer but still - -00:04:48.560 --> 00:04:51.199 -it's not really the best tool and I need - -00:04:51.199 --> 00:04:52.720 -to repeat the command - -00:04:52.720 --> 00:04:55.280 -all the time until I finally build the - -00:04:55.280 --> 00:04:56.000 -right query - -00:04:56.000 --> 00:04:59.120 -and all the time I use I lose my - -00:04:59.120 --> 00:05:02.080 -focus I lose what I'm currently looking - -00:05:02.080 --> 00:05:02.800 -at - -00:05:02.800 --> 00:05:06.080 -I'm seeing the new result it would be so - -00:05:06.080 --> 00:05:08.160 -much nicer to have live feedback - -00:05:08.160 --> 00:05:10.080 -and when working with Emacs we're quite - -00:05:10.080 --> 00:05:11.600 -used to that so there should be an - -00:05:11.600 --> 00:05:12.320 -option - -00:05:12.320 --> 00:05:15.120 -and of course there is it's Emacs right - -00:05:15.120 --> 00:05:17.759 -so you can do anything - -00:05:17.759 --> 00:05:20.880 -there is various good tools for - -00:05:20.880 --> 00:05:22.960 -completion in Emacs - -00:05:22.960 --> 00:05:26.000 -I used ivy for this - -00:05:26.000 --> 00:05:29.039 -I'm going to code the usp for ivy - -00:05:29.039 --> 00:05:31.840 -iv is a generic completion mechanism for - -00:05:31.840 --> 00:05:32.639 -Emacs - -00:05:32.639 --> 00:05:34.639 -while it operates similarly to other - -00:05:34.639 --> 00:05:35.919 -completion schemes - -00:05:35.919 --> 00:05:38.960 -such as icomplete mode iv aims to be - -00:05:38.960 --> 00:05:40.400 -more efficient smaller - -00:05:40.400 --> 00:05:43.120 -simpler and smoother to use yet highly - -00:05:43.120 --> 00:05:45.199 -customizable - -00:05:45.199 --> 00:05:47.840 -and that's true one of the cool things - -00:05:47.840 --> 00:05:49.440 -of iv - -00:05:49.440 --> 00:05:52.479 -compared to other completion mechanisms - -00:05:52.479 --> 00:05:54.320 -in Emacs - -00:05:54.320 --> 00:05:58.160 -is that it can be used on dynamic - -00:05:58.160 --> 00:06:01.600 -data so usually completion works on a - -00:06:01.600 --> 00:06:03.520 -static input for example you're in a - -00:06:03.520 --> 00:06:05.360 -buffer a text buffer - -00:06:05.360 --> 00:06:08.800 -and you use I search maybe with idle - -00:06:08.800 --> 00:06:09.600 -mode - -00:06:09.600 --> 00:06:12.400 -and you find your results that's all - -00:06:12.400 --> 00:06:13.360 -nice - -00:06:13.360 --> 00:06:16.960 -however if I - -00:06:16.960 --> 00:06:19.840 -want to search on dynamic data that - -00:06:19.840 --> 00:06:20.720 -doesn't work - -00:06:20.720 --> 00:06:23.759 -so whenever I type in my query - -00:06:23.759 --> 00:06:27.039 -for jq I actually need to call the jq - -00:06:27.039 --> 00:06:28.000 -binary - -00:06:28.000 --> 00:06:30.160 -and it will give a different result set - -00:06:30.160 --> 00:06:32.840 -back so it's a really - -00:06:32.840 --> 00:06:36.319 -dynamic mechanism that we need here it's - -00:06:36.319 --> 00:06:38.240 -much more like a search engine - -00:06:38.240 --> 00:06:41.440 -and ivy luckily has something built in - -00:06:41.440 --> 00:06:42.560 -and it's called - -00:06:42.560 --> 00:06:46.000 -console so I used council and jq - -00:06:46.000 --> 00:06:48.160 -and combined them and build a new - -00:06:48.160 --> 00:06:49.199 -package - -00:06:49.199 --> 00:06:52.960 -with which we can use Emacs and jq - -00:06:52.960 --> 00:06:56.000 -to have live feedback - -00:06:56.000 --> 00:06:58.800 -it's very easy to use so you just call - -00:06:58.800 --> 00:06:59.840 -console.jq - -00:06:59.840 --> 00:07:02.800 -on a buffer containing json for example - -00:07:02.800 --> 00:07:04.319 -the one we have here - -00:07:04.319 --> 00:07:07.280 -let's call console.jq on it and we - -00:07:07.280 --> 00:07:08.319 -already - -00:07:08.319 --> 00:07:11.280 -get a default query the dot query which - -00:07:11.280 --> 00:07:13.039 -just gives us - -00:07:13.039 --> 00:07:16.080 -the same file but now we can change it - -00:07:16.080 --> 00:07:17.280 -and for example - -00:07:17.280 --> 00:07:19.039 -find all the keys in here and then we - -00:07:19.039 --> 00:07:21.039 -see I had this issue this was the one - -00:07:21.039 --> 00:07:22.800 -that we were interested in - -00:07:22.800 --> 00:07:24.639 -so let's find more information on the - -00:07:24.639 --> 00:07:26.479 -issue what - -00:07:26.479 --> 00:07:28.800 -keys does it have actually have uh it - -00:07:28.800 --> 00:07:31.680 -has assignees that that interests me - -00:07:31.680 --> 00:07:34.800 -so let's check out the assignees in here - -00:07:34.800 --> 00:07:37.039 -there's two of them but I'm only - -00:07:37.039 --> 00:07:39.759 -interested in the first one - -00:07:39.759 --> 00:07:42.000 -I'm making stuff up as I go here of - -00:07:42.000 --> 00:07:43.599 -course - -00:07:43.599 --> 00:07:46.000 -and whenever I hit enter I get a new - -00:07:46.000 --> 00:07:47.039 -buffer - -00:07:47.039 --> 00:07:50.160 -which just shows me this - -00:07:50.160 --> 00:07:53.520 -particular result for the particular - -00:07:53.520 --> 00:07:55.599 -query that I entered - -00:07:55.599 --> 00:07:59.199 -so let me do that again we are in here - -00:07:59.199 --> 00:08:03.199 -we are looking at a - -00:08:03.199 --> 00:08:05.840 -json file this can be very very big - -00:08:05.840 --> 00:08:07.520 -doesn't also need to be a file just - -00:08:07.520 --> 00:08:09.520 -needs to be a buffer - -00:08:09.520 --> 00:08:12.160 -you call console.jq on it and you can do - -00:08:12.160 --> 00:08:13.599 -any kind of - -00:08:13.599 --> 00:08:16.479 -query on it for example let's see if - -00:08:16.479 --> 00:08:18.080 -there is a url here - -00:08:18.080 --> 00:08:20.319 -yes there's a url let's see if there's a - -00:08:20.319 --> 00:08:23.759 -repository here repository - -00:08:23.759 --> 00:08:26.879 -no there isn't what was it called issue - -00:08:26.879 --> 00:08:33.440 -keys repository url it was called okay - -00:08:33.440 --> 00:08:36.640 -so let's see issue pause - -00:08:36.640 --> 00:08:40.240 -url and then we see so - -00:08:40.240 --> 00:08:42.959 -apparently this issue command is for a - -00:08:42.959 --> 00:08:44.159 -repository called - -00:08:44.159 --> 00:08:47.839 -organize I wonder what that might be - -00:08:47.839 --> 00:08:50.320 -okay so that was a very short - -00:08:50.320 --> 00:08:52.640 -introduction to console jq - -00:08:52.640 --> 00:08:54.800 -you can see the timer here I only have - -00:08:54.800 --> 00:08:56.800 -one minute left to go so I'm going to - -00:08:56.800 --> 00:08:57.440 -leave - -00:08:57.440 --> 00:08:59.760 -with a very very short introduction to - -00:08:59.760 --> 00:09:01.519 -the console gq - -00:09:01.519 --> 00:09:05.040 -code it's not even 60 lines - -00:09:05.040 --> 00:09:07.519 -of e-lisp so building something like - -00:09:07.519 --> 00:09:09.600 -this is very very easy - -00:09:09.600 --> 00:09:12.800 -I would encourage you to go and read - -00:09:12.800 --> 00:09:14.560 -through the code in your own time - -00:09:14.560 --> 00:09:16.160 -if you're interested in building - -00:09:16.160 --> 00:09:18.320 -something like this if you're interested - -00:09:18.320 --> 00:09:19.600 -in just using - -00:09:19.600 --> 00:09:22.720 -jq or you're done - -00:09:22.720 --> 00:09:24.320 -these are the links to all the tools - -00:09:24.320 --> 00:09:25.839 -console.jq of course - -00:09:25.839 --> 00:09:28.800 -is readily available on melpah also - -00:09:28.800 --> 00:09:29.519 -developed - -00:09:29.519 --> 00:09:32.959 -under the agpl license on github - -00:09:32.959 --> 00:09:36.080 -and disorganized thing by the way it's - -00:09:36.080 --> 00:09:37.839 -orig mode for mobile and desktop - -00:09:37.839 --> 00:09:39.839 -browsers also a great - -00:09:39.839 --> 00:09:42.399 -free software tool maybe that interests - -00:09:42.399 --> 00:09:43.120 -you - -00:09:43.120 --> 00:09:46.240 -thank you for listening have great time - -00:09:46.240 --> 00:09:48.800 -10 seconds left I am going to stop this - -00:09:48.800 --> 00:09:49.360 -now - -00:09:49.360 --> 00:09:53.920 -enjoy EmacsConf have a great day -- cgit v1.2.3