diff options
Diffstat (limited to '2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv')
| -rw-r--r-- | 2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv | 756 |
1 files changed, 0 insertions, 756 deletions
diff --git a/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv b/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv deleted file mode 100644 index 1c88545f..00000000 --- a/2020/subtitles/emacsconf-2020--25-traverse-complex-json-structures-with-live-feedback-counsel-jq--zen-monk-alain-m-lafon-autogen.sbv +++ /dev/null @@ -1,756 +0,0 @@ -0:00:00.799,0:00:05.520 -hello everyone and welcome to this short - -0:00:04.000,0:00:08.160 -lightning talk - -0:00:05.520,0:00:09.519 -traverse complex json structures with - -0:00:08.160,0:00:13.040 -live feedback - -0:00:09.519,0:00:18.000 -this is a pre-recorded talk and part of - -0:00:13.040,0:00:19.920 -the e-max conf 2020 schedule - -0:00:18.000,0:00:21.840 -this is what we're going to do I'll make - -0:00:19.920,0:00:22.320 -a quick introduction to the topic at - -0:00:21.840,0:00:23.920 -hand - -0:00:22.320,0:00:25.760 -I'll give you a demonstration of some - -0:00:23.920,0:00:29.199 -tools and then we'll leave you - -0:00:25.760,0:00:31.920 -with the links to set tools - -0:00:29.199,0:00:32.399 -before that just a little bit about me I - -0:00:31.920,0:00:36.079 -am the - -0:00:32.399,0:00:39.520 -ceo and co-founder of a company - -0:00:36.079,0:00:40.399 -based in the swiss mountains called 200 - -0:00:39.520,0:00:43.600 -okay - -0:00:40.399,0:00:46.160 -we are a product incubator and - -0:00:43.600,0:00:47.200 -service consultancy but we like to spend - -0:00:46.160,0:00:50.719 -most or at least - -0:00:47.200,0:00:52.719 -as much time as we can building free - -0:00:50.719,0:00:55.520 -software - -0:00:52.719,0:00:56.879 -I'm also an ordained zen monk and abbott - -0:00:55.520,0:01:00.160 -of the lambda zen temple - -0:00:56.879,0:01:01.359 -you can reach me anytime on questions - -0:01:00.160,0:01:05.860 -regarding Emacs - -0:01:01.359,0:01:07.200 -for example at ala at 200ok.ch - -0:01:05.860,0:01:09.600 -[Music] - -0:01:07.200,0:01:11.760 -but back to the topic at hand the - -0:01:09.600,0:01:13.680 -proposition is as following - -0:01:11.760,0:01:15.520 -most work on the computer is based on - -0:01:13.680,0:01:16.479 -either text processing or text - -0:01:15.520,0:01:19.920 -consumption - -0:01:16.479,0:01:20.880 -and very often the text which you need - -0:01:19.920,0:01:23.520 -to process - -0:01:20.880,0:01:24.560 -is in a structured format for example in - -0:01:23.520,0:01:26.640 -json - -0:01:24.560,0:01:28.560 -that might even be if your job is not - -0:01:26.640,0:01:30.400 -programming per se - -0:01:28.560,0:01:33.119 -and reading through such a bigger chunk - -0:01:30.400,0:01:36.479 -of json can be non-trivial however - -0:01:33.119,0:01:39.119 -while just reading and understanding it - -0:01:36.479,0:01:40.320 -will be essential to getting your job - -0:01:39.119,0:01:43.680 -done - -0:01:40.320,0:01:47.200 -so let's quickly check out an example - -0:01:43.680,0:01:50.560 -json file this is from the github api - -0:01:47.200,0:01:53.119 -where which is a request sorry the - -0:01:50.560,0:01:55.280 -response to a request for a specific - -0:01:53.119,0:01:58.560 -issue on the github api so let's - -0:01:55.280,0:01:58.560 -quickly check that one out - -0:01:58.799,0:02:03.280 -okay so here it is open and we can - -0:02:01.280,0:02:05.439 -already see that there is lots of stuff - -0:02:03.280,0:02:08.319 -going on here - -0:02:05.439,0:02:10.319 -it's uh 200 lines it's not going to be - -0:02:08.319,0:02:12.160 -very easy just just to find out what are - -0:02:10.319,0:02:13.920 -the top level things in here what are - -0:02:12.160,0:02:14.560 -the top level attributes of course I can - -0:02:13.920,0:02:16.480 -do this - -0:02:14.560,0:02:17.840 -and maybe it would by hand but that - -0:02:16.480,0:02:20.560 -doesn't scale - -0:02:17.840,0:02:22.560 -I can use cool Emacs facilities like the - -0:02:20.560,0:02:24.720 -height show mode and try to - -0:02:22.560,0:02:27.200 -fold all the things that are top level - -0:02:24.720,0:02:29.760 -but that also doesn't really scale - -0:02:27.200,0:02:32.000 -there must be a better way and of course - -0:02:29.760,0:02:34.959 -there is there is prior art - -0:02:32.000,0:02:36.080 -there is a tool called jq I'm going to - -0:02:34.959,0:02:39.120 -quote the usp - -0:02:36.080,0:02:42.959 -from their website jq is like - -0:02:39.120,0:02:44.720 -sed said for json data you can use it to - -0:02:42.959,0:02:46.720 -slice and filter and map - -0:02:44.720,0:02:47.840 -and transform structured data with the - -0:02:46.720,0:02:51.519 -same ease that - -0:02:47.840,0:02:53.920 -z awk grep and friends let you play with - -0:02:51.519,0:02:53.920 -text - -0:02:54.000,0:02:57.519 -let me give you a quick demonstration of - -0:02:56.160,0:02:59.680 -it by the way - -0:02:57.519,0:03:01.920 -it's written in portable c it has zero - -0:02:59.680,0:03:04.959 -runtime dependency so it's very easy - -0:03:01.920,0:03:09.040 -to get started with it and use it on - -0:03:04.959,0:03:12.800 -pretty much any unix-based - -0:03:09.040,0:03:16.159 -computer sorry no linux based computer - -0:03:12.800,0:03:19.599 -apologies okay so let's explore a - -0:03:16.159,0:03:21.120 -json file with it it's a command line - -0:03:19.599,0:03:24.640 -tool and it has a very - -0:03:21.120,0:03:26.799 -simple command line syntax so you - -0:03:24.640,0:03:27.840 -call the binary and then you give it a - -0:03:26.799,0:03:31.680 -query - -0:03:27.840,0:03:33.840 -and a file and then we'll return - -0:03:31.680,0:03:35.440 -its answer so for example if I want the - -0:03:33.840,0:03:38.319 -top level keys - -0:03:35.440,0:03:39.840 -I will just say jq keys the file and it - -0:03:38.319,0:03:42.319 -will return the keys - -0:03:39.840,0:03:43.519 -simple as that so let's check this out - -0:03:42.319,0:03:46.879 -in a real - -0:03:43.519,0:03:50.080 -shell here I am in eshel - -0:03:46.879,0:03:53.200 -let's run jq keys on the github - -0:03:50.080,0:03:56.959 -issue comment and we can see - -0:03:53.200,0:04:00.000 -that we have actually received a list - -0:03:56.959,0:04:02.000 -back here with the top level - -0:04:00.000,0:04:04.319 -things okay so this issue it looks very - -0:04:02.000,0:04:06.239 -very interesting so let's uh ask it to - -0:04:04.319,0:04:10.080 -give me more information on this - -0:04:06.239,0:04:11.360 -issue then it's hairy again that's a lot - -0:04:10.080,0:04:14.319 -of stuff - -0:04:11.360,0:04:17.759 -I mean lucky for for us we are in Emacs - -0:04:14.319,0:04:21.040 -here so we can use nice shortcuts we can - -0:04:17.759,0:04:22.000 -copy this we go can go in here just - -0:04:21.040,0:04:24.400 -select that - -0:04:22.000,0:04:25.600 -get that out or something like this but - -0:04:24.400,0:04:28.320 -still - -0:04:25.600,0:04:29.600 -this is not really it's not really the - -0:04:28.320,0:04:34.080 -best way - -0:04:29.600,0:04:36.639 -to do that right it gets kind of tedious - -0:04:34.080,0:04:37.680 -at this point the output can be - -0:04:36.639,0:04:39.360 -humongous - -0:04:37.680,0:04:42.479 -the shell is not really the best place - -0:04:39.360,0:04:43.600 -to read through such big output I mean e - -0:04:42.479,0:04:45.440 -shell is - -0:04:43.600,0:04:47.440 -probably one of the better shells for - -0:04:45.440,0:04:48.560 -this because it's just a regular Emacs - -0:04:47.440,0:04:51.199 -buffer but still - -0:04:48.560,0:04:52.720 -it's not really the best tool and I need - -0:04:51.199,0:04:55.280 -to repeat the command - -0:04:52.720,0:04:56.000 -all the time until I finally build the - -0:04:55.280,0:04:59.120 -right query - -0:04:56.000,0:05:02.080 -and all the time I use I lose my - -0:04:59.120,0:05:02.800 -focus I lose what I'm currently looking - -0:05:02.080,0:05:06.080 -at - -0:05:02.800,0:05:08.160 -I'm seeing the new result it would be so - -0:05:06.080,0:05:10.080 -much nicer to have live feedback - -0:05:08.160,0:05:11.600 -and when working with Emacs we're quite - -0:05:10.080,0:05:12.320 -used to that so there should be an - -0:05:11.600,0:05:15.120 -option - -0:05:12.320,0:05:17.759 -and of course there is it's Emacs right - -0:05:15.120,0:05:20.880 -so you can do anything - -0:05:17.759,0:05:22.960 -there is various good tools for - -0:05:20.880,0:05:26.000 -completion in Emacs - -0:05:22.960,0:05:29.039 -I used ivy for this - -0:05:26.000,0:05:31.840 -I'm going to code the usp for ivy - -0:05:29.039,0:05:32.639 -iv is a generic completion mechanism for - -0:05:31.840,0:05:34.639 -Emacs - -0:05:32.639,0:05:35.919 -while it operates similarly to other - -0:05:34.639,0:05:38.960 -completion schemes - -0:05:35.919,0:05:40.400 -such as icomplete mode iv aims to be - -0:05:38.960,0:05:43.120 -more efficient smaller - -0:05:40.400,0:05:45.199 -simpler and smoother to use yet highly - -0:05:43.120,0:05:47.840 -customizable - -0:05:45.199,0:05:49.440 -and that's true one of the cool things - -0:05:47.840,0:05:52.479 -of iv - -0:05:49.440,0:05:54.320 -compared to other completion mechanisms - -0:05:52.479,0:05:58.160 -in Emacs - -0:05:54.320,0:06:01.600 -is that it can be used on dynamic - -0:05:58.160,0:06:03.520 -data so usually completion works on a - -0:06:01.600,0:06:05.360 -static input for example you're in a - -0:06:03.520,0:06:08.800 -buffer a text buffer - -0:06:05.360,0:06:09.600 -and you use I search maybe with idle - -0:06:08.800,0:06:12.400 -mode - -0:06:09.600,0:06:13.360 -and you find your results that's all - -0:06:12.400,0:06:16.960 -nice - -0:06:13.360,0:06:19.840 -however if I - -0:06:16.960,0:06:20.720 -want to search on dynamic data that - -0:06:19.840,0:06:23.759 -doesn't work - -0:06:20.720,0:06:27.039 -so whenever I type in my query - -0:06:23.759,0:06:28.000 -for jq I actually need to call the jq - -0:06:27.039,0:06:30.160 -binary - -0:06:28.000,0:06:32.840 -and it will give a different result set - -0:06:30.160,0:06:36.319 -back so it's a really - -0:06:32.840,0:06:38.240 -dynamic mechanism that we need here it's - -0:06:36.319,0:06:41.440 -much more like a search engine - -0:06:38.240,0:06:42.560 -and ivy luckily has something built in - -0:06:41.440,0:06:46.000 -and it's called - -0:06:42.560,0:06:48.160 -console so I used council and jq - -0:06:46.000,0:06:49.199 -and combined them and build a new - -0:06:48.160,0:06:52.960 -package - -0:06:49.199,0:06:56.000 -with which we can use Emacs and jq - -0:06:52.960,0:06:58.800 -to have live feedback - -0:06:56.000,0:06:59.840 -it's very easy to use so you just call - -0:06:58.800,0:07:02.800 -console.jq - -0:06:59.840,0:07:04.319 -on a buffer containing json for example - -0:07:02.800,0:07:07.280 -the one we have here - -0:07:04.319,0:07:08.319 -let's call console.jq on it and we - -0:07:07.280,0:07:11.280 -already - -0:07:08.319,0:07:13.039 -get a default query the dot query which - -0:07:11.280,0:07:16.080 -just gives us - -0:07:13.039,0:07:17.280 -the same file but now we can change it - -0:07:16.080,0:07:19.039 -and for example - -0:07:17.280,0:07:21.039 -find all the keys in here and then we - -0:07:19.039,0:07:22.800 -see I had this issue this was the one - -0:07:21.039,0:07:24.639 -that we were interested in - -0:07:22.800,0:07:26.479 -so let's find more information on the - -0:07:24.639,0:07:28.800 -issue what - -0:07:26.479,0:07:31.680 -keys does it have actually have uh it - -0:07:28.800,0:07:34.800 -has assignees that that interests me - -0:07:31.680,0:07:37.039 -so let's check out the assignees in here - -0:07:34.800,0:07:39.759 -there's two of them but I'm only - -0:07:37.039,0:07:42.000 -interested in the first one - -0:07:39.759,0:07:43.599 -I'm making stuff up as I go here of - -0:07:42.000,0:07:46.000 -course - -0:07:43.599,0:07:47.039 -and whenever I hit enter I get a new - -0:07:46.000,0:07:50.160 -buffer - -0:07:47.039,0:07:53.520 -which just shows me this - -0:07:50.160,0:07:55.599 -particular result for the particular - -0:07:53.520,0:07:59.199 -query that I entered - -0:07:55.599,0:08:03.199 -so let me do that again we are in here - -0:07:59.199,0:08:05.840 -we are looking at a - -0:08:03.199,0:08:07.520 -json file this can be very very big - -0:08:05.840,0:08:09.520 -doesn't also need to be a file just - -0:08:07.520,0:08:12.160 -needs to be a buffer - -0:08:09.520,0:08:13.599 -you call console.jq on it and you can do - -0:08:12.160,0:08:16.479 -any kind of - -0:08:13.599,0:08:18.080 -query on it for example let's see if - -0:08:16.479,0:08:20.319 -there is a url here - -0:08:18.080,0:08:23.759 -yes there's a url let's see if there's a - -0:08:20.319,0:08:26.879 -repository here repository - -0:08:23.759,0:08:33.360 -no there isn't what was it called issue - -0:08:26.879,0:08:33.360 -keys repository url it was called okay - -0:08:33.440,0:08:40.240 -so let's see issue pause - -0:08:36.640,0:08:42.959 -url and then we see so - -0:08:40.240,0:08:44.159 -apparently this issue command is for a - -0:08:42.959,0:08:47.839 -repository called - -0:08:44.159,0:08:50.320 -organize I wonder what that might be - -0:08:47.839,0:08:52.640 -okay so that was a very short - -0:08:50.320,0:08:54.800 -introduction to console jq - -0:08:52.640,0:08:56.800 -you can see the timer here I only have - -0:08:54.800,0:08:57.440 -one minute left to go so I'm going to - -0:08:56.800,0:08:59.760 -leave - -0:08:57.440,0:09:01.519 -with a very very short introduction to - -0:08:59.760,0:09:05.040 -the console gq - -0:09:01.519,0:09:07.519 -code it's not even 60 lines - -0:09:05.040,0:09:09.600 -of e-lisp so building something like - -0:09:07.519,0:09:12.800 -this is very very easy - -0:09:09.600,0:09:14.560 -I would encourage you to go and read - -0:09:12.800,0:09:16.160 -through the code in your own time - -0:09:14.560,0:09:18.320 -if you're interested in building - -0:09:16.160,0:09:19.600 -something like this if you're interested - -0:09:18.320,0:09:22.720 -in just using - -0:09:19.600,0:09:24.320 -jq or you're done - -0:09:22.720,0:09:25.839 -these are the links to all the tools - -0:09:24.320,0:09:28.800 -console.jq of course - -0:09:25.839,0:09:29.519 -is readily available on melpah also - -0:09:28.800,0:09:32.959 -developed - -0:09:29.519,0:09:36.080 -under the agpl license on github - -0:09:32.959,0:09:37.839 -and disorganized thing by the way it's - -0:09:36.080,0:09:39.839 -orig mode for mobile and desktop - -0:09:37.839,0:09:42.399 -browsers also a great - -0:09:39.839,0:09:43.120 -free software tool maybe that interests - -0:09:42.399,0:09:46.240 -you - -0:09:43.120,0:09:48.800 -thank you for listening have great time - -0:09:46.240,0:09:49.360 -10 seconds left I am going to stop this - -0:09:48.800,0:09:53.920 -now - -0:09:49.360,0:09:53.920 -enjoy EmacsConf have a great day - |