diff options
Diffstat (limited to '2024/captions/emacsconf-2024-transducers--transducers-finally-ergonomic-data-processing-for-emacs--colin-woodbury--answers.vtt')
| -rw-r--r-- | 2024/captions/emacsconf-2024-transducers--transducers-finally-ergonomic-data-processing-for-emacs--colin-woodbury--answers.vtt | 1032 |
1 files changed, 1032 insertions, 0 deletions
diff --git a/2024/captions/emacsconf-2024-transducers--transducers-finally-ergonomic-data-processing-for-emacs--colin-woodbury--answers.vtt b/2024/captions/emacsconf-2024-transducers--transducers-finally-ergonomic-data-processing-for-emacs--colin-woodbury--answers.vtt new file mode 100644 index 00000000..375cad2a --- /dev/null +++ b/2024/captions/emacsconf-2024-transducers--transducers-finally-ergonomic-data-processing-for-emacs--colin-woodbury--answers.vtt @@ -0,0 +1,1032 @@ +WEBVTT chapters by sachac + +00:00:00.000 --> 00:00:07.559 +Hopefully the internet goes well. It's a nice Monday + +00:00:07.560 --> 00:00:31.999 +morning here in Tokyo. + +00:00:32.000 --> 00:00:37.879 +Are we connected all right? + +00:00:37.880 --> 00:00:40.879 +Okay, I seem to be struggling still with my audio. 1 2nd + +00:00:40.880 --> 00:00:44.519 +calling. Yeah, you were muted for a moment there. Okay, + +00:00:44.520 --> 00:00:49.959 +there we are. Okay. All right. Sorry about that. I got a mute + +00:00:49.960 --> 00:00:55.119 +out my, my back office chatter. That's kind of distracting + +00:00:55.120 --> 00:00:58.079 +me a little bit. All right. Sorry. I may have lost the plot a + +00:00:58.080 --> 00:01:04.919 +little bit. I think I did. However, find the 1st question. + +00:01:04.920 --> 00:01:09.919 +I got pretty distracted by conversation backstage. Yeah, + +NOTE Q: When I tried comparing transducers.el to cl-lib and dash (benchmark-compiled), I got the following results + +00:01:09.920 --> 00:01:15.879 +no problem. So the first question here, someone's asking, + +00:01:15.880 --> 00:01:22.279 +when they first tried comparing transducers.el, the cl-lib + +00:01:22.280 --> 00:01:27.959 +and Dash bookmark compiled, and they give some detailed + +00:01:27.960 --> 00:01:32.479 +results we're sharing on the stream. Um, they expected + +00:01:32.480 --> 00:01:36.679 +transducers to be slower than CL loop, but faster than CL lib + +00:01:36.680 --> 00:01:41.119 +or dash. However, this isn't the case, any idea why. And so + +00:01:41.120 --> 00:01:43.639 +I'll, I'll come back into their data to show there's they're + +00:01:43.640 --> 00:01:48.279 +showing, um, you know, there's not a lot of detail on the, on + +00:01:48.280 --> 00:01:52.199 +the, on the use case here. We could certainly click through + +00:01:52.200 --> 00:02:02.559 +it, do it. + +00:02:02.560 --> 00:02:06.999 +Oh, I should've waited to zoom until I find my spot here. + +00:02:07.000 --> 00:02:13.639 +There we are. + +00:02:13.640 --> 00:02:18.599 +All right, so there's our example. + +00:02:18.600 --> 00:02:23.759 +Looks like we are doing a simple map and a sum. + +00:02:23.760 --> 00:02:29.239 +Mm-hmm. Yeah, that's right. Yeah, question about + +00:02:29.240 --> 00:02:36.279 +performance. So a case like this, a simple, I just want to rip + +00:02:36.280 --> 00:02:40.279 +through a collection of numbers and sum them all. That's a + +00:02:40.280 --> 00:02:44.679 +case where basically loop is always going to win because + +00:02:44.680 --> 00:02:51.319 +loop is optimized. This is true in both Emacs Lisp and in + +00:02:51.320 --> 00:02:56.039 +Common Lisp. For a case like this where you're not really + +00:02:56.040 --> 00:03:02.399 +doing two nested of chained calls, like you don't have many + +00:03:02.400 --> 00:03:05.839 +sort of what I was compositional steps. If you're just + +00:03:05.840 --> 00:03:09.999 +ripping through a collection of numbers, loop is always + +00:03:10.000 --> 00:03:15.559 +going to win. Transducers kind of shines when you have to do + +00:03:15.560 --> 00:03:19.639 +things that loop can't in terms of expressing yourself. So + +00:03:19.640 --> 00:03:22.559 +there are lots of different transducers that you can chain + +00:03:22.560 --> 00:03:27.079 +together. And in that case, you're kind of prioritizing + +00:03:27.080 --> 00:03:33.039 +developer time and developer happiness because you're + +00:03:33.040 --> 00:03:36.399 +able to yourself more clearly, whereas sometimes those + +00:03:36.400 --> 00:03:40.679 +kind of algorithms can get very hairy if you're just using + +00:03:40.680 --> 00:03:45.399 +loop. Now that sounds like I'm moving the goalposts, and + +00:03:45.400 --> 00:03:48.639 +there's really no excuse for these things not being as + +00:03:48.640 --> 00:03:54.559 +performant as possible. In this specific case, my guess is + +00:03:54.560 --> 00:03:57.759 +that the transducers is slower because it has to do a whole + +00:03:57.760 --> 00:04:03.239 +bunch of like inner function calls in order to actually do + +00:04:03.240 --> 00:04:09.239 +the adding and the collecting. So there's a lot of stuff that + +00:04:09.240 --> 00:04:12.119 +just the raw loop doesn't have to do, which transducers + +00:04:12.120 --> 00:04:20.439 +does. And so in this case, that's why it would be slower. + +00:04:20.440 --> 00:04:29.079 +All right, makes sense. + +00:04:29.080 --> 00:04:36.239 +Um... I cannot comment against Dash. And also a reminder + +00:04:36.240 --> 00:04:40.159 +that transducers both in CL and in Emacs Lisp here doesn't + +00:04:40.160 --> 00:04:44.919 +attempt to do any, you know, fun, you know, inner rewriting + +00:04:44.920 --> 00:04:48.239 +or, you know, what's called an Haskell fusion. Like if you + +00:04:48.240 --> 00:04:51.359 +have two different map steps, like in a row, it's not gonna + +00:04:51.360 --> 00:04:55.159 +see that and somehow fuse them internally. It's a fairly, in + +00:04:55.160 --> 00:04:59.679 +that sense, the implementation is just as is. + +00:04:59.680 --> 00:05:04.159 +to make it you know as raw fast as possible. The idea being + +00:05:04.160 --> 00:05:12.839 +that ergonomics is more important up front. Yeah, that's + +00:05:12.840 --> 00:05:17.519 +kind of a whole fascinating sub-panel, right? My theme this + +00:05:17.520 --> 00:05:19.799 +conference has been, oh, all these different things we + +00:05:19.800 --> 00:05:24.039 +should try to get sub-panels going for and use that. Maybe + +00:05:24.040 --> 00:05:29.039 +fill in the dev track or even have a third track or whatever. + +00:05:29.040 --> 00:05:31.519 +I'm not that concerned about the logistics of squeezing + +00:05:31.520 --> 00:05:38.519 +into the schedule so much. But anyway, interesting, I mean, + +00:05:38.520 --> 00:05:40.839 +to say. + +NOTE Q: Do you know of any theoretical texts on transducers? + +00:05:40.840 --> 00:05:47.799 +Did we already speak to theoretical texts? No, right? No, + +00:05:47.800 --> 00:05:53.399 +let's continue. Okay, so another question from the group. + +00:05:53.400 --> 00:05:58.879 +Do you know of any theoretical texts on transducers? My + +00:05:58.880 --> 00:06:01.959 +readme, particularly of the Common Lisp implementation, + +00:06:01.960 --> 00:06:06.159 +is the theoretical text on transducers. Rich Hickey has + +00:06:06.160 --> 00:06:10.439 +some YouTube videos which also come close. I mean, he + +00:06:10.440 --> 00:06:14.799 +invented the things. But in terms of having a full + +00:06:14.800 --> 00:06:21.559 +explanation of everything, it's my readme and it's also + +00:06:21.560 --> 00:06:23.319 +the... + +00:06:23.320 --> 00:06:28.559 +The info manual of Guile Scheme, their documentation on + +00:06:28.560 --> 00:06:34.199 +Surfy 171 is what I used to learn transducers and to + +00:06:34.200 --> 00:06:38.399 +re-implement them in other LISPs. So if you just want like a + +00:06:38.400 --> 00:06:41.639 +document explaining them, MyReadMe is actually the + +00:06:41.640 --> 00:06:46.959 +clearest that I've found. Awesome. Okay, next question. + +00:06:46.960 --> 00:06:50.119 +And I'm sorry, you gave a name, you referred to somebody's + +00:06:50.120 --> 00:06:55.439 +videos. Rich Hickey, the inventor of Clojure. Rich Hickey, + +00:06:55.440 --> 00:07:00.399 +thank you. Hope I got the spelling right, and maybe somebody + +00:07:00.400 --> 00:07:04.719 +can catch that and fix it. If not, I'll reach on. Thank you. + +NOTE Q: Did you think about [compiler features, macros] viz your cl, fennel, elisp, porting of your transducers? + +00:07:04.720 --> 00:07:08.239 +Reach on to the next question. Waters (Lazy Series in + +00:07:08.240 --> 00:07:12.799 +Lisp, late 70s) said this should have been done as an + +00:07:12.800 --> 00:07:16.799 +additional compiler feature in compilers, but if not, must + +00:07:16.800 --> 00:07:21.439 +be a macro package. Do you think about that vis your CL, + +00:07:21.440 --> 00:07:27.519 +Fennel, Elisp, porting of transducers? I think that + +00:07:27.520 --> 00:07:28.519 +there's definitely + +00:07:28.520 --> 00:07:36.519 +some Galaxy Brain Lisp author out there is probably smart + +00:07:36.520 --> 00:07:40.599 +enough to turn a bunch of this stuff into macros. I believe + +00:07:40.600 --> 00:07:47.119 +that's how the common Lisp library series works. It sees + +00:07:47.120 --> 00:07:52.079 +that you were calling map or whatever, and it actually knows + +00:07:52.080 --> 00:07:56.639 +that that's a special macro key. in order to be fast. I did not + +00:07:56.640 --> 00:08:01.839 +do that. The implementation as I have it is very simple and + +00:08:01.840 --> 00:08:05.759 +simplicity shouldn't be underestimated. + +00:08:05.760 --> 00:08:13.559 +I love it. What a nice succinct answer. Even I can manage to + +00:08:13.560 --> 00:08:16.578 +type that out as I scroll us to the next question. + +NOTE Q: Does t-buffer-read provide a lazy stream that's linewise, or charwise, or do something else entirely? + +00:08:16.579 --> 00:08:24.079 +So, does t-buffer-read provide a lazy stream + +00:08:24.080 --> 00:08:28.359 +that's line-wise or character-wise or do something else + +00:08:28.360 --> 00:08:29.018 +entirely? + +00:08:29.019 --> 00:08:31.587 +Okay, there are two functions. I showed + +00:08:31.588 --> 00:08:35.073 +t-buffer-read. There's also one called t-file-read, + +00:08:35.074 --> 00:08:38.682 +which does that. You actually have the buffer open, + +00:08:38.683 --> 00:08:40.239 +it's much more clever. + +00:08:40.240 --> 00:08:45.999 +t-buffer-read, I believe, is simpler. As long as you have an + +00:08:46.000 --> 00:08:52.079 +Emacs list, what is called the current buffer active. I'm + +00:08:52.080 --> 00:08:56.679 +fairly sure you're able to just call next-line on it. I don't + +00:08:56.680 --> 00:08:59.479 +believe that I'm doing anything fancy there, looking for + +00:08:59.480 --> 00:09:03.999 +line ends. I believe I'm just grabbing the next line and then + +00:09:04.000 --> 00:09:09.423 +processing that line-wise. Very good. + +NOTE Q: Can the Elisp library be combined with the stream.el API or seq in general? + +00:09:09.424 --> 00:09:17.303 +Can the Elisp library be combined with the stream.el API + +00:09:17.304 --> 00:09:22.830 +or seq in general? I would say that these libraries + +00:09:22.831 --> 00:09:27.596 +are completely orthogonal. You saw that everything + +00:09:27.597 --> 00:09:29.279 +was prefixed by t-. + +00:09:29.280 --> 00:09:36.879 +Basically, transducer is its own zone. However, one thing + +00:09:36.880 --> 00:09:40.239 +that I do in the common lisp, which is theoretically + +00:09:40.240 --> 00:09:44.359 +possible for the Emacs Lisp as well, is kind of like little + +00:09:44.360 --> 00:09:48.919 +shim libraries. So I provide, at least for Common Lisp, for a + +00:09:48.920 --> 00:09:51.799 +number of, you know, popular sort of third-party + +00:09:51.800 --> 00:09:55.239 +collection types, I provide an ability to use them as + +00:09:55.240 --> 00:09:59.559 +sources. Maybe that's what you mean. Like + +00:09:59.560 --> 00:10:04.439 +the built-in containers for Emacs Lisp are already + +00:10:04.440 --> 00:10:06.519 +supported. So, you know, a vector hash table and so on. + +00:10:06.520 --> 00:10:13.719 +make sense so i think what i heard there is yeah go ahead + +00:10:13.720 --> 00:10:17.879 +please sorry in terms of mixing like you know like for + +00:10:17.880 --> 00:10:22.599 +instance you know like seq-map used in transducers + +00:10:22.600 --> 00:10:28.119 +we'll put it that way + +00:10:28.120 --> 00:10:31.879 +i was just gonna say i think it um it it sounds like you're + +00:10:31.880 --> 00:10:37.199 +saying Yeah, probably they are actually. We don't know yet + +00:10:37.200 --> 00:10:41.239 +about any places where they don't play nicely together. So + +00:10:41.240 --> 00:10:45.399 +quite possibly so. We can use sequence and transducers + +00:10:45.400 --> 00:10:49.959 +together, for example. As a source potentially, yeah. It's + +00:10:49.960 --> 00:10:54.159 +very easy because that just uses defgeneric. As long as you + +00:10:54.160 --> 00:10:57.719 +have a new, like if you have a new collection type, as long as + +00:10:57.720 --> 00:11:01.519 +you implement a def method for it somewhere, it'll just + +00:11:01.520 --> 00:11:12.159 +magically work with this library. That's the magic of... + +00:11:12.160 --> 00:11:18.439 +Yeah, as an Emacs user enjoying, you know, sort of the + +00:11:18.440 --> 00:11:21.959 +renaissance of new features it's had, or sorry, Emacs ERC + +00:11:21.960 --> 00:11:27.799 +user for chat. I've seen a lot of awesome stuff get done in the + +00:11:27.800 --> 00:11:32.119 +last couple of years with generic set. JP never was working + +00:11:32.120 --> 00:11:36.679 +on that. And like, that's just making me my eyes pop and go, + +00:11:36.680 --> 00:11:39.279 +wow, that does make a whole lot of things simpler, doesn't + +00:11:39.280 --> 00:11:44.279 +it? I think we're a lot of us running into generics and how + +00:11:44.280 --> 00:11:47.542 +that solves problems in Emacs. + +NOTE Q: How does one debug a t-comp expression? Can you single step and see intermediate results of the different statements you declare? + +00:11:47.543 --> 00:11:50.279 +How does one debug a t-comp + +00:11:50.280 --> 00:11:55.119 +expression? Can you talk in terms of single step, + +00:11:55.120 --> 00:11:58.479 +step-by-step, intermediate results of the different + +00:11:58.480 --> 00:12:08.759 +statements you declare? Yes. So in Common Lisp, this is + +00:12:08.760 --> 00:12:12.919 +and sly stickers and things like that. In Emacs Lisp, it's a + +00:12:12.920 --> 00:12:19.559 +little bit, shall we say, more difficult. For step + +00:12:19.560 --> 00:12:20.479 +debugging, + +00:12:20.480 --> 00:12:25.679 +so what comp does is comp internally, it should be a macro, + +00:12:25.680 --> 00:12:28.839 +but currently it's not, although there's work to improve + +00:12:28.840 --> 00:12:33.559 +that. It's doing an internal reduce and it's turning into + +00:12:33.560 --> 00:12:37.479 +one giant kind of composed lambda inside. So I don't know if + +00:12:37.480 --> 00:12:42.999 +step debugging would work there. However, we do have one + +00:12:43.000 --> 00:12:47.439 +function called log, which lets you inspect intermediate + +00:12:47.440 --> 00:12:50.759 +results. So you could technically use that to inject + +00:12:50.760 --> 00:12:54.279 +yourself somewhere into the transduction chain and, you + +00:12:54.280 --> 00:12:57.239 +know, halt or, you know, inspect the current value, et + +00:12:57.240 --> 00:13:01.119 +cetera. So you get a bunch of questions lined up. I think + +00:13:01.120 --> 00:13:04.199 +we're coming up, uh, within our last five minutes, uh, + +00:13:04.200 --> 00:13:07.919 +before some declared, uh, reset time that we have + +00:13:07.920 --> 00:13:11.919 +internally to just roll our closing credits, so to speak. + +00:13:11.920 --> 00:13:14.839 +Um, not that I would want to cut the question and answer + +00:13:14.840 --> 00:13:18.399 +short, but I might have to step away personally. But, um, as + +00:13:18.400 --> 00:13:21.519 +we discussed before, you can just kind of run the QA, however + +00:13:21.520 --> 00:13:24.879 +you want here. Um, or, or take questions offline if you'd + +00:13:24.880 --> 00:13:27.999 +like to answer them off the pad. And I just want to say one more + +00:13:28.000 --> 00:13:30.959 +time. Kitt said it managed later. Thanks again for your talk + +00:13:30.960 --> 00:13:35.759 +for dedicating the time to this live QA. And I think we can see + +00:13:35.760 --> 00:13:40.279 +by the many questions that are here. So I'll try to kind of + +00:13:40.280 --> 00:13:42.959 +flip us through as many of them as I can with our last couple of + +00:13:42.960 --> 00:13:48.399 +minutes, if that sounds good. Alternately, this might be a + +00:13:48.400 --> 00:13:52.079 +good time if you have kind of wrap it up, final thoughts, as + +00:13:52.080 --> 00:13:58.399 +Leo Sopanda saying. By all means, have at. Sure, thanks a + +00:13:58.400 --> 00:14:01.639 +lot. I'd say that if you are still curious, check out the + +00:14:01.640 --> 00:14:05.159 +read-me's because those have a lot of information, + +00:14:05.160 --> 00:14:09.519 +including a full description of the API and everything + +00:14:09.520 --> 00:14:10.719 +that's available. + +00:14:10.720 --> 00:14:16.599 +Otherwise, just give them a shot. Using these things is the + +00:14:16.600 --> 00:14:21.639 +best way to learn them, of course. I use them everywhere, + +00:14:21.640 --> 00:14:24.719 +basically, all across my Emacs list and all across my common + +00:14:24.720 --> 00:14:29.839 +list now. They get a lot of mileage. All right. You're + +00:14:29.840 --> 00:14:33.639 +speaking our language now. As Emacs users, all our ears poke + +00:14:33.640 --> 00:14:36.039 +up when you say, I'm getting a lot of mileage. I'm using it + +00:14:36.040 --> 00:14:39.879 +across everything. Every Emacs user has a story that + +00:14:39.880 --> 00:14:42.494 +harmonizes with that, I think. + +NOTE Q: Is there a path for transducers to enable elisp processing of otherwise overly large datasets as if just normal Emacs \"buffers\" (i.e. just pulling one thing at a time so essentially stream-like under the hood but buffer-like in interface), with none of the usual perf issues with a traditional buffer structure? + +00:14:42.495 --> 00:14:44.519 +So our next question, is + +00:14:44.520 --> 00:14:48.599 +there a path for transducers to enable Elisp processing or + +00:14:48.600 --> 00:14:53.999 +otherwise overly large data sets as if just normal Emacs + +00:14:54.000 --> 00:14:56.959 +buffers, i.e. just pulling one thing at a time. So + +00:14:56.960 --> 00:15:00.719 +essentially stream like under the hood, but buffer like an + +00:15:00.720 --> 00:15:03.519 +interface. I think that makes sense to me. with none of the + +00:15:03.520 --> 00:15:07.799 +usual performance issues, like as if, you know, the history + +00:15:07.800 --> 00:15:11.399 +with long files is what that brings to mind, I guess. Yes, so + +00:15:11.400 --> 00:15:15.799 +as you saw before, the withBufferRead sort of stream + +00:15:15.800 --> 00:15:19.879 +function does have to have the actual buffer in memory, and + +00:15:19.880 --> 00:15:22.679 +then you can go really fast. But there's another one with + +00:15:22.680 --> 00:15:26.839 +file read. Now, again, I haven't tried to optimize that yet. + +00:15:26.840 --> 00:15:30.119 +But in theory, it is able to read right from the underlying + +00:15:30.120 --> 00:15:32.839 +file without having to open it as a buffer first. + +00:15:32.840 --> 00:15:39.199 +Awesome. Ari, the performance issues mentioned, and that + +00:15:39.200 --> 00:15:43.479 +popped up recently in the list and forums, to what extent + +00:15:43.480 --> 00:15:46.959 +does tail call optimization and other mechanisms like + +00:15:46.960 --> 00:15:50.159 +inlining, garbage collection friendliness, and so on, + +00:15:50.160 --> 00:15:55.159 +could these alleviate issues, enable their use at little to + +00:15:55.160 --> 00:15:58.439 +no extra costs? I feel like we're leading the witness here, + +00:15:58.440 --> 00:16:01.279 +but I'm sure you see where we're going. Yeah, no problem. So + +00:16:01.280 --> 00:16:03.799 +in terms of tail optimization, that's already happening + +00:16:03.800 --> 00:16:09.199 +because the internal loop mechanism is using CL labels. And + +00:16:09.200 --> 00:16:12.199 +in Emacs Lisp, CL labels is just a macro that is like + +00:16:12.200 --> 00:16:16.079 +extremely tail recursive. So that's very, very fast. It's + +00:16:16.080 --> 00:16:19.039 +not tail recursive, but it's using like goto. So it's + +00:16:19.040 --> 00:16:22.519 +extremely, extremely fast, like the raw looping of it. So, + +00:16:22.520 --> 00:16:24.359 +okay, well then where does the slowness come from? It's + +00:16:24.360 --> 00:16:26.439 +probably coming from those lambdas and it's probably + +00:16:26.440 --> 00:16:32.399 +coming from, uh, like extra consing, extra allocation + +00:16:32.400 --> 00:16:35.999 +somewhere, which is, um, sort of what you were, what you're + +00:16:36.000 --> 00:16:38.519 +referring to with the GC friendliness. So perhaps there's + +00:16:38.520 --> 00:16:45.199 +some, um, um, yeah, some, like some fusion that I can do to + +00:16:45.200 --> 00:16:51.199 +speed it up. Yeah, that just sounds fascinating endlessly. + +NOTE Q: Is there an option to read a csv/json and produce an alist or plist instead of a hash table for an entry? + +00:16:51.200 --> 00:16:55.559 +Are there options to like read from a CSV, JSON, produce an + +00:16:55.560 --> 00:17:01.679 +alist or plist instead of hash table? Absolutely. + +00:17:01.680 --> 00:17:06.239 +Yes, I need to double check that, but we can read both CSV and + +00:17:06.240 --> 00:17:10.359 +JSON, and you should be able to just turn on the plist option. + +00:17:10.360 --> 00:17:14.159 +I will double check, but there's fairly free conversion + +00:17:14.160 --> 00:17:18.039 +between those three types because hash table is not always + +00:17:18.040 --> 00:17:22.039 +what you want. And actually, I suspect that slowness that we + +00:17:22.040 --> 00:17:24.559 +saw in the demo before was because it was allocating hash + +00:17:24.560 --> 00:17:29.239 +tables for every, like, all of the 50,000 lines. And had it + +00:17:29.240 --> 00:17:32.599 +been a plist, it would have been faster. Interesting, so + +00:17:32.600 --> 00:17:35.399 +maybe there's opportunities even if you end up with hash + +00:17:35.400 --> 00:17:38.799 +lists, but then they're shared strategically and you pay + +00:17:38.800 --> 00:17:42.039 +the cost of a little extra layer in there that buckets them + +00:17:42.040 --> 00:17:46.439 +together the way that we might group files by the first four + +00:17:46.440 --> 00:17:50.519 +characters in the file name once we've got a million files. + +NOTE Q: Is the common lisp version ready for 'production' use? Is it complete enough and the API stable enough? + +00:17:50.520 --> 00:17:54.479 +Anyway, is the Common Lisp version ready for production + +00:17:54.480 --> 00:17:59.959 +use? Do you want to comment on API stability? I use it all the + +00:17:59.960 --> 00:18:04.159 +time. I'm writing a game in Common Lisp right now, and I'm + +00:18:04.160 --> 00:18:08.559 +using transducers everywhere in there, and it doesn't even + +00:18:08.560 --> 00:18:11.119 +make a dent in the frame rate, and I'm using them + +00:18:11.120 --> 00:18:15.359 +extensively. Okay, well, I'll just read from chat. Thanks + +00:18:15.360 --> 00:18:17.476 +so much for the answers. + +NOTE Q: Do we need a pre-written \"t-\" version for every already existing reducing function like + or is there a function to construct them from already defined reducer 2-arg functions? + +00:18:17.477 --> 00:18:20.439 +Do we need a pre-written or t-minus + +00:18:20.440 --> 00:18:24.959 +version for every already existing reducing function, + +00:18:24.960 --> 00:18:30.239 +plus, as an example? Or is there a function that constructs, + +00:18:30.240 --> 00:18:33.559 +in my, I'm going to add the word, auto-visualifies them + +00:18:33.560 --> 00:18:37.319 +already, auto-defines or something, or just generically + +00:18:37.320 --> 00:18:42.239 +wraps function calls some way? already defined. This is + +00:18:42.240 --> 00:18:49.399 +basically fold. Some built-in functions like plus already + +00:18:49.400 --> 00:18:52.599 +function like reducers. It's a coincidence that they do + +00:18:52.600 --> 00:18:56.799 +that. But there's an example in the README. Max is one that + +00:18:56.800 --> 00:19:00.559 +does not act like that. For instance, maybe I could screen + +00:19:00.560 --> 00:19:06.479 +share later, but if you just type in plus one, If you call plus + +00:19:06.480 --> 00:19:10.519 +one in Emacs or Common Lisp, you get back one. It actually + +00:19:10.520 --> 00:19:15.119 +only needs one argument. If you only type plus, it actually + +00:19:15.120 --> 00:19:20.839 +gives you zero. Plus and multiple satisfy the API of + +00:19:20.840 --> 00:19:24.759 +reducers. But if you have one that doesn't, like the max + +00:19:24.760 --> 00:19:28.759 +function, and similarly, just type in plus as a function + +00:19:28.760 --> 00:19:32.359 +call, just plus with nothing else, and you'll see. No, as a + +00:19:32.360 --> 00:19:37.199 +function. zero will come out. This basically means it + +00:19:37.200 --> 00:19:43.159 +satisfies the reducer API. But a function like max does not. + +00:19:43.160 --> 00:19:48.399 +If you just type in max and then one, it won't work. Pardon me, + +00:19:48.400 --> 00:19:54.239 +it did. But if you type in max with nothing else, it wouldn't + +00:19:54.240 --> 00:19:55.239 +work. + +00:19:55.240 --> 00:19:58.599 +Hence, we have to wrap it in something like fold. I would say + +00:19:58.600 --> 00:20:01.919 +go look at the fold function. Right, which that I won't do. + +00:20:01.920 --> 00:20:04.839 +I'm not that well enough prepped. Darn it. Leo would have + +00:20:04.840 --> 00:20:08.399 +been here, but oh, well, you got me. Yeah, no problem. But + +00:20:08.400 --> 00:20:16.879 +fold is sort of the ultimate reducer function. Great. So is + +00:20:16.880 --> 00:20:26.319 +there, where was I? Here we go. We're way past this, right? So + +NOTE Q: Is the compelling argument for transducers is that it's a better abstraction? + +00:20:26.320 --> 00:20:34.279 +is the compiling argument for transducers that it's a + +00:20:34.280 --> 00:20:38.879 +better abstraction? It seems like there are concerns, + +00:20:38.880 --> 00:20:42.399 +objections, while problematically valid focused on + +00:20:42.400 --> 00:20:45.679 +implementation. Can this abstraction allow for advances + +00:20:45.680 --> 00:20:50.559 +in implementation? Yes, what I've basically done is mostly + +00:20:50.560 --> 00:20:55.999 +followed the pattern of usage that exists in Clojure and in + +00:20:56.000 --> 00:21:01.159 +Scheme's SERP 171. In theory, the service level API is the + +00:21:01.160 --> 00:21:04.999 +same no matter where you're using this, and that's the idea. + +00:21:05.000 --> 00:21:08.039 +If you learn them in one list, you should be able to use them + +00:21:08.040 --> 00:21:12.879 +everywhere. Then what it's actually doing under the hood is + +00:21:12.880 --> 00:21:18.359 +free for us to change around. My implementations are mostly + +00:21:18.360 --> 00:21:23.679 +based on the scheme with a few alterations here and there. + +00:21:23.680 --> 00:21:27.079 +And in the Common Lisp case, like adding some Common Lisp + +00:21:27.080 --> 00:21:27.959 +isms + +00:21:27.960 --> 00:21:34.759 +to improve usage like UX a little bit. But overall, we are + +00:21:34.760 --> 00:21:38.959 +free to do whatever we want internally to speed up + +00:21:38.960 --> 00:21:42.439 +performance. I just haven't done that work. Awesome. + +00:21:42.440 --> 00:21:47.239 +Awesome. So here's where I have to, where we're getting the + +00:21:47.240 --> 00:21:50.079 +hook. We've just been pulled off the stream. The viewers + +00:21:50.080 --> 00:21:54.079 +just saw the crawl by as it sent us over to the other pad where I + +00:21:54.080 --> 00:21:57.919 +get to jump on and get involved with that now. But I can't + +00:21:57.920 --> 00:22:00.359 +thank you enough, Colin. Would you like me to stop the + +00:22:00.360 --> 00:22:03.799 +recording here? Any other comments you'd like to make? Uh, + +00:22:03.800 --> 00:22:06.439 +yeah, sure. Like, I mean, I'll stick around for any more live + +00:22:06.440 --> 00:22:10.639 +questions. I'm looking at both IRC and, and, um, uh, big blue + +00:22:10.640 --> 00:22:13.239 +button here. So if people have more questions, I'll hang + +00:22:13.240 --> 00:22:15.959 +around for a bit. I'm going to leave the channel open. I see + +00:22:15.960 --> 00:22:17.839 +you do have a few people in here, so I'm just going to go ahead + +00:22:17.840 --> 00:22:20.839 +and leave the recording. We can always trim it. Um, trim it + +00:22:20.840 --> 00:22:24.279 +up. If you, uh, let us know, Hey, the last 10 minutes weren't + +00:22:24.280 --> 00:22:26.999 +anything, you know, or whatever. No, no pressure, no + +00:22:27.000 --> 00:22:29.839 +worries, no mistakes. Thank you. Really appreciate you. + +00:22:29.840 --> 00:22:31.959 +Yep. Thanks a lot. + +NOTE Q: Question about how the transducers video was made? Did you use Reveal.js? Do you have a pointer to the html hosted presentation? How did you generate the content for Reveal? + +00:22:31.960 --> 00:22:48.399 +OK, does anyone else have some questions? I see Mohsen in the + +00:22:48.400 --> 00:22:52.839 +BigBlueButton chat is asking how I made the video. So the + +00:22:52.840 --> 00:22:59.079 +presentation itself was done with RevealJS from Org Mode. + +00:22:59.080 --> 00:23:03.639 +So as you saw, I had a raw Org Mode buffer, which was + +00:23:03.640 --> 00:23:09.319 +which was the presentation itself, which I then just + +00:23:09.320 --> 00:23:11.759 +exported with a few certain settings, a few + +00:23:11.760 --> 00:23:15.919 +customizations. And then for screen recording, I used OBS, + +00:23:15.920 --> 00:23:19.719 +which worked flawlessly on Arch Linux. I used Sway, + +00:23:19.720 --> 00:23:23.159 +Wayland, and all of that. So all of that just worked, which + +00:23:23.160 --> 00:23:27.999 +was very impressive. Where do the HTML host the + +00:23:28.000 --> 00:23:51.959 +presentation? I don't have that presentation hosted + +00:23:51.960 --> 00:23:52.599 +anywhere. + +00:23:52.600 --> 00:23:59.119 +I'll look at the. + +00:23:59.120 --> 00:24:00.079 +I don't see that. + +00:24:00.080 --> 00:24:08.159 +Here it is. So we've got the file here as well. + +00:24:08.160 --> 00:24:10.999 +Looks like that's it for questions, basically. + +00:24:11.000 --> 00:24:14.919 +Yep, and it looks like everyone's moved on for now. Let's + +00:24:14.920 --> 00:24:20.159 +see. I mean, it would be so this is answering lounge 81 on IRC. + +NOTE Q: From your investigations and tests so far, do you think there would be the necessity of transducers to eventually go down into the C level code for things like using them to solve "infinitely-big" buffer-like interfaces and such? + +00:24:20.160 --> 00:24:24.599 +Yeah, like, if we really wanted to go that hardcore, maybe + +00:24:24.600 --> 00:24:29.439 +there's some like C level stuff that we could + +00:24:29.440 --> 00:24:36.119 +you know, significant demand for such a thing. You know, so + +00:24:36.120 --> 00:24:39.239 +far there hasn't been such demand, but maybe there will be in + +00:24:39.240 --> 00:24:42.519 +the future. Yeah, perhaps there's some custom stuff we + +00:24:42.520 --> 00:24:43.039 +could do. + +00:24:43.040 --> 00:24:48.599 +And otherwise, magic one. + +00:24:48.600 --> 00:25:00.599 +Well, it looks like some people are quite happy with this. + +00:25:00.600 --> 00:25:14.959 +All right. That's about what I've seen. So why don't we end it + +00:25:14.960 --> 00:25:19.839 +here? I think I can control the recording from my end. If I + +00:25:19.840 --> 00:25:23.800 +pause it, will that work? All right. Thank you, everyone. |