summaryrefslogblamecommitdiffstats
path: root/2021/captions/bindat.md
blob: 062de190ad567bb038543129c8d457e886ddb54e (plain) (tree)
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471




























































                                                                                                                              
                                                                                                                  
























































































































































































































































































































































































































                                                                                                                                       
                                                                                                             




















































































































































                                                                                                                            
<a name="transcript"></a>
# Transcript

[[!template new="1" text="Hi. So I'm going to talk today" start="00:00:01.360" video="mainVideo" id=subtitle]]
[[!template text="about a fun rewrite I did of the BinDat package." start="00:00:04.180" video="mainVideo" id=subtitle]]
[[!template text="I call this Turbo BinDat." start="00:00:10.000" video="mainVideo" id=subtitle]]
[[!template text="Actually, the package hasn't changed name," start="00:00:12.400" video="mainVideo" id=subtitle]]
[[!template text="it's just that the result happens to be faster." start="00:00:14.101" video="mainVideo" id=subtitle]]
[[!template text="The point was not to make it faster though," start="00:00:16.901" video="mainVideo" id=subtitle]]
[[!template text="and the point was not to make you understand" start="00:00:19.621" video="mainVideo" id=subtitle]]
[[!template text="that data is not code." start="00:00:22.341" video="mainVideo" id=subtitle]]
[[!template text="It's just one more experience I've had" start="00:00:23.540" video="mainVideo" id=subtitle]]
[[!template text="where I've seen that treating data as code" start="00:00:27.120" video="mainVideo" id=subtitle]]
[[!template text="is not always a good idea." start="00:00:31.381" video="mainVideo" id=subtitle]]
[[!template text="It's important to keep the difference." start="00:00:33.622" video="mainVideo" id=subtitle]]
[[!template text="So let's get started." start="00:00:36.162" video="mainVideo" id=subtitle]]
[[!template new="1" text="So what is BinDat anyway?" start="00:00:38.881" video="mainVideo" id=subtitle]]
[[!template text="Here's just the overview of basically" start="00:00:40.742" video="mainVideo" id=subtitle]]
[[!template text="what I'm going to present." start="00:00:43.602" video="mainVideo" id=subtitle]]
[[!template text="So I'm first going to present BinDat itself" start="00:00:45.062" video="mainVideo" id=subtitle]]
[[!template text="for those who don't know it," start="00:00:47.843" video="mainVideo" id=subtitle]]
[[!template text="which is probably the majority of you." start="00:00:49.039" video="mainVideo" id=subtitle]]
[[!template text="Then I'm going to talk about the actual problems" start="00:00:51.923" video="mainVideo" id=subtitle]]
[[!template text="that I encountered with this package" start="00:00:55.363" video="mainVideo" id=subtitle]]
[[!template text="that motivated me to rewrite it." start="00:00:58.882" video="mainVideo" id=subtitle]]
[[!template text="Most of them were lack of flexibility," start="00:01:01.843" video="mainVideo" id=subtitle]]
[[!template text="and some of it was just poor behavior" start="00:01:05.044" video="mainVideo" id=subtitle]]
[[!template text="with respect to scoping and variables," start="00:01:09.924" video="mainVideo" id=subtitle]]
[[!template text="which of course, you know, is bad --" start="00:01:13.364" video="mainVideo" id=subtitle]]
[[!template text="basically uses of eval or, &quot ;eval is evil.&quot ;" start="00:01:16.424" video="mainVideo" id=subtitle]]
[[!template text="Then I'm going to talk about the new design --" start="00:01:20.724" video="mainVideo" id=subtitle]]
[[!template text="how I redesigned it" start="00:01:24.985" video="mainVideo" id=subtitle]]
[[!template text="to make it both simpler and more flexible," start="00:01:28.105" video="mainVideo" id=subtitle]]
[[!template text="and where the key idea was" start="00:01:31.365" video="mainVideo" id=subtitle]]
[[!template text="to expose code as code" start="00:01:33.065" video="mainVideo" id=subtitle]]
[[!template text="instead of having it as data," start="00:01:35.305" video="mainVideo" id=subtitle]]
[[!template text="and so here the distinction between the two" start="00:01:37.625" video="mainVideo" id=subtitle]]
[[!template text="is important and made things simpler." start="00:01:39.706" video="mainVideo" id=subtitle]]
[[!template text="I tried to keep efficiency in mind," start="00:01:44.085" video="mainVideo" id=subtitle]]
[[!template text="which resulted in some of the aspects of the design" start="00:01:46.405" video="mainVideo" id=subtitle]]
[[!template text="which are not completely satisfactory," start="00:01:52.505" video="mainVideo" id=subtitle]]
[[!template text="but the result is actually fairly efficient." start="00:01:54.886" video="mainVideo" id=subtitle]]
[[!template text="Even though it was not the main motivation," start="00:01:57.146" video="mainVideo" id=subtitle]]
[[!template text="it was one of the nice outcomes." start="00:01:59.287" video="mainVideo" id=subtitle]]
[[!template text="And then I'm going to present some examples." start="00:02:02.967" video="mainVideo" id=subtitle]]
[[!template new="1" text="So first: what is BinDat?" start="00:02:06.007" video="mainVideo" id=subtitle]]
[[!template text="Oh actually, rather than present THIS," start="00:02:08.267" video="mainVideo" id=subtitle]]
[[!template text="I'm going to go straight to the code," start="00:02:10.667" video="mainVideo" id=subtitle]]
[[!template text="because BinDat actually had" start="00:02:12.507" video="mainVideo" id=subtitle]]
[[!template text="an introduction which was fairly legible." start="00:02:14.346" video="mainVideo" id=subtitle]]
[[!template text="So here we go: this is the old BinDat from Emacs 27" start="00:02:16.748" video="mainVideo" id=subtitle]]
[[!template text="and the commentary starts by explaining" start="00:02:21.128" video="mainVideo" id=subtitle]]
[[!template text="what is BinDat? Basically BinDat is a package" start="00:02:23.448" video="mainVideo" id=subtitle]]
[[!template text="that lets you parse and unparse" start="00:02:25.948" video="mainVideo" id=subtitle]]
[[!template text="basically binary data." start="00:02:30.247" video="mainVideo" id=subtitle]]
[[!template text="The intent is to have typically network data" start="00:02:31.627" video="mainVideo" id=subtitle]]
[[!template text="or something like this." start="00:02:34.749" video="mainVideo" id=subtitle]]
[[!template text="So assuming you have network data," start="00:02:35.949" video="mainVideo" id=subtitle]]
[[!template text="presented or defined" start="00:02:38.328" video="mainVideo" id=subtitle]]
[[!template text="with some kind of C-style structs, typically," start="00:02:41.628" video="mainVideo" id=subtitle]]
[[!template text="or something along these lines." start="00:02:44.669" video="mainVideo" id=subtitle]]
[[!template text="So you presumably start with documentation" start="00:02:46.109" video="mainVideo" id=subtitle]]
[[!template text="that presents something like those structs here," start="00:02:49.120" video="mainVideo" id=subtitle]]
[[!template text="and you want to be able to generate such packets" start="00:02:52.810" video="mainVideo" id=subtitle]]
[[!template text="and read such packets," start="00:02:57.230" video="mainVideo" id=subtitle]]
[[!template text="so the way you do it is" start="00:03:00.349" video="mainVideo" id=subtitle]]
[[!template text="you rewrite those specifications" start="00:03:02.190" video="mainVideo" id=subtitle]]
[[!template text="into the BinDat syntax." start="00:03:04.670" video="mainVideo" id=subtitle]]
[[!template text="So here's the BinDat syntax" start="00:03:06.110" video="mainVideo" id=subtitle]]
[[!template text="for the the previous specification." start="00:03:07.529" video="mainVideo" id=subtitle]]
[[!template text="So here, for example," start="00:03:10.491" video="mainVideo" id=subtitle]]
[[!template text="you see the case for a data packet" start="00:03:11.610" video="mainVideo" id=subtitle]]
[[!template text="which will have a 'type' field which is a byte" start="00:03:16.970" video="mainVideo" id=subtitle]]
[[!template text="(an unsigned 8-bit entity)," start="00:03:20.411" video="mainVideo" id=subtitle]]
[[!template text="then an 'opcode' which is also a byte," start="00:03:24.091" video="mainVideo" id=subtitle]]
[[!template text="then a 'length' which is a 16-bit unsigned integer" start="00:03:26.411" video="mainVideo" id=subtitle]]
[[!template text="in little endian order," start="00:03:30.732" video="mainVideo" id=subtitle]]
[[!template text="and then some 'id' for this entry, which is" start="00:03:34.092" video="mainVideo" id=subtitle]]
[[!template text="8 bytes containing a zero-terminated string," start="00:03:38.732" video="mainVideo" id=subtitle]]
[[!template text="and then the actual data, basically the payload," start="00:03:43.531" video="mainVideo" id=subtitle]]
[[!template text="which is in this case a vector of bytes," start="00:03:47.532" video="mainVideo" id=subtitle]]
[[!template text="('bytes' here doesn't doesn't need to be specified)" start="00:03:51.453" video="mainVideo" id=subtitle]]
[[!template text="and here we specify the length of this vector." start="00:03:54.812" video="mainVideo" id=subtitle]]
[[!template text="This 'length' here" start="00:03:58.172" video="mainVideo" id=subtitle]]
[[!template text="happens to be actually the name of THIS field," start="00:03:59.773" video="mainVideo" id=subtitle]]
[[!template text="so the length of the data" start="00:04:02.252" video="mainVideo" id=subtitle]]
[[!template text="is specified by the 'length' field here," start="00:04:03.854" video="mainVideo" id=subtitle]]
[[!template text="and BinDat will understand this part," start="00:04:06.574" video="mainVideo" id=subtitle]]
[[!template text="which is the the nice part of BinDat." start="00:04:08.574" video="mainVideo" id=subtitle]]
[[!template text="And then you have an alignment field at the end," start="00:04:12.333" video="mainVideo" id=subtitle]]
[[!template text="which is basically padding." start="00:04:15.774" video="mainVideo" id=subtitle]]
[[!template text="It says that it is padded" start="00:04:18.253" video="mainVideo" id=subtitle]]
[[!template text="until the next multiple of four." start="00:04:20.575" video="mainVideo" id=subtitle]]
[[!template text="Okay. So this works reasonably well." start="00:04:23.295" video="mainVideo" id=subtitle]]
[[!template text="This is actually very nice." start="00:04:25.855" video="mainVideo" id=subtitle]]
[[!template text="With this, you can then call" start="00:04:27.455" video="mainVideo" id=subtitle]]
[[!template text="bindat-pack or bindat-unpack," start="00:04:30.335" video="mainVideo" id=subtitle]]
[[!template text="passing it a string, or passing it an alist," start="00:04:32.975" video="mainVideo" id=subtitle]]
[[!template text="to do the packing and unpacking." start="00:04:37.774" video="mainVideo" id=subtitle]]
[[!template text="So, for example, if you take this string--" start="00:04:40.416" video="mainVideo" id=subtitle]]
[[!template text="actually, in this case, it's a vector of bytes" start="00:04:43.296" video="mainVideo" id=subtitle]]
[[!template text="but it works the same; it works in both ways--" start="00:04:45.856" video="mainVideo" id=subtitle]]
[[!template text="if you pass this to bindat-unpack," start="00:04:49.456" video="mainVideo" id=subtitle]]
[[!template text="it will presumably return you this structure" start="00:04:53.536" video="mainVideo" id=subtitle]]
[[!template text="if you've given it the corresponding type." start="00:04:57.457" video="mainVideo" id=subtitle]]
[[!template text="So it will extract--" start="00:05:00.017" video="mainVideo" id=subtitle]]
[[!template text="you will see that there is an IP address," start="00:05:01.776" video="mainVideo" id=subtitle]]
[[!template text="which is a destination IP, a source IP," start="00:05:05.617" video="mainVideo" id=subtitle]]
[[!template text="and some port number," start="00:05:08.017" video="mainVideo" id=subtitle]]
[[!template text="and some actual data here and there, etc." start="00:05:09.857" video="mainVideo" id=subtitle]]
[[!template text="So this is quite convenient if you need to do this," start="00:05:12.977" video="mainVideo" id=subtitle]]
[[!template text="and that's what it was designed for." start="00:05:18.018" video="mainVideo" id=subtitle]]
[[!template text="So here we are. Let's go back to the actual talk." start="00:05:20.898" video="mainVideo" id=subtitle]]
[[!template new="1" text="I converted BinDat to lexical scoping at some point" start="00:05:27.538" video="mainVideo" id=subtitle]]
[[!template text="and things seemed to work fine," start="00:05:34.339" video="mainVideo" id=subtitle]]
[[!template text="except, at some point, probably weeks later," start="00:05:37.299" video="mainVideo" id=subtitle]]
[[!template text="I saw a bug report" start="00:05:42.819" video="mainVideo" id=subtitle]]
[[!template text="about the new version using lexical scoping" start="00:05:47.139" video="mainVideo" id=subtitle]]
[[!template text="not working correctly with WeeChat." start="00:05:53.059" video="mainVideo" id=subtitle]]
[[!template text="So here's the actual chunk of code" start="00:05:56.339" video="mainVideo" id=subtitle]]
[[!template text="that appears in WeeChat." start="00:06:00.580" video="mainVideo" id=subtitle]]
[[!template text="Here you see that they also define a BinDat spec." start="00:06:02.820" video="mainVideo" id=subtitle]]
[[!template text="It's a packet that has a 32-bit unsigned length," start="00:06:08.421" video="mainVideo" id=subtitle]]
[[!template text="then some compression byte/compression information," start="00:06:14.741" video="mainVideo" id=subtitle]]
[[!template text="then an id which contains basically another struct" start="00:06:18.500" video="mainVideo" id=subtitle]]
[[!template text="(which is specified elsewhere; doesn't matter here)," start="00:06:23.780" video="mainVideo" id=subtitle]]
[[!template text="and after that, a vector" start="00:06:26.902" video="mainVideo" id=subtitle]]
[[!template text="whose size is not just specified by 'length'," start="00:06:28.661" video="mainVideo" id=subtitle]]
[[!template text="but is computed from 'length'." start="00:06:33.382" video="mainVideo" id=subtitle]]
[[!template text="So here's how they used to compute it in WeeChat." start="00:06:35.142" video="mainVideo" id=subtitle]]
[[!template text="So the length here can be specified in BinDat." start="00:06:39.142" video="mainVideo" id=subtitle]]
[[!template text="Instead of having" start="00:06:42.822" video="mainVideo" id=subtitle]]
[[!template text="just a reference to one of the fields," start="00:06:43.942" video="mainVideo" id=subtitle]]
[[!template text="or having a constant, you can actually compute it," start="00:06:45.863" video="mainVideo" id=subtitle]]
[[!template text="where you have to use this '(eval'," start="00:06:48.903" video="mainVideo" id=subtitle]]
[[!template text="and then followed by the actual expression" start="00:06:52.502" video="mainVideo" id=subtitle]]
[[!template text="where you say how you compute it." start="00:06:54.743" video="mainVideo" id=subtitle]]
[[!template text="And here you see that it actually computes it" start="00:06:58.103" video="mainVideo" id=subtitle]]
[[!template text="based on the 'length of the structure --" start="00:07:01.464" video="mainVideo" id=subtitle]]
[[!template text="that's supposed to be this 'length' field here --" start="00:07:04.904" video="mainVideo" id=subtitle]]
[[!template text="and it's referred to using the bindat-get-field" start="00:07:07.783" video="mainVideo" id=subtitle]]
[[!template text="to extract the field from the variable 'struct'." start="00:07:11.223" video="mainVideo" id=subtitle]]
[[!template text="And then it subtracts four, it subtracts one," start="00:07:14.503" video="mainVideo" id=subtitle]]
[[!template text="and adds some other things" start="00:07:17.943" video="mainVideo" id=subtitle]]
[[!template text="which depend on some field" start="00:07:19.468" video="mainVideo" id=subtitle]]
[[!template text="that's found in this 'id' field here." start="00:07:22.185" video="mainVideo" id=subtitle]]
[[!template text="And the problem with this code" start="00:07:26.905" video="mainVideo" id=subtitle]]
[[!template text="was that it broke" start="00:07:28.425" video="mainVideo" id=subtitle]]
[[!template text="because of this 'struct' variable here," start="00:07:30.425" video="mainVideo" id=subtitle]]
[[!template text="because this 'struct' variable is not defined" start="00:07:32.745" video="mainVideo" id=subtitle]]
[[!template text="anywhere in the specification of BinDat." start="00:07:35.145" video="mainVideo" id=subtitle]]
[[!template text="It was used internally as a local variable," start="00:07:38.106" video="mainVideo" id=subtitle]]
[[!template text="and because it was using dynamic scoping," start="00:07:41.866" video="mainVideo" id=subtitle]]
[[!template text="it actually happened to be available here," start="00:07:45.306" video="mainVideo" id=subtitle]]
[[!template text="but the documentation nowhere specifies it." start="00:07:47.386" video="mainVideo" id=subtitle]]
[[!template text="So it was not exactly" start="00:07:50.826" video="mainVideo" id=subtitle]]
[[!template text="a bug of the conversion to lexical scoping," start="00:07:52.506" video="mainVideo" id=subtitle]]
[[!template text="but it ended up breaking this code." start="00:07:55.547" video="mainVideo" id=subtitle]]
[[!template text="And there was no way to actually" start="00:07:58.906" video="mainVideo" id=subtitle]]
[[!template text="fix the code within the specification of BinDat." start="00:08:01.226" video="mainVideo" id=subtitle]]
[[!template text="You had to go outside the specification of BinDat" start="00:08:05.066" video="mainVideo" id=subtitle]]
[[!template text="to fix this problem." start="00:08:08.287" video="mainVideo" id=subtitle]]
[[!template text="This is basically how I started looking at BinDat." start="00:08:10.427" video="mainVideo" id=subtitle]]
[[!template text="Then I went to actually investigate a bit more" start="00:08:14.347" video="mainVideo" id=subtitle]]
[[!template text="what was going on," start="00:08:17.808" video="mainVideo" id=subtitle]]
[[!template text="and the thing I noticed along the way" start="00:08:19.627" video="mainVideo" id=subtitle]]
[[!template text="was basically that the specification of BinDat" start="00:08:22.108" video="mainVideo" id=subtitle]]
[[!template text="is fairly complex and has a lot of eval" start="00:08:25.787" video="mainVideo" id=subtitle]]
[[!template text="and things like this." start="00:08:29.528" video="mainVideo" id=subtitle]]
[[!template new="1" text="So let's take a look" start="00:08:30.749" video="mainVideo" id=subtitle]]
[[!template text="at what the BinDat specification looks like." start="00:08:32.288" video="mainVideo" id=subtitle]]
[[!template text="So here it's actually documented" start="00:08:35.068" video="mainVideo" id=subtitle]]
[[!template text="as a kind of grammar rules." start="00:08:36.589" video="mainVideo" id=subtitle]]
[[!template text="A specification is basically a sequence of items," start="00:08:40.269" video="mainVideo" id=subtitle]]
[[!template text="and then each of the items is basically" start="00:08:45.308" video="mainVideo" id=subtitle]]
[[!template text="a FIELD of a struct, so it has a FIELD name," start="00:08:47.389" video="mainVideo" id=subtitle]]
[[!template text="and then a TYPE." start="00:08:51.249" video="mainVideo" id=subtitle]]
[[!template text="Instead of a TYPE," start="00:08:53.249" video="mainVideo" id=subtitle]]
[[!template text="it could have some other FORM for eval," start="00:08:54.510" video="mainVideo" id=subtitle]]
[[!template text="which was basically never used as far as I know," start="00:08:56.590" video="mainVideo" id=subtitle]]
[[!template text="or it can be some filler," start="00:08:58.989" video="mainVideo" id=subtitle]]
[[!template text="or you can have some 'align' specification," start="00:09:00.190" video="mainVideo" id=subtitle]]
[[!template text="or you can refer to another struct." start="00:09:02.750" video="mainVideo" id=subtitle]]
[[!template text="It could also be some kind of union," start="00:09:05.150" video="mainVideo" id=subtitle]]
[[!template text="or it can be some kind of repetition of something." start="00:09:07.391" video="mainVideo" id=subtitle]]
[[!template text="And then you have the TYPE specified here," start="00:09:10.430" video="mainVideo" id=subtitle]]
[[!template text="which can be some integers, strings, or a vector," start="00:09:12.430" video="mainVideo" id=subtitle]]
[[!template text="and there are a few other special cases." start="00:09:18.271" video="mainVideo" id=subtitle]]
[[!template text="And then the actual field itself" start="00:09:21.631" video="mainVideo" id=subtitle]]
[[!template text="can be either a NAME, or something that's computed," start="00:09:25.311" video="mainVideo" id=subtitle]]
[[!template text="and then everywhere here, you have LEN," start="00:09:28.192" video="mainVideo" id=subtitle]]
[[!template text="which specifies the length of vectors," start="00:09:30.752" video="mainVideo" id=subtitle]]
[[!template text="for example, or length of strings." start="00:09:32.480" video="mainVideo" id=subtitle]]
[[!template text="This is actually either nil to mean one," start="00:09:34.672" video="mainVideo" id=subtitle]]
[[!template text="or it can be an ARG," start="00:09:37.632" video="mainVideo" id=subtitle]]
[[!template text="where ARG is defined to be" start="00:09:39.072" video="mainVideo" id=subtitle]]
[[!template text="either an integer or DEREF," start="00:09:40.952" video="mainVideo" id=subtitle]]
[[!template text="where DEREF is basically a specification" start="00:09:42.673" video="mainVideo" id=subtitle]]
[[!template text="that can refer, for example, to the 'length' field" start="00:09:46.673" video="mainVideo" id=subtitle]]
[[!template text="-- that's what we saw between parentheses: (length)" start="00:09:48.833" video="mainVideo" id=subtitle]]
[[!template text="was this way to refer to the 'length' field." start="00:09:51.956" video="mainVideo" id=subtitle]]
[[!template text="Or it can be an expression, which is what we saw" start="00:09:56.273" video="mainVideo" id=subtitle]]
[[!template text="in the computation of the length for WeeChat," start="00:09:59.794" video="mainVideo" id=subtitle]]
[[!template text="where you just had a '(eval'" start="00:10:02.834" video="mainVideo" id=subtitle]]
[[!template text="and then some computation" start="00:10:04.914" video="mainVideo" id=subtitle]]
[[!template text="of the length of the payload." start="00:10:06.334" video="mainVideo" id=subtitle]]
[[!template text="And so if you look here, you see that" start="00:10:10.274" video="mainVideo" id=subtitle]]
[[!template text="it is fairly large and complex," start="00:10:12.354" video="mainVideo" id=subtitle]]
[[!template text="and it uses eval everywhere. And actually," start="00:10:14.674" video="mainVideo" id=subtitle]]
[[!template text="it's not just that it has eval in its syntax," start="00:10:18.515" video="mainVideo" id=subtitle]]
[[!template text="but the implementation has to use eval everywhere," start="00:10:20.675" video="mainVideo" id=subtitle]]
[[!template text="because, if you go back" start="00:10:23.395" video="mainVideo" id=subtitle]]
[[!template text="to see the kind of code we see," start="00:10:25.314" video="mainVideo" id=subtitle]]
[[!template text="we see here we just define" start="00:10:27.475" video="mainVideo" id=subtitle]]
[[!template text="weechat--relay-message-spec as a constant!" start="00:10:29.538" video="mainVideo" id=subtitle]]
[[!template text="It's nothing than just data, right?" start="00:10:34.195" video="mainVideo" id=subtitle]]
[[!template text="So within this data" start="00:10:37.315" video="mainVideo" id=subtitle]]
[[!template text="there are things we need to evaluate," start="00:10:38.836" video="mainVideo" id=subtitle]]
[[!template text="but it's pure data," start="00:10:41.076" video="mainVideo" id=subtitle]]
[[!template text="so it will have to be evaluated" start="00:10:42.356" video="mainVideo" id=subtitle]]
[[!template text="by passing it to eval. It can't be compiled," start="00:10:44.356" video="mainVideo" id=subtitle]]
[[!template text="because it's within a quote, right?" start="00:10:46.596" video="mainVideo" id=subtitle]]
[[!template text="And so for that reason, kittens really" start="00:10:50.196" video="mainVideo" id=subtitle]]
[[!template text="suffer terribly with uses of BinDat." start="00:10:52.837" video="mainVideo" id=subtitle]]
[[!template text="You really have to be very careful with that." start="00:10:55.956" video="mainVideo" id=subtitle]]
[[!template text="More seriously," start="00:10:59.957" video="mainVideo" id=subtitle]]
[[!template text="the 'struct' variable was not documented," start="00:11:02.037" video="mainVideo" id=subtitle]]
[[!template text="and yet it's indispensable" start="00:11:05.157" video="mainVideo" id=subtitle]]
[[!template text="for important applications," start="00:11:07.797" video="mainVideo" id=subtitle]]
[[!template text="such as using in WeeChat." start="00:11:08.996" video="mainVideo" id=subtitle]]
[[!template text="So clearly this needs to be fixed." start="00:11:11.158" video="mainVideo" id=subtitle]]
[[!template text="Of course, we can just document 'struct'" start="00:11:13.078" video="mainVideo" id=subtitle]]
[[!template text="as some variable that's used there," start="00:11:15.481" video="mainVideo" id=subtitle]]
[[!template text="but of course we don't want to do that," start="00:11:18.038" video="mainVideo" id=subtitle]]
[[!template text="because 'struct' is not obviously" start="00:11:19.798" video="mainVideo" id=subtitle]]
[[!template text="a dynamically scoped variable," start="00:11:23.398" video="mainVideo" id=subtitle]]
[[!template text="so it's not very clean." start="00:11:25.398" video="mainVideo" id=subtitle]]
[[!template text="Also other problems I noticed was that the grammar" start="00:11:29.318" video="mainVideo" id=subtitle]]
[[!template text="is significantly more complex than necessary." start="00:11:31.939" video="mainVideo" id=subtitle]]
[[!template text="We have nine distinct non-terminals." start="00:11:35.239" video="mainVideo" id=subtitle]]
[[!template text="There is ambiguity." start="00:11:38.199" video="mainVideo" id=subtitle]]
[[!template text="If you try to use a field whose name is 'align'," start="00:11:39.639" video="mainVideo" id=subtitle]]
[[!template text="or 'fill', or something like this," start="00:11:44.919" video="mainVideo" id=subtitle]]
[[!template text="then it's going to be misinterpreted," start="00:11:48.680" video="mainVideo" id=subtitle]]
[[!template text="or it can be misinterpreted." start="00:11:50.920" video="mainVideo" id=subtitle]]
[[!template text="The vector length can be either an expression," start="00:11:54.920" video="mainVideo" id=subtitle]]
[[!template text="or an integer, or a reference to a label," start="00:11:58.760" video="mainVideo" id=subtitle]]
[[!template text="but the expression" start="00:12:02.280" video="mainVideo" id=subtitle]]
[[!template text="should already be the general case," start="00:12:03.720" video="mainVideo" id=subtitle]]
[[!template text="and this expression can itself be" start="00:12:06.361" video="mainVideo" id=subtitle]]
[[!template text="just a constant integer," start="00:12:08.041" video="mainVideo" id=subtitle]]
[[!template text="so this complexity is probably not indispensable," start="00:12:09.401" video="mainVideo" id=subtitle]]
[[!template text="or it could be replaced with something simpler." start="00:12:13.961" video="mainVideo" id=subtitle]]
[[!template text="That's what I felt like." start="00:12:15.641" video="mainVideo" id=subtitle]]
[[!template text="And basically lots of places" start="00:12:17.401" video="mainVideo" id=subtitle]]
[[!template text="allow an (eval EXP) form somewhere" start="00:12:19.161" video="mainVideo" id=subtitle]]
[[!template text="to open up the door for more flexibility," start="00:12:21.721" video="mainVideo" id=subtitle]]
[[!template text="but not all of them do," start="00:12:25.082" video="mainVideo" id=subtitle]]
[[!template text="and we don't really want" start="00:12:26.922" video="mainVideo" id=subtitle]]
[[!template text="to have this eval there, right?" start="00:12:29.482" video="mainVideo" id=subtitle]]
[[!template text="It's not very convenient syntactically either." start="00:12:31.001" video="mainVideo" id=subtitle]]
[[!template text="So it makes the uses of eval" start="00:12:33.802" video="mainVideo" id=subtitle]]
[[!template text="a bit heavier than they need to be," start="00:12:36.042" video="mainVideo" id=subtitle]]
[[!template text="and so I didn't really like this part." start="00:12:38.362" video="mainVideo" id=subtitle]]
[[!template text="Another part is that" start="00:12:41.723" video="mainVideo" id=subtitle]]
[[!template text="when I tried to figure out what was going on," start="00:12:42.603" video="mainVideo" id=subtitle]]
[[!template text="dog barks and distracts Stefan" start="00:12:45.183" video="mainVideo" id=subtitle]]
[[!template text="I had trouble... Winnie as well, as you can hear." start="00:12:46.666" video="mainVideo" id=subtitle]]
[[!template text="She had trouble as well." start="00:12:50.043" video="mainVideo" id=subtitle]]
[[!template text="But one of the troubles was that" start="00:12:50.923" video="mainVideo" id=subtitle]]
[[!template text="there was no way to debug the code" start="00:12:53.083" video="mainVideo" id=subtitle]]
[[!template text="via Edebug, because it's just data," start="00:12:55.002" video="mainVideo" id=subtitle]]
[[!template text="so Edebug doesn't know that it has to look at it" start="00:12:57.562" video="mainVideo" id=subtitle]]
[[!template text="and instrument it." start="00:13:00.524" video="mainVideo" id=subtitle]]
[[!template text="And of course it was not conveniently extensible." start="00:13:02.683" video="mainVideo" id=subtitle]]
[[!template text="That's also one of the things" start="00:13:05.644" video="mainVideo" id=subtitle]]
[[!template text="I noticed along the way." start="00:13:07.164" video="mainVideo" id=subtitle]]
[[!template text="Okay, so here's an example of" start="00:13:09.084" video="mainVideo" id=subtitle]]
[[!template text="problems not that I didn't just see there," start="00:13:12.844" video="mainVideo" id=subtitle]]
[[!template text="but that were actually present in code." start="00:13:15.485" video="mainVideo" id=subtitle]]
[[!template text="I went to look at code that was using BinDat" start="00:13:18.684" video="mainVideo" id=subtitle]]
[[!template text="to see what uses looked like," start="00:13:22.124" video="mainVideo" id=subtitle]]
[[!template text="and I saw that BinDat was not used very heavily," start="00:13:24.285" video="mainVideo" id=subtitle]]
[[!template text="but some of the main uses" start="00:13:28.765" video="mainVideo" id=subtitle]]
[[!template text="were just to read and write integers." start="00:13:30.365" video="mainVideo" id=subtitle]]
[[!template text="And here you can see a very typical case." start="00:13:33.885" video="mainVideo" id=subtitle]]
[[!template text="This is also coming from WeeChat." start="00:13:37.565" video="mainVideo" id=subtitle]]
[[!template text="We do a bindat-get-field" start="00:13:41.726" video="mainVideo" id=subtitle]]
[[!template text="of the length of some struct we read." start="00:13:43.565" video="mainVideo" id=subtitle]]
[[!template text="Actually, the struct we read is here." start="00:13:48.445" video="mainVideo" id=subtitle]]
[[!template text="It has a single field," start="00:13:50.685" video="mainVideo" id=subtitle]]
[[!template text="because the only thing we want to do" start="00:13:51.647" video="mainVideo" id=subtitle]]
[[!template text="is actually to unpack a 32-bit integer," start="00:13:53.006" video="mainVideo" id=subtitle]]
[[!template text="but the only way we can do that" start="00:13:56.287" video="mainVideo" id=subtitle]]
[[!template text="is by specifying a struct with one field." start="00:13:58.287" video="mainVideo" id=subtitle]]
[[!template text="And so we have to extract this struct of one field," start="00:14:01.647" video="mainVideo" id=subtitle]]
[[!template text="which constructs an alist" start="00:14:04.847" video="mainVideo" id=subtitle]]
[[!template text="containing the actual integer," start="00:14:07.246" video="mainVideo" id=subtitle]]
[[!template text="and then we just use get-field to extract it." start="00:14:09.648" video="mainVideo" id=subtitle]]
[[!template text="So this doesn't seem very elegant" start="00:14:11.887" video="mainVideo" id=subtitle]]
[[!template text="to have to construct an alist" start="00:14:15.007" video="mainVideo" id=subtitle]]
[[!template text="just to then extract the integer from it." start="00:14:16.528" video="mainVideo" id=subtitle]]
[[!template text="Same thing if you try to pack it:" start="00:14:20.368" video="mainVideo" id=subtitle]]
[[!template text="you first have to construct the alist" start="00:14:21.648" video="mainVideo" id=subtitle]]
[[!template text="to pass it to bindat-pack unnecessarily." start="00:14:25.007" video="mainVideo" id=subtitle]]
[[!template text="Another problem that I saw in this case" start="00:14:31.248" video="mainVideo" id=subtitle]]
[[!template text="(it was in the websocket package)" start="00:14:33.248" video="mainVideo" id=subtitle]]
[[!template text="was here, where they actually have a function" start="00:14:35.729" video="mainVideo" id=subtitle]]
[[!template text="where they need to write" start="00:14:39.568" video="mainVideo" id=subtitle]]
[[!template text="an integer of a size that will vary" start="00:14:41.169" video="mainVideo" id=subtitle]]
[[!template text="depending on the circumstances." start="00:14:43.888" video="mainVideo" id=subtitle]]
[[!template text="And so they have to test the value of this integer," start="00:14:45.889" video="mainVideo" id=subtitle]]
[[!template text="and depending on which one it is," start="00:14:49.650" video="mainVideo" id=subtitle]]
[[!template text="they're going to use different types." start="00:14:52.210" video="mainVideo" id=subtitle]]
[[!template text="So here it's a case" start="00:14:54.449" video="mainVideo" id=subtitle]]
[[!template text="where we want to have some kind of way to eval --" start="00:14:56.290" video="mainVideo" id=subtitle]]
[[!template text="to compute the length of the integer --" start="00:14:59.490" video="mainVideo" id=subtitle]]
[[!template text="instead of it being predefined or fixed." start="00:15:02.531" video="mainVideo" id=subtitle]]
[[!template text="So this is one of the cases" start="00:15:08.130" video="mainVideo" id=subtitle]]
[[!template text="where the lack of eval was a problem." start="00:15:10.211" video="mainVideo" id=subtitle]]
[[!template text="And actually in all of websocket," start="00:15:16.531" video="mainVideo" id=subtitle]]
[[!template text="BinDat is only used to pack and unpack integers," start="00:15:20.051" video="mainVideo" id=subtitle]]
[[!template text="even though there are many more opportunities" start="00:15:22.612" video="mainVideo" id=subtitle]]
[[!template text="to use BinDat in there." start="00:15:24.612" video="mainVideo" id=subtitle]]
[[!template text="But it's not very convenient to use BinDat," start="00:15:26.772" video="mainVideo" id=subtitle]]
[[!template text="as it stands, for those other cases." start="00:15:29.331" video="mainVideo" id=subtitle]]
[[!template new="1" text="So what does the new design look like?" start="00:15:35.891" video="mainVideo" id=subtitle]]
[[!template text="Well in the new design, here's the problematic code" start="00:15:39.733" video="mainVideo" id=subtitle]]
[[!template text="for WeeChat." start="00:15:44.132" video="mainVideo" id=subtitle]]
[[!template text="So we basically have the same fields as before," start="00:15:46.373" video="mainVideo" id=subtitle]]
[[!template text="you just see that instead of u32," start="00:15:49.012" video="mainVideo" id=subtitle]]
[[!template text="we now have 'uint 32' separately." start="00:15:50.853" video="mainVideo" id=subtitle]]
[[!template text="The idea is that now this 32" start="00:15:53.733" video="mainVideo" id=subtitle]]
[[!template text="can be an expression you can evaluate," start="00:15:55.332" video="mainVideo" id=subtitle]]
[[!template text="and so the u8 is also replaced by 'uint 8'," start="00:15:59.094" video="mainVideo" id=subtitle]]
[[!template text="and the id type is basically the same as before," start="00:16:04.054" video="mainVideo" id=subtitle]]
[[!template text="and here another difference we see," start="00:16:07.253" video="mainVideo" id=subtitle]]
[[!template text="and the main difference..." start="00:16:08.854" video="mainVideo" id=subtitle]]
[[!template text="Actually, it's the second main difference." start="00:16:11.654" video="mainVideo" id=subtitle]]
[[!template text="The first main difference is that" start="00:16:13.494" video="mainVideo" id=subtitle]]
[[!template text="we don't actually quote this whole thing." start="00:16:15.175" video="mainVideo" id=subtitle]]
[[!template text="Instead, we pass it to the bindat-type macro." start="00:16:18.694" video="mainVideo" id=subtitle]]
[[!template text="So this is a macro" start="00:16:23.095" video="mainVideo" id=subtitle]]
[[!template text="that's going to actually build the type." start="00:16:25.095" video="mainVideo" id=subtitle]]
[[!template text="This is a big difference" start="00:16:27.574" video="mainVideo" id=subtitle]]
[[!template text="in terms of performance also," start="00:16:29.254" video="mainVideo" id=subtitle]]
[[!template text="because by making it a macro," start="00:16:30.535" video="mainVideo" id=subtitle]]
[[!template text="we can pre-compute the code" start="00:16:32.695" video="mainVideo" id=subtitle]]
[[!template text="that's going to pack and unpack this thing," start="00:16:34.296" video="mainVideo" id=subtitle]]
[[!template text="instead of having to interpret it" start="00:16:37.255" video="mainVideo" id=subtitle]]
[[!template text="every time we pack and unpack." start="00:16:38.936" video="mainVideo" id=subtitle]]
[[!template text="So this macro will generate more efficient code" start="00:16:41.096" video="mainVideo" id=subtitle]]
[[!template text="along the way." start="00:16:43.815" video="mainVideo" id=subtitle]]
[[!template text="Also it makes the code that appears in here" start="00:16:45.815" video="mainVideo" id=subtitle]]
[[!template text="visible to the compiler" start="00:16:48.695" video="mainVideo" id=subtitle]]
[[!template text="because we can give an Edebug spec for it." start="00:16:50.297" video="mainVideo" id=subtitle]]
[[!template text="And so here as an argument to vec," start="00:16:54.617" video="mainVideo" id=subtitle]]
[[!template text="instead of having to specify" start="00:16:57.497" video="mainVideo" id=subtitle]]
[[!template text="that this is an evaluated expression," start="00:16:59.016" video="mainVideo" id=subtitle]]
[[!template text="we just write the expression directly," start="00:17:00.937" video="mainVideo" id=subtitle]]
[[!template text="because all the expressions that appear there" start="00:17:02.777" video="mainVideo" id=subtitle]]
[[!template text="will just be evaluated," start="00:17:05.096" video="mainVideo" id=subtitle]]
[[!template text="and we don't need to use the 'struct' variable" start="00:17:07.418" video="mainVideo" id=subtitle]]
[[!template text="and then extract the length field from it." start="00:17:11.418" video="mainVideo" id=subtitle]]
[[!template text="We can just use length as a variable." start="00:17:14.137" video="mainVideo" id=subtitle]]
[[!template text="So this variable 'length' here" start="00:17:16.938" video="mainVideo" id=subtitle]]
[[!template text="will refer to this field here," start="00:17:18.698" video="mainVideo" id=subtitle]]
[[!template text="and then this variable 'id' here" start="00:17:20.778" video="mainVideo" id=subtitle]]
[[!template text="will refer to this field here," start="00:17:23.578" video="mainVideo" id=subtitle]]
[[!template text="and so we can just use the field values" start="00:17:25.898" video="mainVideo" id=subtitle]]
[[!template text="as local variables, which is very natural" start="00:17:27.738" video="mainVideo" id=subtitle]]
[[!template text="and very efficient also," start="00:17:30.459" video="mainVideo" id=subtitle]]
[[!template text="because the code would actually directly do that," start="00:17:31.679" video="mainVideo" id=subtitle]]
[[!template text="and the code that unpacks those data" start="00:17:34.618" video="mainVideo" id=subtitle]]
[[!template text="will just extract an integer" start="00:17:37.899" video="mainVideo" id=subtitle]]
[[!template text="and bind it to the length variable," start="00:17:40.299" video="mainVideo" id=subtitle]]
[[!template text="and so that makes it immediately available there." start="00:17:42.219" video="mainVideo" id=subtitle]]
[[!template new="1" text="Okay, let's see also" start="00:17:47.580" video="mainVideo" id=subtitle]]
[[!template text="what the actual documentation looks like." start="00:17:51.340" video="mainVideo" id=subtitle]]
[[!template text="And so if we look at the doc of BinDat," start="00:17:54.220" video="mainVideo" id=subtitle]]
[[!template text="we see the actual specification of the grammar." start="00:17:57.739" video="mainVideo" id=subtitle]]
[[!template text="And so here we see instead of having" start="00:18:01.181" video="mainVideo" id=subtitle]]
[[!template text="these nine different non-terminals," start="00:18:03.181" video="mainVideo" id=subtitle]]
[[!template text="we basically have two:" start="00:18:06.461" video="mainVideo" id=subtitle]]
[[!template text="we have the non-terminal for TYPE," start="00:18:08.061" video="mainVideo" id=subtitle]]
[[!template text="which can be either a uint, a uintr, or a string," start="00:18:10.781" video="mainVideo" id=subtitle]]
[[!template text="or bits, or fill, or align, or vec," start="00:18:15.021" video="mainVideo" id=subtitle]]
[[!template text="or those various other forms;" start="00:18:17.421" video="mainVideo" id=subtitle]]
[[!template text="or it can be a struct, in which case," start="00:18:19.902" video="mainVideo" id=subtitle]]
[[!template text="in the case of struct," start="00:18:22.621" video="mainVideo" id=subtitle]]
[[!template text="then it will be followed by a sequence --" start="00:18:23.981" video="mainVideo" id=subtitle]]
[[!template text="a list of FIELDs, where each of the FIELDs" start="00:18:27.502" video="mainVideo" id=subtitle]]
[[!template text="is basically a LABEL followed by another TYPE." start="00:18:30.142" video="mainVideo" id=subtitle]]
[[!template text="And so this makes the whole specification" start="00:18:33.902" video="mainVideo" id=subtitle]]
[[!template text="much simpler. We don't have any distinction now" start="00:18:37.343" video="mainVideo" id=subtitle]]
[[!template text="between struct being a special case," start="00:18:39.823" video="mainVideo" id=subtitle]]
[[!template text="as opposed to just the normal types." start="00:18:42.862" video="mainVideo" id=subtitle]]
[[!template text="struct is just now one of the possible types" start="00:18:46.383" video="mainVideo" id=subtitle]]
[[!template text="that can appear here." start="00:18:49.263" video="mainVideo" id=subtitle]]
[[!template text="The other thing is that" start="00:18:52.543" video="mainVideo" id=subtitle]]
[[!template text="the LABEL is always present in the structure," start="00:18:53.263" video="mainVideo" id=subtitle]]
[[!template text="so there's no ambiguity." start="00:18:55.743" video="mainVideo" id=subtitle]]
[[!template text="Also all the above things," start="00:18:58.384" video="mainVideo" id=subtitle]]
[[!template text="like the BITLEN we have here," start="00:19:00.304" video="mainVideo" id=subtitle]]
[[!template text="the LEN we have here," start="00:19:03.103" video="mainVideo" id=subtitle]]
[[!template text="the COUNT for vector we have here," start="00:19:04.384" video="mainVideo" id=subtitle]]
[[!template text="these are all plain Elisp expressions," start="00:19:07.504" video="mainVideo" id=subtitle]]
[[!template text="so they are implicitly evaluated if necessary." start="00:19:10.224" video="mainVideo" id=subtitle]]
[[!template text="If you want them to be constant," start="00:19:13.025" video="mainVideo" id=subtitle]]
[[!template text="and really constant, you can just use quotes," start="00:19:14.705" video="mainVideo" id=subtitle]]
[[!template text="for those rare cases where it's necessary." start="00:19:16.705" video="mainVideo" id=subtitle]]
[[!template text="Another thing is that you can extend it" start="00:19:20.145" video="mainVideo" id=subtitle]]
[[!template text="with with bindat-defmacro." start="00:19:21.905" video="mainVideo" id=subtitle]]
[[!template text="Okay, let's go back here." start="00:19:25.505" video="mainVideo" id=subtitle]]
[[!template new="1" text="So what are the advantages of this approach?" start="00:19:30.226" video="mainVideo" id=subtitle]]
[[!template text="As I said, one of the main advantages" start="00:19:32.706" video="mainVideo" id=subtitle]]
[[!template text="is that we now have support for Edebug." start="00:19:34.625" video="mainVideo" id=subtitle]]
[[!template text="We don't have 'struct', 'repeat', and 'align'" start="00:19:39.346" video="mainVideo" id=subtitle]]
[[!template text="as special cases anymore." start="00:19:41.426" video="mainVideo" id=subtitle]]
[[!template text="These are just normal types." start="00:19:42.946" video="mainVideo" id=subtitle]]
[[!template text="Before, there was uint as type, int as type," start="00:19:44.625" video="mainVideo" id=subtitle]]
[[!template text="and those kinds of things." start="00:19:48.067" video="mainVideo" id=subtitle]]
[[!template text="'struct' and 'repeat' and 'align'" start="00:19:49.267" video="mainVideo" id=subtitle]]
[[!template text="were in a different case." start="00:19:51.110" video="mainVideo" id=subtitle]]
[[!template text="So there were" start="00:19:53.267" video="mainVideo" id=subtitle]]
[[!template text="some subtle differences between those" start="00:19:54.387" video="mainVideo" id=subtitle]]
[[!template text="that completely disappeared." start="00:19:56.787" video="mainVideo" id=subtitle]]
[[!template text="Also in the special cases, there was 'union'," start="00:19:59.027" video="mainVideo" id=subtitle]]
[[!template text="and union now has completely disappeared." start="00:20:02.626" video="mainVideo" id=subtitle]]
[[!template text="We don't need it anymore, because instead," start="00:20:05.027" video="mainVideo" id=subtitle]]
[[!template text="we can actually use code anywhere." start="00:20:07.828" video="mainVideo" id=subtitle]]
[[!template text="That's one of the things I didn't mention here," start="00:20:09.588" video="mainVideo" id=subtitle]]
[[!template text="but in this note here," start="00:20:11.908" video="mainVideo" id=subtitle]]
[[!template text="that's one of the important notes." start="00:20:17.268" video="mainVideo" id=subtitle]]
[[!template text="Not only are BITLEN, LEN, COUNT etc." start="00:20:19.747" video="mainVideo" id=subtitle]]
[[!template text="Elisp expressions," start="00:20:21.987" video="mainVideo" id=subtitle]]
[[!template text="but the type itself -- any type itself --" start="00:20:23.028" video="mainVideo" id=subtitle]]
[[!template text="is basically an expression." start="00:20:26.789" video="mainVideo" id=subtitle]]
[[!template text="And so you can, instead of having 'uint BITLEN'," start="00:20:29.029" video="mainVideo" id=subtitle]]
[[!template text="you can have '(if blah-blah-blah uint string)'," start="00:20:32.709" video="mainVideo" id=subtitle]]
[[!template text="and so you can have a field" start="00:20:36.628" video="mainVideo" id=subtitle]]
[[!template text="that can be either string or an int," start="00:20:38.149" video="mainVideo" id=subtitle]]
[[!template text="depending on some condition." start="00:20:40.549" video="mainVideo" id=subtitle]]
[[!template text="And for that reason we don't need a union." start="00:20:44.790" video="mainVideo" id=subtitle]]
[[!template text="Instead of having a union," start="00:20:46.869" video="mainVideo" id=subtitle]]
[[!template text="we can just have a 'cond' or a 'pcase'" start="00:20:47.910" video="mainVideo" id=subtitle]]
[[!template text="that will return the type we want to use," start="00:20:50.710" video="mainVideo" id=subtitle]]
[[!template text="depending on the context," start="00:20:53.590" video="mainVideo" id=subtitle]]
[[!template text="which will generally depend on some previous field." start="00:20:55.109" video="mainVideo" id=subtitle]]
[[!template text="Also we don't need to use single-field structs" start="00:21:00.951" video="mainVideo" id=subtitle]]
[[!template text="for simple types anymore," start="00:21:03.750" video="mainVideo" id=subtitle]]
[[!template text="because there's no distinction between struct" start="00:21:05.351" video="mainVideo" id=subtitle]]
[[!template text="and other types." start="00:21:09.271" video="mainVideo" id=subtitle]]
[[!template text="So we can pass to bindat-pack and bindat-unpack" start="00:21:11.271" video="mainVideo" id=subtitle]]
[[!template text="a specification which just says &quot ;here's an integer&quot ;" start="00:21:17.191" video="mainVideo" id=subtitle]]
[[!template text="and we'll just pack and unpack the integer." start="00:21:20.952" video="mainVideo" id=subtitle]]
[[!template text="And of course now all the code is exposed," start="00:21:24.392" video="mainVideo" id=subtitle]]
[[!template text="so not only Edebug works, but also Flymake," start="00:21:26.472" video="mainVideo" id=subtitle]]
[[!template text="and the compiler, etc. --" start="00:21:29.192" video="mainVideo" id=subtitle]]
[[!template text="they can complain about it," start="00:21:30.392" video="mainVideo" id=subtitle]]
[[!template text="and give you warnings and errors as we like them." start="00:21:33.111" video="mainVideo" id=subtitle]]
[[!template text="And of course the kittens are much happier." start="00:21:38.872" video="mainVideo" id=subtitle]]
[[!template text="Okay. This is going a bit over time," start="00:21:44.553" video="mainVideo" id=subtitle]]
[[!template text="so let's try to go faster." start="00:21:48.153" video="mainVideo" id=subtitle]]
[[!template new="1" text="Here are some of the new features" start="00:21:51.273" video="mainVideo" id=subtitle]]
[[!template text="that are introduced." start="00:21:53.753" video="mainVideo" id=subtitle]]
[[!template text="I already mentioned briefly" start="00:21:54.794" video="mainVideo" id=subtitle]]
[[!template text="that you can define new types with bindat-defmacro." start="00:21:56.314" video="mainVideo" id=subtitle]]
[[!template text="that's one of the important novelties," start="00:22:00.633" video="mainVideo" id=subtitle]]
[[!template text="and you can extend BinDat with new types this way." start="00:22:04.474" video="mainVideo" id=subtitle]]
[[!template text="The other thing you can do is" start="00:22:08.794" video="mainVideo" id=subtitle]]
[[!template text="you can control how values or packets" start="00:22:10.714" video="mainVideo" id=subtitle]]
[[!template text="are unpacked, and how they are represented." start="00:22:16.234" video="mainVideo" id=subtitle]]
[[!template text="In the old BinDat," start="00:22:20.315" video="mainVideo" id=subtitle]]
[[!template text="the packet is necessarily represented," start="00:22:22.555" video="mainVideo" id=subtitle]]
[[!template text="when you unpack it, as an alist, basically," start="00:22:24.315" video="mainVideo" id=subtitle]]
[[!template text="or a struct becomes an alist," start="00:22:28.635" video="mainVideo" id=subtitle]]
[[!template text="and that's all there is." start="00:22:30.396" video="mainVideo" id=subtitle]]
[[!template text="You don't have any choice about it." start="00:22:31.676" video="mainVideo" id=subtitle]]
[[!template text="With the new system," start="00:22:34.076" video="mainVideo" id=subtitle]]
[[!template text="by default, it also returns just an alist," start="00:22:35.596" video="mainVideo" id=subtitle]]
[[!template text="but you can actually control what it's unpacked as," start="00:22:38.076" video="mainVideo" id=subtitle]]
[[!template text="or what it's packed from, using these keywords." start="00:22:41.916" video="mainVideo" id=subtitle]]
[[!template text="With :unpack-val, you can give an expression" start="00:22:46.396" video="mainVideo" id=subtitle]]
[[!template text="that will construct the unpacked value" start="00:22:49.597" video="mainVideo" id=subtitle]]
[[!template text="from the various fields." start="00:22:53.357" video="mainVideo" id=subtitle]]
[[!template text="And with :pack-val and :pack-var," start="00:22:56.957" video="mainVideo" id=subtitle]]
[[!template text="you can specify how to extract the information" start="00:22:59.197" video="mainVideo" id=subtitle]]
[[!template text="from the unpacked value" start="00:23:02.557" video="mainVideo" id=subtitle]]
[[!template text="to generate the pack value." start="00:23:05.117" video="mainVideo" id=subtitle]]
[[!template new="1" text="So here are some examples." start="00:23:08.078" video="mainVideo" id=subtitle]]
[[!template text="Here's an example taken from osc." start="00:23:12.637" video="mainVideo" id=subtitle]]
[[!template text="osc actually doesn't use BinDat currently," start="00:23:15.358" video="mainVideo" id=subtitle]]
[[!template text="but I have played with it" start="00:23:17.438" video="mainVideo" id=subtitle]]
[[!template text="to see what it would look like" start="00:23:22.479" video="mainVideo" id=subtitle]]
[[!template text="if we were to use BinDat." start="00:23:23.758" video="mainVideo" id=subtitle]]
[[!template text="So here's the definition" start="00:23:26.159" video="mainVideo" id=subtitle]]
[[!template text="of the timetag representation," start="00:23:28.638" video="mainVideo" id=subtitle]]
[[!template text="which represents timestamps in osc." start="00:23:30.638" video="mainVideo" id=subtitle]]
[[!template text="So you would use bindat-type" start="00:23:35.279" video="mainVideo" id=subtitle]]
[[!template text="and then you have here :pack-var" start="00:23:37.998" video="mainVideo" id=subtitle]]
[[!template text="basically gives a name" start="00:23:40.559" video="mainVideo" id=subtitle]]
[[!template text="when we try to pack a timestamp." start="00:23:42.080" video="mainVideo" id=subtitle]]
[[!template text="'time' will be the variable whose name contains" start="00:23:48.559" video="mainVideo" id=subtitle]]
[[!template text="the actual timestamp we will receive." start="00:23:51.520" video="mainVideo" id=subtitle]]
[[!template text="So we want to represent the unpacked value" start="00:23:54.159" video="mainVideo" id=subtitle]]
[[!template text="as a normal Emacs timestamp," start="00:23:57.520" video="mainVideo" id=subtitle]]
[[!template text="and then basically convert from this timestamp" start="00:24:00.240" video="mainVideo" id=subtitle]]
[[!template text="to a string, or from a string to this timestamp." start="00:24:02.480" video="mainVideo" id=subtitle]]
[[!template text="When we receive it, it will be called time," start="00:24:06.401" video="mainVideo" id=subtitle]]
[[!template text="so we can refer to it," start="00:24:10.080" video="mainVideo" id=subtitle]]
[[!template text="and so in order to actually encode it," start="00:24:12.240" video="mainVideo" id=subtitle]]
[[!template text="we basically turn this timestamp into an integer --" start="00:24:15.360" video="mainVideo" id=subtitle]]
[[!template text="that's what this :pack-val does." start="00:24:18.320" video="mainVideo" id=subtitle]]
[[!template text="It says when we try to pack it," start="00:24:20.799" video="mainVideo" id=subtitle]]
[[!template text="here's the the value that we should use." start="00:24:23.442" video="mainVideo" id=subtitle]]
[[!template text="We turn it into an integer," start="00:24:26.082" video="mainVideo" id=subtitle]]
[[!template text="and then this integer is going to be encoded" start="00:24:27.760" video="mainVideo" id=subtitle]]
[[!template text="as a uint 64-bit. So a 64-bit unsigned integer." start="00:24:30.320" video="mainVideo" id=subtitle]]
[[!template text="When we try to unpack the value," start="00:24:36.163" video="mainVideo" id=subtitle]]
[[!template text="this 'ticks' field" start="00:24:38.960" video="mainVideo" id=subtitle]]
[[!template text="will contain an unsigned int of 64 bits." start="00:24:40.720" video="mainVideo" id=subtitle]]
[[!template text="We want to return instead a timestamp --" start="00:24:45.679" video="mainVideo" id=subtitle]]
[[!template text="a time value -- from Emacs." start="00:24:50.559" video="mainVideo" id=subtitle]]
[[!template text="Here we use the representation of time" start="00:24:53.924" video="mainVideo" id=subtitle]]
[[!template text="as a pair of number of ticks" start="00:24:59.363" video="mainVideo" id=subtitle]]
[[!template text="and the corresponding frequency of those ticks." start="00:25:02.799" video="mainVideo" id=subtitle]]
[[!template text="So that's what we do here with :unpack-val," start="00:25:06.720" video="mainVideo" id=subtitle]]
[[!template text="which is construct the cons corresponding to it." start="00:25:09.120" video="mainVideo" id=subtitle]]
[[!template text="With this definition, bindat-pack/unpack" start="00:25:12.004" video="mainVideo" id=subtitle]]
[[!template text="are going to convert to and from" start="00:25:16.400" video="mainVideo" id=subtitle]]
[[!template text="proper time values on one side," start="00:25:19.039" video="mainVideo" id=subtitle]]
[[!template text="and binary strings on the other." start="00:25:21.760" video="mainVideo" id=subtitle]]
[[!template text="Note, of course," start="00:25:26.159" video="mainVideo" id=subtitle]]
[[!template text="that I complained that the old BinDat" start="00:25:27.520" video="mainVideo" id=subtitle]]
[[!template text="had to use single-field structs for simple types," start="00:25:30.320" video="mainVideo" id=subtitle]]
[[!template text="and here, basically," start="00:25:36.080" video="mainVideo" id=subtitle]]
[[!template text="I'm back using single-field structs as well" start="00:25:37.039" video="mainVideo" id=subtitle]]
[[!template text="for this particular case --" start="00:25:39.840" video="mainVideo" id=subtitle]]
[[!template text="actually a reasonably frequent case, to be honest." start="00:25:41.120" video="mainVideo" id=subtitle]]
[[!template text="But at least this is not so problematic," start="00:25:44.640" video="mainVideo" id=subtitle]]
[[!template text="because we actually control what is returned," start="00:25:49.279" video="mainVideo" id=subtitle]]
[[!template text="so even though it's a single-field struct," start="00:25:51.840" video="mainVideo" id=subtitle]]
[[!template text="it's not going to construct an alist" start="00:25:54.159" video="mainVideo" id=subtitle]]
[[!template text="or force you to construct an alist." start="00:25:56.640" video="mainVideo" id=subtitle]]
[[!template text="Instead, it really receives and takes a value" start="00:25:58.320" video="mainVideo" id=subtitle]]
[[!template text="in the ideal representation that we chose." start="00:26:02.720" video="mainVideo" id=subtitle]]
[[!template text="Here we have a more complex example," start="00:26:07.367" video="mainVideo" id=subtitle]]
[[!template text="where the actual type is recursive," start="00:26:10.007" video="mainVideo" id=subtitle]]
[[!template text="because it's representing those &quot ;LEB&quot ;..." start="00:26:12.488" video="mainVideo" id=subtitle]]
[[!template text="I can't remember what &quot ;LEB&quot ; stands for," start="00:26:18.640" video="mainVideo" id=subtitle]]
[[!template text="but it's a representation" start="00:26:20.400" video="mainVideo" id=subtitle]]
[[!template text="for arbitrary length integers," start="00:26:22.559" video="mainVideo" id=subtitle]]
[[!template text="where basically" start="00:26:25.600" video="mainVideo" id=subtitle]]
[[!template text="every byte is either smaller than 128," start="00:26:27.520" video="mainVideo" id=subtitle]]
[[!template text="in which case it's the end of the of the value," start="00:26:33.360" video="mainVideo" id=subtitle]]
[[!template text="or it's a value bigger than 128," start="00:26:36.799" video="mainVideo" id=subtitle]]
[[!template text="in which case there's an extra byte on the end" start="00:26:39.760" video="mainVideo" id=subtitle]]
[[!template text="that's going to continue." start="00:26:42.159" video="mainVideo" id=subtitle]]
[[!template text="Here we see the representation" start="00:26:44.490" video="mainVideo" id=subtitle]]
[[!template text="is basically a structure that starts with a byte," start="00:26:46.640" video="mainVideo" id=subtitle]]
[[!template text="which contains this value," start="00:26:52.240" video="mainVideo" id=subtitle]]
[[!template text="which can be either the last value or not," start="00:26:53.679" video="mainVideo" id=subtitle]]
[[!template text="and the tail, which will either be empty," start="00:26:56.000" video="mainVideo" id=subtitle]]
[[!template text="or contain something else." start="00:26:59.770" video="mainVideo" id=subtitle]]
[[!template text="The empty case is here;" start="00:27:01.279" video="mainVideo" id=subtitle]]
[[!template text="if the head value is smaller than 128," start="00:27:04.000" video="mainVideo" id=subtitle]]
[[!template text="then the type of this tail is going to be (unit 0)," start="00:27:07.039" video="mainVideo" id=subtitle]]
[[!template text="so basically 'unit' is the empty type," start="00:27:11.840" video="mainVideo" id=subtitle]]
[[!template text="and 0 is the value we will receive when we read it." start="00:27:16.492" video="mainVideo" id=subtitle]]
[[!template text="And if not, then it has as type 'loop'," start="00:27:20.880" video="mainVideo" id=subtitle]]
[[!template text="which is the type we're defining," start="00:27:25.520" video="mainVideo" id=subtitle]]
[[!template text="so it's the recursive case," start="00:27:28.240" video="mainVideo" id=subtitle]]
[[!template text="where then the rest of the type is the type itself." start="00:27:30.491" video="mainVideo" id=subtitle]]
[[!template text="And so this lets us pack and unpack." start="00:27:35.132" video="mainVideo" id=subtitle]]
[[!template text="We pass it an arbitrary size integer," start="00:27:37.120" video="mainVideo" id=subtitle]]
[[!template text="and it's going to turn it into" start="00:27:39.600" video="mainVideo" id=subtitle]]
[[!template text="this LEB128 binary representation, and vice versa." start="00:27:42.240" video="mainVideo" id=subtitle]]
[[!template text="I have other examples if you're interested," start="00:27:48.492" video="mainVideo" id=subtitle]]
[[!template text="but anyway, here's the conclusion." start="00:27:52.480" video="mainVideo" id=subtitle]]
[[!template new="1" text="We have a simpler, more flexible," start="00:27:56.094" video="mainVideo" id=subtitle]]
[[!template text="and more powerful BinDat now," start="00:27:58.320" video="mainVideo" id=subtitle]]
[[!template text="which is also significantly faster." start="00:28:01.039" video="mainVideo" id=subtitle]]
[[!template text="And I can't remember the exact speed-up," start="00:28:03.454" video="mainVideo" id=subtitle]]
[[!template text="but it's definitely not a few percents." start="00:28:06.799" video="mainVideo" id=subtitle]]
[[!template text="I vaguely remember about 4x faster in my tests," start="00:28:08.720" video="mainVideo" id=subtitle]]
[[!template text="but it's probably very different in different cases" start="00:28:12.640" video="mainVideo" id=subtitle]]
[[!template text="so it might be just 4x, 2x -- who knows?" start="00:28:16.815" video="mainVideo" id=subtitle]]
[[!template text="Try it for yourself, but I was pretty pleased," start="00:28:20.159" video="mainVideo" id=subtitle]]
[[!template text="because it wasn't the main motivation, so anyway..." start="00:28:23.374" video="mainVideo" id=subtitle]]
[[!template new="1" text="The negatives are here." start="00:28:28.336" video="mainVideo" id=subtitle]]
[[!template text="In the new system, there's this bindat-defmacro" start="00:28:31.135" video="mainVideo" id=subtitle]]
[[!template text="which lets us define, kind of, new types," start="00:28:34.480" video="mainVideo" id=subtitle]]
[[!template text="and bindat-type also lets us define new types," start="00:28:36.720" video="mainVideo" id=subtitle]]
[[!template text="and the distinction between them is a bit subtle;" start="00:28:40.895" video="mainVideo" id=subtitle]]
[[!template text="it kind of depends on..." start="00:28:45.360" video="mainVideo" id=subtitle]]
[[!template text="well it has an impact on efficiency" start="00:28:48.080" video="mainVideo" id=subtitle]]
[[!template text="more than anything, so it's not very satisfactory." start="00:28:50.880" video="mainVideo" id=subtitle]]
[[!template text="There's a bit of redundancy between the two." start="00:28:53.520" video="mainVideo" id=subtitle]]
[[!template text="There is no bit-level control, just as before." start="00:28:56.737" video="mainVideo" id=subtitle]]
[[!template text="We can only manipulate basically bytes." start="00:28:59.039" video="mainVideo" id=subtitle]]
[[!template text="So this is definitely not usable" start="00:29:02.098" video="mainVideo" id=subtitle]]
[[!template text="for a Huffman encoding kind of thing." start="00:29:03.360" video="mainVideo" id=subtitle]]
[[!template text="Also, it's not nearly as flexible" start="00:29:09.058" video="mainVideo" id=subtitle]]
[[!template text="as some of the alternatives." start="00:29:10.880" video="mainVideo" id=subtitle]]
[[!template text="So you know GNU Poke" start="00:29:12.240" video="mainVideo" id=subtitle]]
[[!template text="has been a vague inspiration for this work," start="00:29:13.760" video="mainVideo" id=subtitle]]
[[!template text="and GNU Poke gives you a lot more power" start="00:29:20.018" video="mainVideo" id=subtitle]]
[[!template text="in how to specify the types, etc." start="00:29:22.480" video="mainVideo" id=subtitle]]
[[!template text="And of course one of the main downsides" start="00:29:25.059" video="mainVideo" id=subtitle]]
[[!template text="is that it's still not used very much." start="00:29:26.579" video="mainVideo" id=subtitle]]
[[!template text="Actually the new BinDat" start="00:29:28.018" video="mainVideo" id=subtitle]]
[[!template text="is not used by any package" start="00:29:29.283" video="mainVideo" id=subtitle]]
[[!template text="as far as I know right now," start="00:29:31.039" video="mainVideo" id=subtitle]]
[[!template text="but even the old one is not used very often," start="00:29:33.059" video="mainVideo" id=subtitle]]
[[!template text="so who knows" start="00:29:35.279" video="mainVideo" id=subtitle]]
[[!template text="whether it's actually going to" start="00:29:36.799" video="mainVideo" id=subtitle]]
[[!template text="work very much better or not?" start="00:29:38.799" video="mainVideo" id=subtitle]]
[[!template text="Anyway, this is it for this talk." start="00:29:41.520" video="mainVideo" id=subtitle]]
[[!template text="Thank you very much. Have a nice day." start="00:29:44.399" video="mainVideo" id=subtitle]]
[[!template text="(captions by John Cummings)" start="00:29:46.683" video="mainVideo" id=subtitle]]