summaryrefslogtreecommitdiffstats
path: root/2023/info/test-after.md
blob: 0d627cafb832acd2b31aecf86a3d3e2bbe485907 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
<!-- Automatically generated by emacsconf-publish-after-page -->


<a name="test-mainVideo-transcript"></a>
# Transcript


[[!template new="1" text="""Introduction""" start="00:00:03.120" video="mainVideo-test" id="subtitle"]]

[[!template text="""Hi everyone! I'm Mats Lidell.""" start="00:00:03.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""I'm going to talk about my journey""" start="00:00:07.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""writing test cases for GNU Hyperbole""" start="00:00:09.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""and what I learned on the way.""" start="00:00:12.481" video="mainVideo-test" id="subtitle"]]
[[!template text="""So, why write tests for GNU Hyperbole?""" start="00:00:19.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""There is some background.""" start="00:00:24.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""I'm the co-maintainer of GNU Hyperbole""" start="00:00:25.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""together with Bob Weiner. Bob is the author of the package.""" start="00:00:27.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""The package is available through""" start="00:00:33.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""the Emacs package manager and GNU Elpa""" start="00:00:34.681" video="mainVideo-test" id="subtitle"]]
[[!template text="""if you would want to try it out.""" start="00:00:38.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""The package has some age. I think it dates back to""" start="00:00:42.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""a first release around 1993, which is also""" start="00:00:46.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""when I got in contact with the package the first time.""" start="00:00:50.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""I was a user of the package for many years.""" start="00:00:54.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""Later, I became the maintainer of the package for the FSF.""" start="00:00:58.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""That was although I did not have""" start="00:01:03.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""much knowledge of Emacs Lisp,""" start="00:01:04.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""and I still have a lot to learn.""" start="00:01:09.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""A few years ago, we started to work actively on the package,""" start="00:01:12.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""with setting up goals and having meetings.""" start="00:01:15.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""So my starting point is that I had experience""" start="00:01:20.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""with test automation from development""" start="00:01:24.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""in C++, Java and Python""" start="00:01:27.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""using different x-unit frameworks like cppunit, junit.""" start="00:01:30.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""That was in my daytime work where""" start="00:01:37.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""the technique of using pull requests""" start="00:01:40.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""with changes backed up by tests were the daily routine.""" start="00:01:41.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""It was really a requirement for a change to go in""" start="00:01:46.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""to have supporting test cases.""" start="00:01:49.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""I believe, a quite common setup and requirement these days.""" start="00:01:52.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""I also had been an Emacs user for many years,""" start="00:01:58.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""but with focus on being a user.""" start="00:02:02.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""So as I mentioned, I have limited Emacs Lisp knowledge.""" start="00:02:04.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""When we decided to start""" start="00:02:09.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""to work actively on Hyperbole again,""" start="00:02:11.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""it was natural for me to look into""" start="00:02:13.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""raising the quality by adding unit tests.""" start="00:02:15.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""This also goes hand in hand""" start="00:02:18.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""with running these regularly as part of a build process.""" start="00:02:20.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""All in all, following the current best practice""" start="00:02:25.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""of software development.""" start="00:02:28.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""But since Hyperbole had no tests at all,""" start="00:02:31.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""it would not be enough just to add tests""" start="00:02:36.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""for new or changed functionality.""" start="00:02:38.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""We wanted to add it even broader; ideally, everywhere.""" start="00:02:41.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""So work started with adding tests here and there""" start="00:02:44.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""based on our gut feeling where it would be most useful.""" start="00:02:48.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""This work is still ongoing.""" start="00:02:52.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""So this is where my journey starts""" start="00:02:55.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""with much functionality to test,""" start="00:02:58.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""no knowledge of what testing frameworks existed,""" start="00:03:00.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""and not really knowing a lot about Emacs Lisp at all.""" start="00:03:03.360" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""ERT: Emacs Lisp Regression Testing""" start="00:03:11.160" video="mainVideo-test" id="subtitle"]]

[[!template text="""Luckily there is a package for writing tests in Emacs.""" start="00:03:11.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""It is called ERT: Emacs Lisp Regression Testing.""" start="00:03:13.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""It contains both support for defining tests and running them.""" start="00:03:17.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""Defining a test is done with the macro `ert-deftest`.""" start="00:03:20.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""In its simplest form, a test has a name, a doc string, and a body.""" start="00:03:24.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""The doc string is where you typically can give""" start="00:03:28.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""a detailed description of the test""" start="00:03:31.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""and has space for more info""" start="00:03:33.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""than what can be given in the test name.""" start="00:03:35.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""The body is where all the interesting things happen.""" start="00:03:42.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""It is here you prepare the test, run it and verify the outcome.""" start="00:03:45.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""Schematically, it looks like this.""" start="00:03:51.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""You have the ert-deftest, you have the test name,""" start="00:03:54.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""and the doc string, and then the body.""" start="00:04:00.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""It is in the body where everything interesting happens.""" start="00:04:02.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""The test is prepared, the function of the test is executed,""" start="00:04:06.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""and the outcome of the test is evaluated.""" start="00:04:09.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""Did the test succeed or not?""" start="00:04:13.120" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Assertions with `should`""" start="00:04:14.360" video="mainVideo-test" id="subtitle"]]

[[!template text="""The verification of a test is performed with""" start="00:04:14.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""one or more so-called assertions.""" start="00:04:18.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""In ERT, they are implemented""" start="00:04:21.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""with the macro `should`""" start="00:04:25.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""together with a set of related macros.""" start="00:04:26.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""`should` takes a form as argument,""" start="00:04:33.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""and if the form evaluates to nil,""" start="00:04:35.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""the test has failed. So let's look at an example.""" start="00:04:37.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""This simple test verifies that the function `+`""" start="00:04:48.581" video="mainVideo-test" id="subtitle"]]
[[!template text="""can add the numbers 2 and 3 and get the result 5.""" start="00:04:51.920" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Running a test case""" start="00:04:56.920" video="mainVideo-test" id="subtitle"]]

[[!template text="""So now we have defined a test case. How do we run it?""" start="00:04:56.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""The ERT package has the function (or""" start="00:05:01.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""rather convenience alias) `ert`. It takes a test selector.""" start="00:05:03.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""The test name works as a selector for running just one test.""" start="00:05:09.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""So here we have the example. Let's evaluate it.""" start="00:05:19.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""We define it and then we run it using ERT.""" start="00:05:27.901" video="mainVideo-test" id="subtitle"]]
[[!template text="""As you see, we get prompted for a test selector""" start="00:05:34.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""but we only have one test case defined at the moment.""" start="00:05:42.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""It's the example 0. So let's hit RET.""" start="00:05:46.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""As you see here, we get some output""" start="00:05:55.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""describing what we have just done.""" start="00:05:58.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""There is one test case it has passed, zero failed,""" start="00:06:01.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""zero skipped, total 1 of 1 test case""" start="00:06:04.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""and some time stamps for the execution.""" start="00:06:07.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""We also see this green mark here indicating one test case""" start="00:06:14.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""and that it was successful.""" start="00:06:18.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""For inspecting the test, we can hit the letter `l`""" start="00:06:23.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""which shows all the `should` forms""" start="00:06:29.660" video="mainVideo-test" id="subtitle"]]
[[!template text="""that was executed during this test case.""" start="00:06:32.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""So here we see that we have the `should`,""" start="00:06:37.780" video="mainVideo-test" id="subtitle"]]
[[!template text="""one `should` executed, and we see the form equals to 2,""" start="00:06:39.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""and it was 5 equals to 5.""" start="00:06:48.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""So a good example of a successful test case.""" start="00:06:49.800" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Debug a test""" start="00:06:54.560" video="mainVideo-test" id="subtitle"]]

[[!template text="""So now we've seen how we can run a test case.""" start="00:06:54.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""Can we debug it? Yes. For debugging a test case,""" start="00:06:57.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""the `ert-deftest` can be set up using `edebug-defun`,""" start="00:07:03.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""just as a function or macro is set up""" start="00:07:07.940" video="mainVideo-test" id="subtitle"]]
[[!template text="""or instrumented for debugging. So let's try that.""" start="00:07:10.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""So we try `edebug-defun` here.""" start="00:07:18.820" video="mainVideo-test" id="subtitle"]]
[[!template text="""Now it's instrumented for debugging.""" start="00:07:24.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""And we run it, `ert`, and we're inside the debugger,""" start="00:07:28.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""and we can inspect here what's happening.""" start="00:07:35.660" video="mainVideo-test" id="subtitle"]]
[[!template text="""Step through it and yes it succeeded just as before.""" start="00:07:40.680" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Commercial break: Hyperbole""" start="00:07:50.380" video="mainVideo-test" id="subtitle"]]

[[!template text="""It's time for a commercial break!""" start="00:07:50.380" video="mainVideo-test" id="subtitle"]]
[[!template text="""Hyperbole itself can help with running tests""" start="00:07:56.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""and also help with running them in debug mode.""" start="00:08:00.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""That is because hyperbole identifies the `ert-deftest`""" start="00:08:03.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""as an implicit button. An implicit button is basically""" start="00:08:08.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""a string or pattern""" start="00:08:12.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""that Hyperbole has assigned some meaning to.""" start="00:08:13.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""For the string `ert-deftest`, it is to run the test case.""" start="00:08:16.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""You activate the button with the action-key.""" start="00:08:19.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""The standard binding is the middle mouse button,""" start="00:08:24.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""or from the keyboard, M-RET.""" start="00:08:27.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""So let's try that.""" start="00:08:33.041" video="mainVideo-test" id="subtitle"]]
[[!template text="""We move the cursor here and then we type M-RET.""" start="00:08:34.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""And boom, the test case was executed.""" start="00:08:42.220" video="mainVideo-test" id="subtitle"]]
[[!template text="""And to run it in debug mode we type C-u M-RET""" start="00:08:47.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""to get the assist key, and then we're in the debugger.""" start="00:08:54.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""So that's pretty useful and convenient.""" start="00:08:57.720" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Instrument function on the fly""" start="00:09:10.480" video="mainVideo-test" id="subtitle"]]

[[!template text="""A related useful feature here is the step-in functionality""" start="00:09:10.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""bound to the letter i in `debug-mode`.""" start="00:09:13.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""It allows you to step into a function""" start="00:09:16.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""and continue debugging from there.""" start="00:09:18.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""For the cases where your test does not do what you want,""" start="00:09:20.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""looking at what happens in the function of the test""" start="00:09:22.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""can be really useful. Let's try that with another example.""" start="00:09:25.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""So here we have two helper functions, one `f1-add`,""" start="00:09:37.260" video="mainVideo-test" id="subtitle"]]
[[!template text="""that use the built-in `+` function""" start="00:09:43.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""and then we have `my-add` that uses that function.""" start="00:09:47.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""So we're going to test myadd.""" start="00:09:52.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""And then let's run this.""" start="00:09:59.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's run this using hyperbole in debug mode""" start="00:10:02.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""C-u M-RET. We're in the debugger again,""" start="00:10:05.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""and let's step up front to my function under test""" start="00:10:10.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""and then press `i` for getting it instrumented""" start="00:10:15.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""and going into it for debugging.""" start="00:10:19.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""And here we can expect that it's getting""" start="00:10:23.020" video="mainVideo-test" id="subtitle"]]
[[!template text="""the arguments 1 and 3,""" start="00:10:25.140" video="mainVideo-test" id="subtitle"]]
[[!template text="""and it returns the result 4 as expected.""" start="00:10:26.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""And yes, of course, our test case will then succeed.""" start="00:10:31.000" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Mocking""" start="00:10:39.120" video="mainVideo-test" id="subtitle"]]

[[!template text="""The next tool in our toolbox is mocking.""" start="00:10:39.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""Mocking is needed when we want to simulate the response""" start="00:10:41.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""from a function used by the function under test.""" start="00:10:46.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""That is the implementation of the function.""" start="00:10:49.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""This could be for various reasons.""" start="00:10:53.140" video="mainVideo-test" id="subtitle"]]
[[!template text="""One example could be because it would be hard or impossible""" start="00:10:56.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""in the test setup to get the behavior you want to test for,""" start="00:11:00.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""like an external error case.""" start="00:11:04.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""But the mock can also be used to verify""" start="00:11:06.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""that the function is called with a specific argument.""" start="00:11:08.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""We can view it as a way to isolate the function on the test""" start="00:11:11.620" video="mainVideo-test" id="subtitle"]]
[[!template text="""from its dependencies.""" start="00:11:14.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""So in order to test the function in isolation,""" start="00:11:16.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""we need to cut out any dependencies to external behavior.""" start="00:11:18.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""Most obvious would be dependencies to external resources,""" start="00:11:22.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""such as web pages. As an example:""" start="00:11:25.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""Hyperbole contains functionality to link you to""" start="00:11:27.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""social media resources and other resources on the net.""" start="00:11:30.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""Testing that would require the test system to call out""" start="00:11:34.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""to the social media resources""" start="00:11:37.900" video="mainVideo-test" id="subtitle"]]
[[!template text="""and would depend on it being available, etc.""" start="00:11:39.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""Nothing technically stops a test case""" start="00:11:43.540" video="mainVideo-test" id="subtitle"]]
[[!template text="""to depend on the external resources,""" start="00:11:45.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""but would, if nothing else, be flaky or slow.""" start="00:11:47.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""It could be part of an end-to-end suite""" start="00:11:51.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""where we want to test that it works all the way.""" start="00:11:53.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""In this case, we want to look at the isolated case""" start="00:11:57.180" video="mainVideo-test" id="subtitle"]]
[[!template text="""that can be run with no dependency on external resources.""" start="00:11:59.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""What you want to do is to replace the function with a mock""" start="00:12:04.100" video="mainVideo-test" id="subtitle"]]
[[!template text="""that behaves as the real function would do.""" start="00:12:06.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""The package I have found""" start="00:12:10.340" video="mainVideo-test" id="subtitle"]]
[[!template text="""and have used for mocking is `el-mock`.""" start="00:12:11.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""The workhorse in this package is the `with-mock` macro.""" start="00:12:14.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""It looks like this: `with-mock` followed by a body.""" start="00:12:21.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""In the execution of the body, stubs and mocks""" start="00:12:26.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""defined in the body is respected.""" start="00:12:30.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's look at some examples to make that clearer.""" start="00:12:32.900" video="mainVideo-test" id="subtitle"]]
[[!template text="""In this case, we have the macro `with-mock`.""" start="00:12:39.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""It works so that the expression""" start="00:12:42.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""`stub + => 10` is interpreted""" start="00:12:43.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""so that the function `+` will be replaced with the stub.""" start="00:12:48.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""The stub will return 10 regardless how it is called.""" start="00:12:51.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""Note that the stub function""" start="00:12:56.780" video="mainVideo-test" id="subtitle"]]
[[!template text="""does not have to be called at this level""" start="00:12:58.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""but could be called at any level in the call chain.""" start="00:13:00.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""By knowing how the function under test is implemented""" start="00:13:02.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""and how the implementation works,""" start="00:13:07.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""you can find function calls you want to mock""" start="00:13:09.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""to force certain behavior that you want to test,""" start="00:13:11.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""or to avoid calls to external resources, slow calls, etc.""" start="00:13:15.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""Simply isolate the function under test""" start="00:13:19.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""and simulate its environment.""" start="00:13:21.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""Mock is a little bit more sophisticated""" start="00:13:26.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""and depends on the arguments""" start="00:13:28.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""that the mock function is called with.""" start="00:13:30.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""Or more precise, it is checked""" start="00:13:31.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""after the `with-mock` clause""" start="00:13:33.848" video="mainVideo-test" id="subtitle"]]
[[!template text="""that the arguments match the arguments it was called with""" start="00:13:35.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""or even if it was called at all.""" start="00:13:38.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""If it is called with other arguments""" start="00:13:39.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""there will be an error,""" start="00:13:41.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""and if it's not called, it is also an error.""" start="00:13:43.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""So this way, we are sure that the function""" start="00:13:46.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""we were expected to be called actually was called.""" start="00:13:48.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""An important piece of the testing.""" start="00:13:51.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""So we are sure that the mock we have provided""" start="00:13:53.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""actually is triggered by the test case.""" start="00:13:56.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""So here we have an example of `with-mock`""" start="00:14:04.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""where the `f1-add` function is mocked,""" start="00:14:08.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""so that if it's called with 2 and 3 as arguments,""" start="00:14:18.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""it will return 10. Then we have a test case""" start="00:14:22.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""where we try the `my-add` function,""" start="00:14:24.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""as you might remember, and call that with 2 and 3""" start="00:14:28.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""and see that it should also then return 10""" start="00:14:30.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""because it's using `f1-add`.""" start="00:14:32.800" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""cl-letf""" start="00:14:41.240" video="mainVideo-test" id="subtitle"]]

[[!template text="""Moving over to `cl-letf`.""" start="00:14:41.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""In rare occasions, the limitations of `el-mock` means""" start="00:14:44.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""you would want to implement a full-fledged function""" start="00:14:47.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""to be used under test.""" start="00:14:50.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""Then the macro `cl-letf` can be useful.""" start="00:14:52.980" video="mainVideo-test" id="subtitle"]]
[[!template text="""However, you need to handle the case yourself""" start="00:14:55.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""if the function was not called.""" start="00:14:57.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""Looking through the test cases where I have used `cl-letf`,""" start="00:15:00.100" video="mainVideo-test" id="subtitle"]]
[[!template text="""I think most can be implemented using plain mocking.""" start="00:15:03.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""Cases left is where the args to the mock might be different""" start="00:15:06.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""due to environment issues.""" start="00:15:11.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""In that case, a static mock will not work.""" start="00:15:13.740" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Hooks""" start="00:15:24.100" video="mainVideo-test" id="subtitle"]]

[[!template text="""Another trick is that functions that uses hooks.""" start="00:15:24.100" video="mainVideo-test" id="subtitle"]]
[[!template text="""You can overload or replace the hooks to do the testing.""" start="00:15:30.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""So you can use the hook function just to do the verification""" start="00:15:35.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""and not do anything useful in the hook.""" start="00:15:40.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""Also, here you need to be careful""" start="00:15:43.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""to make sure the test handler is called and nothing else.""" start="00:15:45.080" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Side effects and initial buffer state""" start="00:15:55.720" video="mainVideo-test" id="subtitle"]]

[[!template text="""So far we have been talking about testing""" start="00:15:55.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""and what the function returns.""" start="00:15:57.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""In the best of words, we have a pure function""" start="00:15:59.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""that only depends on its arguments""" start="00:16:01.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""and produces no side effects.""" start="00:16:02.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""Many operations produce side effects""" start="00:16:04.940" video="mainVideo-test" id="subtitle"]]
[[!template text="""or operate on the contents of buffers""" start="00:16:06.900" video="mainVideo-test" id="subtitle"]]
[[!template text="""such as writing a message in the message buffer,""" start="00:16:09.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""change the state of a buffer, move point etc.""" start="00:16:12.380" video="mainVideo-test" id="subtitle"]]
[[!template text="""Hyperbole is not an exception. Quite the contrary.""" start="00:16:15.660" video="mainVideo-test" id="subtitle"]]
[[!template text="""Much of the functions creating links""" start="00:16:18.860" video="mainVideo-test" id="subtitle"]]
[[!template text="""are just about updating buffers.""" start="00:16:20.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""This poses a special problem for tests.""" start="00:16:24.421" video="mainVideo-test" id="subtitle"]]
[[!template text="""The test gets longer""" start="00:16:28.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""since you need to create buffers and files,""" start="00:16:29.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""initialize the contents.""" start="00:16:31.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""Verifying the outcome becomes trickier""" start="00:16:33.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""since you need to make sure you look at the right place.""" start="00:16:35.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""At the end of the test, you need to clean up,""" start="00:16:39.020" video="mainVideo-test" id="subtitle"]]
[[!template text="""both for not leaving a lot of garbage""" start="00:16:41.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""in buffers and files around,""" start="00:16:43.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""and even worse, not cause later tests""" start="00:16:45.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""to depend on the leftovers from the other tests.""" start="00:16:48.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""Here are some functions and variables""" start="00:16:50.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""I have found useful for this.""" start="00:16:53.080" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""with-temp-buffer""" start="00:17:05.100" video="mainVideo-test" id="subtitle"]]

[[!template text="""For creating tests: `with-temp-buffer`:""" start="00:17:05.100" video="mainVideo-test" id="subtitle"]]
[[!template text="""it provides you a temp buffer that you visit,""" start="00:17:09.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""and afterwards, there is no need to clean up.""" start="00:17:11.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""This is the first choice if that is all you need.""" start="00:17:13.720" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""make-temp-file""" start="00:17:16.520" video="mainVideo-test" id="subtitle"]]

[[!template text="""`make-temp-file`: If you need a file,""" start="00:17:16.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""this is the function to use.""" start="00:17:20.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""It creates a temp file or a directory.""" start="00:17:21.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""The file can be filled with initial contents.""" start="00:17:24.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""This needs to be cleaned up after a test.""" start="00:17:26.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""Moving on to verifying and debugging:""" start="00:17:31.020" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""buffer-string""" start="00:17:33.288" video="mainVideo-test" id="subtitle"]]

[[!template text="""`buffer-string`: returns the full contents""" start="00:17:33.288" video="mainVideo-test" id="subtitle"]]
[[!template text="""of the buffer as a string.""" start="00:17:38.248" video="mainVideo-test" id="subtitle"]]
[[!template text="""That can sound a bit voluminous,""" start="00:17:39.500" video="mainVideo-test" id="subtitle"]]
[[!template text="""but since tests are normally small, this often works well.""" start="00:17:41.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""I have in particular found good use of comparing""" start="00:17:46.140" video="mainVideo-test" id="subtitle"]]
[[!template text="""the contents of buffers with the empty string.""" start="00:17:48.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""That would give an error, but as we have seen""" start="00:17:50.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""with the output produced by the `should` assertion,""" start="00:17:53.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""this is almost like a print statement""" start="00:17:56.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""and can be compared with the good old technique""" start="00:17:58.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""of debugging with print statements.""" start="00:18:01.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""There might be other ways to do the same""" start="00:18:04.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""as we saw with debugging.""" start="00:18:06.248" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""buffer-name""" start="00:18:09.920" video="mainVideo-test" id="subtitle"]]

[[!template text="""buffer-name: Getting the buffer name is good""" start="00:18:09.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""to verify what buffer we are looking at.""" start="00:18:13.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""I often found it useful to check""" start="00:18:16.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""that my assumptions on what buffer I am acting on""" start="00:18:18.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""is correct by adding `should` clauses""" start="00:18:21.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""in the middle of the test execution""" start="00:18:23.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""or after preparing the test input.""" start="00:18:25.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""Sometimes Emacs can switch buffers in strange ways,""" start="00:18:27.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""maybe because the test case is badly written,""" start="00:18:31.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""and making sure your assumptions are correct""" start="00:18:34.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""is a good sanity check.""" start="00:18:37.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""Even the ert package does""" start="00:18:40.340" video="mainVideo-test" id="subtitle"]]
[[!template text="""some buffer and windows manipulation for its reporting""" start="00:18:42.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""that I have not fully learned how to master,""" start="00:18:44.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""so assertion for checking the sanity of the test is good.""" start="00:18:47.488" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""major-mode""" start="00:18:51.980" video="mainVideo-test" id="subtitle"]]

[[!template text="""Finally, `major-mode`: Verify the buffer has the proper mode.""" start="00:18:51.980" video="mainVideo-test" id="subtitle"]]
[[!template text="""Can also be very useful and is a good sanity check.""" start="00:18:55.680" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""unwind-protect""" start="00:19:02.680" video="mainVideo-test" id="subtitle"]]

[[!template text="""Finally, cleaning up. `unwind-protect`.""" start="00:19:02.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""The tool for cleaning up is the `unwind-protect` form""" start="00:19:06.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""which ensures that the unwind forms""" start="00:19:09.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""always are executed regardless of the outcome of the body.""" start="00:19:12.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""So if your test fails, you are sure the cleanup is executed.""" start="00:19:15.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's look at unwind-protect together with""" start="00:19:20.420" video="mainVideo-test" id="subtitle"]]
[[!template text="""the temporary file example. Many tests look like this.""" start="00:19:22.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""You create some resource, you call `unwind-protect`,""" start="00:19:30.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""you do the test, and then afterwards you do the cleanup.""" start="00:19:35.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""The cleanup for a file and a buffer is so common,""" start="00:19:42.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""so I have created a helper for that.""" start="00:19:46.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""It looks like this.""" start="00:19:51.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""The trick with the `buffer-modified` flag""" start="00:19:56.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""is to avoid getting prompted""" start="00:19:59.180" video="mainVideo-test" id="subtitle"]]
[[!template text="""for killing a buffer that is not saved.""" start="00:20:00.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""The test buffers are often in the state""" start="00:20:03.220" video="mainVideo-test" id="subtitle"]]
[[!template text="""where they have not been saved but modified.""" start="00:20:05.440" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Input, with-simulated-input""" start="00:20:15.100" video="mainVideo-test" id="subtitle"]]

[[!template text="""Another problem for tests are input.""" start="00:20:15.100" video="mainVideo-test" id="subtitle"]]
[[!template text="""In the middle of execution a function""" start="00:20:19.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""might want to have some interaction with the user.""" start="00:20:21.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""Testing this poses a problem, not only in that""" start="00:20:24.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""the input matters, but also as how even to get the test case""" start="00:20:26.960" video="mainVideo-test" id="subtitle"]]
[[!template text="""to recognize the input!?""" start="00:20:31.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""Ideally the tests are run in batch mode,""" start="00:20:34.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""which in some sense means no user interaction.""" start="00:20:36.040" video="mainVideo-test" id="subtitle"]]
[[!template text="""In batch mode, there is no event loop running.""" start="00:20:38.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""Fortunately, there is a package `with-simulated-input`""" start="00:20:43.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""that gets you around these issues.""" start="00:20:47.180" video="mainVideo-test" id="subtitle"]]
[[!template text="""This is a macro that allows us""" start="00:20:53.260" video="mainVideo-test" id="subtitle"]]
[[!template text="""to define a set of characters""" start="00:20:55.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""that will be read by the function under the test,""" start="00:20:57.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""and all of this works in batch mode. It looks like this.""" start="00:20:59.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""We have `with-simulated-input`,""" start="00:21:02.580" video="mainVideo-test" id="subtitle"]]
[[!template text="""and then a string of characters, and then a body.""" start="00:21:04.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""The form takes a string of keys""" start="00:21:09.840" video="mainVideo-test" id="subtitle"]]
[[!template text="""and runs the rest of the body,""" start="00:21:11.648" video="mainVideo-test" id="subtitle"]]
[[!template text="""and if there are input required,""" start="00:21:13.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""it is picked from the string of keys.""" start="00:21:15.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""In our example, the `read-string` call""" start="00:21:18.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""will read up until RET,""" start="00:21:20.422" video="mainVideo-test" id="subtitle"]]
[[!template text="""and then return the characters read.""" start="00:21:21.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""As you see in the example, space needs to be provided""" start="00:21:26.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""by the string SPC, as return by the string RET.""" start="00:21:29.640" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Running all tests""" start="00:21:38.460" video="mainVideo-test" id="subtitle"]]

[[!template text="""So now we have seen ways to create test cases""" start="00:21:38.460" video="mainVideo-test" id="subtitle"]]
[[!template text="""and even make it possible to run some of them""" start="00:21:40.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""that has I/O in batch mode.""" start="00:21:43.220" video="mainVideo-test" id="subtitle"]]
[[!template text="""But the initial goal was to run them all at once.""" start="00:21:44.680" video="mainVideo-test" id="subtitle"]]
[[!template text="""How do you do that?""" start="00:21:47.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's go back to the `ert` command.""" start="00:21:48.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""It prompts for a test selector.""" start="00:21:51.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""If we give it the selector `t`,""" start="00:21:53.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""it will run all tests we have currently defined.""" start="00:21:56.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's try that with the subset of the Hyperbole tests.""" start="00:21:59.260" video="mainVideo-test" id="subtitle"]]
[[!template text="""Here is the test folder in the Hyperbole directory.""" start="00:22:05.780" video="mainVideo-test" id="subtitle"]]
[[!template text="""Let's go up here and load all the demo tests.""" start="00:22:09.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""And then try to run `ert`.""" start="00:22:18.820" video="mainVideo-test" id="subtitle"]]
[[!template text="""Now we see that we have a bunch of test cases.""" start="00:22:21.208" video="mainVideo-test" id="subtitle"]]
[[!template text="""We can all run them individually,""" start="00:22:26.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""but we can run them with `t` instead.""" start="00:22:27.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""We will run them all at once.""" start="00:22:31.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""So now, ert is executing all our test cases.""" start="00:22:35.460" video="mainVideo-test" id="subtitle"]]
[[!template text="""So here we have a nice green display""" start="00:22:51.420" video="mainVideo-test" id="subtitle"]]
[[!template text="""with all the test cases.""" start="00:22:57.080" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Batch mode""" start="00:23:03.220" video="mainVideo-test" id="subtitle"]]

[[!template text="""So that was fine, but we were still running it manually""" start="00:23:03.220" video="mainVideo-test" id="subtitle"]]
[[!template text="""by calling ert. How could we run it from the command line?""" start="00:23:08.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""Ert comes with functions for running it in batch mode.""" start="00:23:17.180" video="mainVideo-test" id="subtitle"]]
[[!template text="""For Hyperbole, we use `make` for repetitive tasks.""" start="00:23:21.500" video="mainVideo-test" id="subtitle"]]
[[!template text="""So we have a make target""" start="00:23:25.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""that uses the ert batch functionality,""" start="00:23:27.120" video="mainVideo-test" id="subtitle"]]
[[!template text="""and this is the line from the Makefile.""" start="00:23:29.280" video="mainVideo-test" id="subtitle"]]
[[!template text="""This is a bit detailed,""" start="00:23:33.260" video="mainVideo-test" id="subtitle"]]
[[!template text="""but you see that we have a part here""" start="00:23:35.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""where we load the test dependencies.""" start="00:23:37.540" video="mainVideo-test" id="subtitle"]]
[[!template text="""For getting the packages""" start="00:23:40.780" video="mainVideo-test" id="subtitle"]]
[[!template text="""such as `el-mock` and `with-simulated-input` etc. loaded.""" start="00:23:43.521" video="mainVideo-test" id="subtitle"]]
[[!template text="""We also have... I also want to point out here the call to""" start="00:23:48.460" video="mainVideo-test" id="subtitle"]]
[[!template text="""or the setting of `auto-save-default` to `nil`""" start="00:23:53.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""to get away with the prompt for excessive backup files""" start="00:23:58.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""that can pile up after running the tests a few times.""" start="00:24:02.440" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Skipping tests""" start="00:24:05.060" video="mainVideo-test" id="subtitle"]]

[[!template text="""Even with the help of simulated input,""" start="00:24:05.060" video="mainVideo-test" id="subtitle"]]
[[!template text="""not all tests can be run in batch mode.""" start="00:24:06.880" video="mainVideo-test" id="subtitle"]]
[[!template text="""They would simply not work there""" start="00:24:08.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""and have to be run in an interactive Emacs""" start="00:24:10.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""with the running event loop.""" start="00:24:12.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""One trick still to be able to use batch mode for automation""" start="00:24:14.180" video="mainVideo-test" id="subtitle"]]
[[!template text="""is to put the guard at the top of each test case""" start="00:24:17.920" video="mainVideo-test" id="subtitle"]]
[[!template text="""as the first thing to be executed,""" start="00:24:20.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""so that it kicks in before anything else and stops Emacs""" start="00:24:22.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""to try to run the test case.""" start="00:24:25.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""Now, it looks like this: `(skip-unless (not noninteractive))`.""" start="00:24:27.200" video="mainVideo-test" id="subtitle"]]
[[!template text="""So when ert sees that the test should be skipped, it skips it""" start="00:24:35.520" video="mainVideo-test" id="subtitle"]]
[[!template text="""and makes a note of that,""" start="00:24:38.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""so you will see how many tests that have been skipped.""" start="00:24:40.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""Too bad. We have a number of test cases defined,""" start="00:24:44.580" video="mainVideo-test" id="subtitle"]]
[[!template text="""and to run them, we need to run them manually. Well sort of.""" start="00:24:47.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""Not being able to run all tests easily""" start="00:24:51.360" video="mainVideo-test" id="subtitle"]]
[[!template text="""is a bit counterproductive""" start="00:24:53.808" video="mainVideo-test" id="subtitle"]]
[[!template text="""since our goal is to run all tests.""" start="00:24:58.420" video="mainVideo-test" id="subtitle"]]
[[!template text="""There is however no ert function to run tests in batch mode""" start="00:25:01.000" video="mainVideo-test" id="subtitle"]]
[[!template text="""with an interactive Emacs.""" start="00:25:04.720" video="mainVideo-test" id="subtitle"]]
[[!template text="""The closest I have got is either""" start="00:25:06.780" video="mainVideo-test" id="subtitle"]]
[[!template text="""to start the Emacs from the command line""" start="00:25:08.480" video="mainVideo-test" id="subtitle"]]
[[!template text="""calling the ert function as we just have seen,""" start="00:25:10.080" video="mainVideo-test" id="subtitle"]]
[[!template text="""and then killing it manually when done;""" start="00:25:12.440" video="mainVideo-test" id="subtitle"]]
[[!template text="""or add a function to extract the contents of the ERT buffer""" start="00:25:14.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""when done and echo it to standard output.""" start="00:25:19.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""This is how it looks in the Makefile""" start="00:25:24.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""to get the behavior of cutting and paste,""" start="00:25:27.801" video="mainVideo-test" id="subtitle"]]
[[!template text="""getting the ERT output into a file""" start="00:25:31.208" video="mainVideo-test" id="subtitle"]]
[[!template text="""so we can then kill Emacs""" start="00:25:34.581" video="mainVideo-test" id="subtitle"]]
[[!template text="""and spit out the content of the ERT buffer.""" start="00:25:36.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""One final word here is that""" start="00:25:44.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""when you run this in a continuous integration pipeline,""" start="00:25:47.740" video="mainVideo-test" id="subtitle"]]
[[!template text="""you might not have a TTY for getting Emacs to start,""" start="00:25:54.560" video="mainVideo-test" id="subtitle"]]
[[!template text="""and that is then another problem""" start="00:25:59.400" video="mainVideo-test" id="subtitle"]]
[[!template text="""with getting the interactive mode.""" start="00:26:03.201" video="mainVideo-test" id="subtitle"]]

[[!template new="1" text="""Conclusion""" start="00:26:08.460" video="mainVideo-test" id="subtitle"]]

[[!template text="""We have reached the end of the talk.""" start="00:26:08.460" video="mainVideo-test" id="subtitle"]]
[[!template text="""If you have any new ideas""" start="00:26:11.121" video="mainVideo-test" id="subtitle"]]
[[!template text="""or have some suggestions for improvements,""" start="00:26:14.160" video="mainVideo-test" id="subtitle"]]
[[!template text="""feel free to reach out""" start="00:26:16.760" video="mainVideo-test" id="subtitle"]]
[[!template text="""because I am still on the learning curve of writing,""" start="00:26:18.240" video="mainVideo-test" id="subtitle"]]
[[!template text="""how to write good test cases.""" start="00:26:21.101" video="mainVideo-test" id="subtitle"]]
[[!template text="""If you look at the test cases we have in Hyperbole""" start="00:26:25.300" video="mainVideo-test" id="subtitle"]]
[[!template text="""and you think they might contradict what I am saying here,""" start="00:26:27.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""it is OK. It is probably right.""" start="00:26:29.800" video="mainVideo-test" id="subtitle"]]
[[!template text="""I have changed the style as I go""" start="00:26:32.580" video="mainVideo-test" id="subtitle"]]
[[!template text="""and we have not yet refactored all tests""" start="00:26:34.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""to benefit from new designs.""" start="00:26:36.640" video="mainVideo-test" id="subtitle"]]
[[!template text="""That is also the beauty of the test case.""" start="00:26:38.580" video="mainVideo-test" id="subtitle"]]
[[!template text="""As long as it serves its purpose, it is not terrible""" start="00:26:40.600" video="mainVideo-test" id="subtitle"]]
[[!template text="""if it is not optimal or not having the best style.""" start="00:26:43.320" video="mainVideo-test" id="subtitle"]]
[[!template text="""And yes, thanks for listening. Bye.""" start="00:26:47.800" video="mainVideo-test" id="subtitle"]]

Questions or comments? Please e-mail [matsl@gnu.org](mailto:matsl@gnu.org?subject=Comment%20for%20EmacsConf%202023%20test%3A%20What%20I%20learned%20by%20writing%20test%20cases%20for%20GNU%20Hyperbole)


<!-- End of emacsconf-publish-after-page -->