path: root/2023/captions/emacsconf-2023-test--what-i-learned-by-writing-test-cases-for-gnu-hyperbole--mats-lidell--answers.vtt

WEBVTT


00:00:08.740 --> 00:00:09.240
[Speaker 0]: Do we have any listeners?

00:00:13.340 --> 00:00:13.840
It's you and I. I have a question.

00:00:16.420 --> 00:00:16.640
How many tests do you have for hyperbole and

00:00:18.800 --> 00:00:19.279
How would you rate the test coverage compared

00:00:21.279 --> 00:00:21.500
to other packages? Well,

00:00:28.279 --> 00:00:28.700
that's a tricky 1. Shall I spell it out loud

00:00:31.100 --> 00:00:31.600
and then maybe type it at the same time?

00:00:36.420 --> 00:00:36.920
So, I believe it's around like more than 300

00:00:43.660 --> 00:00:44.059
test cases now. But I cannot compare the test

00:00:45.220 --> 00:00:45.720
coverage to any other

00:01:00.020 --> 00:01:00.520
other package. Maybe I can type that later.

00:01:01.560 --> 00:01:02.060
What do you say, Badal?

00:01:02.660 --> 00:01:02.900
[Speaker 1]: package. I have no knowledge of any Yeah,

00:01:03.840 --> 00:01:04.239
sure, yeah, that's totally fine.

00:01:05.660 --> 00:01:06.160
Feel free to just answer them with voice.

00:01:08.720 --> 00:01:09.220
[Speaker 0]: Yeah, yeah. There's another question.

00:01:10.520 --> 00:01:10.920
1 small suggestion to me,

00:01:11.440 --> 00:01:11.940
should means optional,

00:01:13.660 --> 00:01:14.160
where shall or must means required.

00:01:15.940 --> 00:01:16.220
Not sure if it is too late to make a major

00:01:17.220 --> 00:01:17.540
grammar change like that.

00:01:18.080 --> 00:01:18.580
Very nice presentation.

00:01:19.840 --> 00:01:20.340
So thanks for presentation,

00:01:24.380 --> 00:01:24.780
but the package ERT, well,

00:01:27.920 --> 00:01:28.080
it's not something that we have come up with.

00:01:28.920 --> 00:01:29.340
It's a standard package.

00:01:32.320 --> 00:01:32.560
So I believe it has been around for a long

00:01:37.760 --> 00:01:38.000
time. So, but please feel free to make

00:01:39.680 --> 00:01:40.180
suggestions and maybe you can,

00:01:43.660 --> 00:01:43.860
you know, like do a copy or like an alias for

00:01:46.080 --> 00:01:46.200
that. If you believe it makes more sense for

00:01:48.080 --> 00:01:48.580
your test cases to have that instead.

00:01:53.540 --> 00:01:53.720
And then we have another question here.

00:01:55.540 --> 00:01:55.680
For your info, you may find this helpful for

00:01:58.780 --> 00:01:59.020
running MX test lint both from a command line

00:02:01.220 --> 00:02:01.720
and from within MX with a transit menu.

00:02:03.600 --> 00:02:04.040
GitHub alpha papa make sure,

00:02:06.760 --> 00:02:07.100
yes. It also works on remote CI.

00:02:08.240 --> 00:02:08.740
Yeah, thank you, Alpha Papa.

00:02:10.580 --> 00:02:11.080
I think I've looked into that,

00:02:13.440 --> 00:02:13.940
but we haven't made any use of that.

00:02:17.920 --> 00:02:18.080
But maybe you'll inspire me to give it

00:02:18.400 --> 00:02:18.900
another look.

00:02:29.260 --> 00:02:29.760
[Speaker 2]: Hey guys.

00:02:34.120 --> 00:02:34.460
[Speaker 0]: I remember, I recognize that voice.

00:02:37.160 --> 00:02:37.660
Hi, Bob. Hey, how are you?

00:02:40.240 --> 00:02:40.580
Congratulations, man. Thanks,

00:02:43.020 --> 00:02:43.320
Hugh. Thank you. I have another question

00:02:45.520 --> 00:02:45.900
here. It is easy to run ad hoc tests inside

00:02:48.400 --> 00:02:48.600
an Emacs session given the command line

00:02:51.180 --> 00:02:51.560
scripts you need to run to get the batch test

00:02:54.960 --> 00:02:55.120
session running? You said it's to run an

00:03:05.680 --> 00:03:05.920
ad-hoc test. I'm not sure I understand that

00:03:14.440 --> 00:03:14.940
question. Yes, please.

00:03:15.660 --> 00:03:16.160
[Speaker 1]: Maybe I can rephrase. Sure.

00:03:19.900 --> 00:03:20.400
So I think what I understand is that since

00:03:22.540 --> 00:03:23.040
you have to use some of these command lines

00:03:25.440 --> 00:03:25.940
scripts to get a batch test session running,

00:03:28.780 --> 00:03:29.180
is it easy to run ad hoc tests in an Emacs

00:03:30.700 --> 00:03:30.900
session or does that, like in your

00:03:32.040 --> 00:03:32.540
experience, has that been difficult?

00:03:36.820 --> 00:03:37.320
[Speaker 0]: Well, from the command line,

00:03:38.660 --> 00:03:38.940
if you look at the command line,

00:03:44.160 --> 00:03:44.340
you'll see that it's only like a few image

00:03:46.480 --> 00:03:46.980
functions to call to get that behavior to run

00:03:55.080 --> 00:03:55.240
the batch tests. So I think we made some

00:03:57.100 --> 00:03:57.600
support function for that in hyperbole.

00:04:02.800 --> 00:04:02.960
So it's not, I don't think it's possible out

00:04:05.540 --> 00:04:06.040
of the box to do it, but it's not complicated

00:04:08.060 --> 00:04:08.560
to do it.

00:04:12.190 --> 00:04:12.340
[Speaker 2]: You can define a test anytime,

00:04:14.780 --> 00:04:15.280
right? Just like a new function.

00:04:18.899 --> 00:04:19.240
So that's ad hoc. You just write your test

00:04:20.019 --> 00:04:20.519
and you can run it.

00:04:22.900 --> 00:04:23.400
[Speaker 0]: Yeah, yeah, I mean, of course,

00:04:25.900 --> 00:04:26.180
but I got the impression it was about running

00:04:28.620 --> 00:04:29.060
all your tests like we did with the command

00:04:35.740 --> 00:04:36.020
line. Well, so the question is more about how

00:04:38.260 --> 00:04:38.600
would you run all your test cases from within

00:04:44.860 --> 00:04:45.140
Emacs? And the easy answer to that is

00:04:48.420 --> 00:04:48.860
actually you load all your test case files,

00:04:51.760 --> 00:04:52.080
and then you run ERT with the T as the test

00:04:53.600 --> 00:04:53.880
selector and then it will run all your test

00:04:53.880 --> 00:04:54.380
cases.

00:05:01.780 --> 00:05:01.960
[Speaker 1]: Right. And I think they have expanded on

00:05:03.180 --> 00:05:03.520
their question a little bit as well,

00:05:04.960 --> 00:05:05.220
clarifying that. In other words,

00:05:07.200 --> 00:05:07.360
can you tweak tests in an Emacs session and

00:05:08.860 --> 00:05:09.360
run them right away? Which I believe,

00:05:11.400 --> 00:05:11.640
if I understand correctly what Bob was

00:05:13.820 --> 00:05:14.320
saying, you can basically define or redefine

00:05:15.920 --> 00:05:16.080
functions on the fly and then have them be

00:05:16.440 --> 00:05:16.940
run, right?

00:05:22.200 --> 00:05:22.360
[Speaker 0]: Yes, yes. You just go into that test case and

00:05:24.120 --> 00:05:24.620
you just change it and you run it again.

00:05:29.060 --> 00:05:29.200
And either you have to sort of load it or you

00:05:31.560 --> 00:05:32.060
can use like the commercial thing I did.

00:05:36.140 --> 00:05:36.340
You use hyperbole and just hit meta return on

00:05:38.560 --> 00:05:38.860
the test case and it will load it and run the

00:05:42.240 --> 00:05:42.360
test case again. So that's of course what you

00:05:44.220 --> 00:05:44.720
normally do when you're defining a test or

00:05:47.440 --> 00:05:47.940
debug a test case or develop a test case.

00:05:49.960 --> 00:05:50.460
Just start with something small,

00:05:52.700 --> 00:05:53.200
just make sure maybe you can prepare the test

00:05:55.320 --> 00:05:55.680
properly and run it again and again and again

00:05:56.720 --> 00:05:57.220
until you're ready with it.

00:05:59.760 --> 00:05:59.960
That's a good point. You can definitely do

00:06:02.800 --> 00:06:03.280
that and that's part of how I normally

00:06:06.420 --> 00:06:06.920
develop the test cases that I mean start with

00:06:09.160 --> 00:06:09.400
something small so I can see that I get there

00:06:12.180 --> 00:06:12.240
maybe the right input in the buffer that I

00:06:14.180 --> 00:06:14.340
want to test on or something and I expand on

00:06:18.160 --> 00:06:18.400
that more and more and add more and more more

00:06:18.460 --> 00:06:18.960
and more more

00:06:31.040 --> 00:06:31.540
[Speaker 2]: tests to it. You might tell them a bit about

00:06:33.280 --> 00:06:33.480
how many test cases you have.

00:06:36.020 --> 00:06:36.440
I guess you commented on that and like what

00:06:40.320 --> 00:06:40.820
happens, you know, with the CICD pipeline,

00:06:43.780 --> 00:06:44.020
every time we commit, you know,

00:06:46.360 --> 00:06:46.560
across all the versions and what you have set

00:06:48.760 --> 00:06:49.040
up there because you know I wish people could

00:06:53.940 --> 00:06:54.140
see it. You can go and check on GitHub and

00:06:57.440 --> 00:06:57.720
you can see the logs right of any of the

00:06:59.760 --> 00:06:59.960
builds and but tell them a bit about that

00:07:01.080 --> 00:07:01.320
Mats because I think that's pretty

00:07:01.320 --> 00:07:01.820
impressive.

00:07:07.280 --> 00:07:07.760
[Speaker 0]: Well, that's part of more the CI,

00:07:11.760 --> 00:07:12.160
CD, part of how we developed this using

00:07:15.460 --> 00:07:15.580
GitHub and workflows that you get out of the

00:07:20.740 --> 00:07:20.900
box from there. So this more than 300 test

00:07:23.440 --> 00:07:23.720
cases on our round for I think 5 different

00:07:26.480 --> 00:07:26.980
versions of Emacs when we do a pull request

00:07:33.900 --> 00:07:34.400
or a commit. So that's a good way to ensure

00:07:38.040 --> 00:07:38.540
that it works from version 27.2

00:07:42.240 --> 00:07:42.740
up to the latest master version because

00:07:45.860 --> 00:07:46.360
there's some changes in Emacs over different

00:07:48.940 --> 00:07:49.340
versions that can affect your functions or

00:07:49.600 --> 00:07:50.100
your code.

00:07:56.580 --> 00:07:56.720
[Speaker 2]: They all run in parallel and so typically in

00:08:00.580 --> 00:08:00.780
under 60 seconds I think you've got all of

00:08:03.960 --> 00:08:04.460
them run so you've got pretty extensive

00:08:08.860 --> 00:08:09.240
testing which does catch interesting bugs

00:08:09.760 --> 00:08:10.260
here and there, right?

00:08:13.320 --> 00:08:13.820
[Speaker 0]: Yes, of course it does.

00:08:18.060 --> 00:08:18.560
I mean, you normally develop with 1 version

00:08:20.280 --> 00:08:20.540
and then you think everything is okay.

00:08:21.720 --> 00:08:21.820
But then when you're tested with the

00:08:23.460 --> 00:08:23.960
different versions, you find out that there

00:08:26.080 --> 00:08:26.580
are some changes and there are things you

00:08:30.140 --> 00:08:30.400
might not sort of keep track of what's

00:08:34.340 --> 00:08:34.640
happening also. So that's a way to get

00:08:38.559 --> 00:08:38.940
noticed that the core developers of Emacs

00:08:41.120 --> 00:08:41.480
have changed something that you sort of based

00:08:44.380 --> 00:08:44.840
your code on. Now I got another question

00:08:47.900 --> 00:08:48.160
here. Did you have to change hyperbole code

00:08:50.580 --> 00:08:50.760
and design to be more readily testable as you

00:08:52.160 --> 00:08:52.660
were increasing your test coverage?

00:08:55.520 --> 00:08:56.020
Well, we haven't done that to a lot,

00:09:00.160 --> 00:09:00.320
to a big degree, although I believe that that

00:09:03.760 --> 00:09:04.260
is an important thing for sort of the future

00:09:06.020 --> 00:09:06.500
to do that because some of the hyperbolic

00:09:08.520 --> 00:09:08.720
functions are very complicated and long and

00:09:10.640 --> 00:09:11.140
that makes testing them rather difficult.

00:09:14.660 --> 00:09:14.900
So, at a few places we have sort of broken up

00:09:17.260 --> 00:09:17.720
functions in smaller pieces so it'd be easier

00:09:20.280 --> 00:09:20.660
to do like unit tests of the different parts

00:09:27.740 --> 00:09:27.980
of it. But there's a lot of more work that

00:09:28.680 --> 00:09:29.180
has to be done there.

00:09:33.820 --> 00:09:34.020
[Speaker 2]: 1 of the nice things is you know the great

00:09:36.760 --> 00:09:36.820
environment in Lisp where we're able to do a

00:09:40.520 --> 00:09:40.900
lot of interactive bottom-up testing before

00:09:42.840 --> 00:09:43.280
we even get to lighting tech pieces.

00:09:48.740 --> 00:09:49.140
So it does tend to be more higher level bugs,

00:09:51.140 --> 00:09:51.640
I think, that get caught in cross-functional

00:09:55.940 --> 00:09:56.100
interaction. We had 1 recently that was an

00:09:58.100 --> 00:09:58.600
Emacs version change. It had been a function

00:10:01.100 --> 00:10:01.600
that had existed for a long time.

00:10:03.340 --> 00:10:03.840
It had an and rest in it,

00:10:05.740 --> 00:10:06.240
in its argument list, so it would assemble

00:10:08.600 --> 00:10:09.100
the list of arguments from individual

00:10:10.320 --> 00:10:10.820
arguments that you would give it,

00:10:13.100 --> 00:10:13.600
and they decided in a recent version,

00:10:15.200 --> 00:10:15.700
I think with Stefan's input,

00:10:19.400 --> 00:10:19.840
to change that to a list and allow the prior

00:10:22.740 --> 00:10:22.900
behavior, but it would issue a warning if you

00:10:23.620 --> 00:10:24.060
use the prior behavior.

00:10:25.560 --> 00:10:25.840
So all of a sudden, the way you were supposed

00:10:27.180 --> 00:10:27.680
to do it became semi-invalid.

00:10:30.440 --> 00:10:30.940
And so we started getting the warning,

00:10:32.760 --> 00:10:33.040
and we've tried to eliminate all those

00:10:35.600 --> 00:10:36.060
warnings in recent hyperbole developments.

00:10:37.120 --> 00:10:37.620
So we're like, what do we do?

00:10:39.020 --> 00:10:39.440
You know, because we wanted to be backward

00:10:42.140 --> 00:10:42.640
compatible to where you couldn't use a list.

00:10:44.620 --> 00:10:45.120
It required you to use individual arguments.

00:10:48.380 --> 00:10:48.560
And now it's sort of requiring you to do

00:10:51.660 --> 00:10:51.820
that. And all of that was caused by the

00:10:52.940 --> 00:10:53.440
automatic testing on it.

00:11:08.680 --> 00:11:08.860
So you said, Max, you were going to tell us

00:11:12.740 --> 00:11:13.220
what you learned. So what are the major

00:11:15.368 --> 00:11:15.396
things that you learned in doing all of this

00:11:15.680 --> 00:11:16.180
work? All of this work?

00:11:26.520 --> 00:11:26.740
[Speaker 0]: Well, I tried to cover some of it in the

00:11:29.380 --> 00:11:29.800
presentation, but as I was going along,

00:11:33.420 --> 00:11:33.920
the presentation became like twice as long as

00:11:36.180 --> 00:11:36.680
fitted into the time we had so I had to cut

00:11:42.380 --> 00:11:42.880
it out. But I think some of the core things

00:11:44.340 --> 00:11:44.840
still is in the presentation.

00:11:49.560 --> 00:11:50.060
From a personal perspective,

00:11:52.440 --> 00:11:52.940
And this might not be hard to realize,

00:11:56.960 --> 00:11:57.460
but forcing yourself to test functions,

00:12:02.900 --> 00:12:03.060
test code really forces you to understand the

00:12:05.080 --> 00:12:05.280
code a little bit better in a way that sort

00:12:07.300 --> 00:12:07.400
of makes it easier than just to read the

00:12:11.460 --> 00:12:11.960
code. I don't know how it is for the rest

00:12:13.780 --> 00:12:13.980
listening to this, but for me it works so

00:12:16.580 --> 00:12:17.080
that if I just read the code then I don't

00:12:20.140 --> 00:12:20.320
sort of become as sharp as I should be but if

00:12:22.500 --> 00:12:22.640
I try to write the test case for it then I

00:12:24.680 --> 00:12:24.880
really need to understand better of all the

00:12:27.660 --> 00:12:28.160
edge cases and all the sort of states and etc

00:12:30.060 --> 00:12:30.320
that is involved and I think that's That's

00:12:33.080 --> 00:12:33.200
what's sort of 1 of the learning things I

00:12:34.960 --> 00:12:35.280
wanted to communicate as well that I don't

00:12:38.940 --> 00:12:39.080
think I covered in detail in the

00:12:41.480 --> 00:12:41.980
presentation. Maybe all this,

00:12:48.060 --> 00:12:48.340
but try it. 1 other sort of more from the fun

00:12:50.740 --> 00:12:51.000
side is that I really think it's fun to write

00:12:55.080 --> 00:12:55.440
the test. So if you haven't tests in your

00:12:58.020 --> 00:12:58.520
package, you should start doing that because

00:13:05.740 --> 00:13:06.080
it is fun. It might feel like some extra

00:13:08.080 --> 00:13:08.580
work, but it really pays off in the long run,

00:13:10.320 --> 00:13:10.760
especially if you have it in like a pipeline

00:13:12.520 --> 00:13:12.980
and where you can run it regularly when you

00:13:13.940 --> 00:13:14.380
do new commits, et cetera.

00:13:16.560 --> 00:13:17.060
So, I mean, that's maybe obvious from,

00:13:19.160 --> 00:13:19.440
if you look from the commercial side or your

00:13:21.080 --> 00:13:21.340
work side to do it like that.

00:13:22.260 --> 00:13:22.660
But even for your hobby project,

00:13:26.260 --> 00:13:26.760
it can be very sort of pay off really well.

00:13:32.900 --> 00:13:33.160
[Speaker 2]: It's worked really well when we're adding new

00:13:35.020 --> 00:13:35.180
functionality or we're changing some of the

00:13:36.560 --> 00:13:37.060
plumbing in the system.

00:13:40.400 --> 00:13:40.580
You know, you go and you do some surgery and

00:13:41.320 --> 00:13:41.820
then you run the tests.

00:13:45.400 --> 00:13:45.900
And sometimes 6 to 10 tests will fail.

00:13:48.260 --> 00:13:48.420
And you find there, you know,

00:13:50.460 --> 00:13:50.660
it tends to be they're all interconnected and

00:13:52.920 --> 00:13:53.320
it leads you back to the single source.

00:13:56.660 --> 00:13:56.980
You fix that and you know it could be an edge

00:14:00.560 --> 00:14:00.760
case and off by 1 or Sometimes it's an

00:14:03.520 --> 00:14:03.800
assumption about the way something is used

00:14:05.980 --> 00:14:06.480
and it's not actually always true.

00:14:09.520 --> 00:14:10.020
And so, Matt's just really good at

00:14:13.540 --> 00:14:14.040
identifying some of those scenarios and

00:14:17.480 --> 00:14:17.980
keeping us honest, I guess I would say.

00:14:22.900 --> 00:14:23.400
So I love, I run it as much as I before,

00:14:26.400 --> 00:14:26.900
you know, even before I commit something.

00:14:29.960 --> 00:14:30.060
So I get to see, you know,

00:14:30.940 --> 00:14:31.440
if anything has progressed.

00:14:39.480 --> 00:14:39.920
So yeah, I really recommend this process to

00:14:42.120 --> 00:14:42.620
people. I haven't seen it done.

00:14:45.720 --> 00:14:46.080
I don't think that, I don't know any other

00:14:47.800 --> 00:14:48.300
package that has done it to this level.

00:14:51.560 --> 00:14:51.820
And it's been working really great for us.

00:14:55.440 --> 00:14:55.640
And I think, well, we'll see too when we

00:14:56.780 --> 00:14:57.280
release to the general public.

00:15:04.380 --> 00:15:04.540
[Speaker 0]: But Bob, also, maybe the test part of

00:15:06.400 --> 00:15:06.560
different packages is not the first thing you

00:15:08.900 --> 00:15:09.100
look at. So I know there are packages that

00:15:10.960 --> 00:15:11.380
have testing, a lot of testing,

00:15:13.860 --> 00:15:14.160
but how much, much testing they have or not,

00:15:16.060 --> 00:15:16.220
I don't know. It's not what you normally look

00:15:17.900 --> 00:15:18.400
into when you look at someone's else code.

00:15:20.600 --> 00:15:20.820
You look maybe on the functionality side but

00:15:22.760 --> 00:15:23.000
not on how they've done the sort of the

00:15:26.540 --> 00:15:26.760
quality side. So there could be other

00:15:28.780 --> 00:15:29.280
packages out there that are well equipped.

00:15:31.800 --> 00:15:32.300
[Speaker 2]: I hope so. I hope so.

00:15:39.860 --> 00:15:40.180
[Speaker 0]: What's the craziest bug you found when

00:15:44.700 --> 00:15:45.200
writing these tests? Well,

00:15:50.760 --> 00:15:50.940
What springs to my mind just now is that we

00:15:52.760 --> 00:15:52.960
were doing some tests or I would do some

00:15:55.920 --> 00:15:56.420
tests for when you narrow,

00:15:57.940 --> 00:15:58.440
what do you say that? When you,

00:16:04.500 --> 00:16:05.000
in outlining, when you sort of compress

00:16:06.480 --> 00:16:06.980
things in an outline, so you just,

00:16:08.540 --> 00:16:09.040
sorry Bob, maybe you have it,

00:16:12.100 --> 00:16:12.600
[Speaker 2]: When you hide text.

00:16:12.740 --> 00:16:13.240
[Speaker 0]: What I'm looking for? Yeah,

00:16:15.580 --> 00:16:15.920
when you hide. So I was doing some cursor

00:16:17.780 --> 00:16:17.980
movement over that. And I always assume that

00:16:22.540 --> 00:16:22.900
if you do like a prefix argument to like a

00:16:23.800 --> 00:16:24.240
simple cursor movement,

00:16:26.420 --> 00:16:26.920
like control F moving 1 character position,

00:16:28.340 --> 00:16:28.840
and you would give it the,

00:16:36.580 --> 00:16:37.080
and then the prefix, like you want to move

00:16:39.140 --> 00:16:39.640
like 2 or 3 positions,

00:16:43.040 --> 00:16:43.140
you would do like control U 3 and then

00:16:44.240 --> 00:16:44.740
control F and you move 3.

00:16:46.560 --> 00:16:46.960
I always assumed that that would be exactly

00:16:49.240 --> 00:16:49.440
the same as if you just hit the key control F

00:16:50.740 --> 00:16:51.240
3 times, but it's not.

00:16:53.160 --> 00:16:53.560
So it's not the bug, it's a feature,

00:16:54.620 --> 00:16:55.080
but that was the craziest thing.

00:16:58.180 --> 00:16:58.360
I spent the night trying to figure out why

00:17:00.720 --> 00:17:01.000
our code was wrong, but It turns out that's

00:17:03.560 --> 00:17:04.060
how Emacs behaves. Try it out yourself.

00:17:07.920 --> 00:17:08.300
Try to move over the 3 dots at the end of

00:17:09.140 --> 00:17:09.640
that and see what happens.

00:17:14.060 --> 00:17:14.240
Do it with cursor hitting the key or using a

00:17:16.260 --> 00:17:16.680
prefix argument and you see it behaves

00:17:18.720 --> 00:17:19.220
differently. That was the craziest thing.

00:17:21.960 --> 00:17:22.339
I think there was some other crazy thing or

00:17:24.280 --> 00:17:24.480
deep learning also, but I can't come up with

00:17:26.599 --> 00:17:26.760
it at the moment. So maybe I can write it in

00:17:27.900 --> 00:17:28.400
the Q&A later.

00:17:31.200 --> 00:17:31.440
[Speaker 1]: I think we're out of time on the stream,

00:17:33.360 --> 00:17:33.600
but people are welcome to join Mats and Bob

00:17:35.280 --> 00:17:35.640
here on BigBlueButton to further discuss

00:17:36.480 --> 00:17:36.980
this. Thank you both.

00:17:38.674 --> 00:17:38.792
[Speaker 0]: Okay, thank you. Thanks,

00:17:46.100 --> 00:17:46.600
Makaay. Thank you. I don't know,

00:17:48.740 --> 00:17:49.240
Is it only me and Bob here?

00:17:50.680 --> 00:17:51.180
So Bob, do you want to say something?

00:17:57.440 --> 00:17:57.940
[Speaker 2]: Well, I think it's been a great day.

00:18:00.720 --> 00:18:01.220
And I'm glad we did this.

00:18:02.280 --> 00:18:02.780
It takes a lot of energy.

00:18:15.140 --> 00:18:15.640
I'm just really excited about the progress

00:18:20.580 --> 00:18:20.740
that this, and we're actually doing a lot of

00:18:23.940 --> 00:18:24.160
QA at work and my professional software work

00:18:28.500 --> 00:18:28.840
and looking at you know how we can do more

00:18:32.980 --> 00:18:33.480
test driven development and so everybody's

00:18:35.980 --> 00:18:36.200
talking about this you know we've got AI over

00:18:37.540 --> 00:18:38.040
here that can generate test cases.

00:18:40.200 --> 00:18:40.700
But, you know, strangely enough,

00:18:43.100 --> 00:18:43.380
with the rapidity of development and web

00:18:46.720 --> 00:18:47.220
applications, I think the level of testing

00:18:50.140 --> 00:18:50.280
has gone down in recent years compared to

00:18:51.500 --> 00:18:51.780
where it used to be, right?

00:18:53.040 --> 00:18:53.540
Because the pace has gone up.

00:18:57.340 --> 00:18:57.840
And so I think it's starting to turn again

00:18:58.740 --> 00:18:59.240
where people are saying,

00:19:01.940 --> 00:19:02.440
we can't just release crap into the

00:19:08.120 --> 00:19:08.620
Webisphere and we have to better ourselves.

00:19:13.620 --> 00:19:13.820
And with all these advanced tool sets that

00:19:16.100 --> 00:19:16.600
you have, that you can do CICD testing,

00:19:19.860 --> 00:19:20.180
you know, I just, I just see it coming

00:19:21.900 --> 00:19:22.100
around, you know, as people develop new

00:19:24.000 --> 00:19:24.160
things. So That's kind of exciting to me

00:19:26.980 --> 00:19:27.480
because I came from a manufacturing culture

00:19:30.300 --> 00:19:30.780
originally where we, our company actually

00:19:33.800 --> 00:19:34.300
started a lot of the manufacturing quality

00:19:37.420 --> 00:19:37.920
efforts that you saw in Japan and elsewhere

00:19:40.600 --> 00:19:40.740
in America for a long time and that was you

00:19:42.040 --> 00:19:42.540
know entirely through testing.

00:19:46.640 --> 00:19:47.020
We used to just build incredible test cases

00:19:49.120 --> 00:19:49.320
because we were combining software with

00:19:51.100 --> 00:19:51.380
hardware. And if, you know,

00:19:53.460 --> 00:19:53.600
the hardware doesn't work and you ship a

00:19:55.080 --> 00:19:55.520
million units, you're,

00:19:57.340 --> 00:19:57.840
you're in trouble. So,

00:20:00.260 --> 00:20:00.760
that was just something we had to do.

00:20:04.280 --> 00:20:04.780
And so it's nice to start to see that curve

00:20:07.020 --> 00:20:07.520
come around. And I think,

00:20:10.380 --> 00:20:10.880
you know, Matt Vance is very modest,

00:20:16.680 --> 00:20:16.920
but I think he's really the 1 that started us

00:20:20.400 --> 00:20:20.580
down this path and really made it into a

00:20:24.620 --> 00:20:24.840
reality. So everybody else just gets to

00:20:25.760 --> 00:20:26.260
benefit from that work.

00:20:27.540 --> 00:20:28.040
So thanks.

00:20:32.760 --> 00:20:33.260
[Speaker 1]: That's awesome.

00:20:39.960 --> 00:20:40.460
[Speaker 0]: Thanks. Okay. Yeah. So if there's nothing

00:20:43.200 --> 00:20:43.520
more here, then maybe we should just close

00:20:45.440 --> 00:20:45.940
this and I go over to write in the etherpad

00:20:47.960 --> 00:20:48.460
the replies we had.

00:20:51.900 --> 00:20:52.120
[Speaker 1]: Right, yeah, I think, let's see,

00:20:53.520 --> 00:20:53.760
I see 1 other person here,

00:20:55.080 --> 00:20:55.580
I believe Ihor just joined us.

00:20:58.780 --> 00:20:59.060
Yeah. Yeah, so if you do want to discuss with

00:21:00.220 --> 00:21:00.480
Mats and Bob, you're welcome to,

00:21:02.200 --> 00:21:02.700
otherwise, yeah, we can close the room now.

00:21:05.800 --> 00:21:06.020
[Speaker 3]: Well, I think I missed most of the talk

00:21:06.900 --> 00:21:07.400
because I had power outage,

00:21:12.180 --> 00:21:12.440
but the part I heard was about the mock

00:21:16.860 --> 00:21:17.220
library. And you mentioned that you don't

00:21:20.200 --> 00:21:20.700
like CL-let, but instead you use mock.

00:21:29.700 --> 00:21:29.800
[Speaker 0]: Yeah, I was more saying that you have to do a

00:21:31.560 --> 00:21:32.040
lot more work when you use the CL letdef.

00:21:34.540 --> 00:21:34.780
It's for more ambitious and maybe more

00:21:37.000 --> 00:21:37.500
complicated cases where you want to really

00:21:38.840 --> 00:21:39.340
make a new implementation,

00:21:41.940 --> 00:21:42.440
test implementation. If you use the mock,

00:21:44.380 --> 00:21:44.880
you get a lot of things out of the box,

00:21:47.440 --> 00:21:47.940
verifying that you actually,

00:21:50.820 --> 00:21:51.040
like the mock was actually called for

00:21:53.320 --> 00:21:53.820
instance, whereas if you do with the CLLatf,

00:21:56.520 --> 00:21:56.780
you would have to take correct track of that

00:22:02.020 --> 00:22:02.520
yourself. And so, so a lot of more work.

00:22:03.760 --> 00:22:04.260
Oh yeah.

00:22:07.940 --> 00:22:08.200
[Speaker 3]: I'm saying that most of the time CLLess is

00:22:09.720 --> 00:22:10.220
used for simple cases actually.

00:22:12.320 --> 00:22:12.820
Because, just for example,

00:22:15.100 --> 00:22:15.600
the function always returns the same.

00:22:17.980 --> 00:22:18.420
And it tends to be simple lambda that ignores

00:22:19.040 --> 00:22:19.540
all the input arguments.

00:22:23.000 --> 00:22:23.480
So that's really trivial most of the time but

00:22:25.520 --> 00:22:25.920
I actually thought the opposite that mock is

00:22:27.640 --> 00:22:28.140
supposed to be used for non-trivial cases.

00:22:32.280 --> 00:22:32.520
[Speaker 0]: Sorry, what was the question?

00:22:35.280 --> 00:22:35.780
Mock was supposed to be used for non-trivial.

00:22:47.680 --> 00:22:48.180
Yeah I mean I don't know how to explain this.

00:22:50.140 --> 00:22:50.640
I mean, CLF can be used for non-trivial

00:22:54.400 --> 00:22:54.840
definitely. You can define then any behavior

00:22:56.180 --> 00:22:56.680
you want. You can write your own function,

00:22:58.440 --> 00:22:58.660
but you need to keep track of whether that

00:22:59.620 --> 00:23:00.100
function is called or not,

00:23:06.260 --> 00:23:06.380
for instance. So you have to make note of

00:23:08.440 --> 00:23:08.940
that the function was called so you can fire

00:23:12.440 --> 00:23:12.800
sort of an error in case your function wasn't

00:23:16.960 --> 00:23:17.440
called because that would be 1 error case.

00:23:20.660 --> 00:23:20.860
[Speaker 3]: So you mean the mock fires an error if the

00:23:22.580 --> 00:23:23.080
mocked function was actually not called?

00:23:30.060 --> 00:23:30.560
[Speaker 0]: Yes, it does. Yes. So if your assumptions,

00:23:33.900 --> 00:23:34.120
you sort of document with the mock also your

00:23:37.080 --> 00:23:37.220
assumptions how your code is going to be

00:23:40.020 --> 00:23:40.380
called. And if those are wrong,

00:23:41.120 --> 00:23:41.540
you will get an error.

00:23:43.680 --> 00:23:44.060
So you would, so if the implementation would

00:23:44.840 --> 00:23:45.100
maybe change, for instance,

00:23:46.640 --> 00:23:47.140
and not call the thing you're mocking,

00:23:50.460 --> 00:23:50.960
then you will notice that.

00:23:53.100 --> 00:23:53.560
But if you see a letdef,

00:23:54.840 --> 00:23:55.040
then you will have to keep track of that

00:23:57.560 --> 00:23:58.060
yourself. Okay, I see.

00:23:58.260 --> 00:23:58.760
I see.

00:24:01.240 --> 00:24:01.740
[Speaker 3]: And you know, our mode also uses a lot of

00:24:09.340 --> 00:24:09.620
test. In our mode, we have a lot of tests

00:24:13.940 --> 00:24:14.440
[Speaker 0]: Ah, okay. Yeah. Yeah. I'm sure I have.

00:24:15.900 --> 00:24:16.400
[Speaker 3]: also. We rely on CLLatF for,

00:24:19.220 --> 00:24:19.720
we don't use third-party libraries at all.

00:24:22.140 --> 00:24:22.640
[Speaker 0]: Oh, you use CLLatF, okay.

00:24:26.680 --> 00:24:27.180
Yeah. Yeah. Yeah. At First I found it very

00:24:29.480 --> 00:24:29.700
powerful to use that, but then I sort of,

00:24:32.120 --> 00:24:32.320
I learned more about how we can use the

00:24:34.340 --> 00:24:34.840
mocking library for what I needed.

00:24:36.900 --> 00:24:37.400
And I prefer that at the moment.

00:24:40.560 --> 00:24:41.060
[Speaker 3]: I see, that is interesting.

00:24:42.500 --> 00:24:42.700
Because I had seen it,

00:24:45.440 --> 00:24:45.600
but I didn't consider that it's gonna be

00:24:46.800 --> 00:24:47.300
useful even in simple cases.

00:24:52.640 --> 00:24:53.140
[Speaker 0]: It has its limitations.

00:24:58.260 --> 00:24:58.760
So it's like life, how you turn depends.

00:25:03.740 --> 00:25:04.020
But maybe I should look more into the org

00:25:05.880 --> 00:25:06.100
mode and the test case to learn more about

00:25:07.480 --> 00:25:07.980
that. So thanks for pointing that out.

00:25:14.620 --> 00:25:15.120
[Speaker 3]: We are trying to cover as much as we can.

00:25:17.520 --> 00:25:17.740
It's almost impossible for org.

00:25:20.500 --> 00:25:21.000
But yeah, we keep adding more tests.

00:25:22.780 --> 00:25:23.280
[Speaker 0]: That's great.

00:25:52.720 --> 00:25:53.200
Someone's typing. I don't know.

00:25:54.340 --> 00:25:54.840
Any more questions? No?

00:26:01.060 --> 00:26:01.560
Okay, then I'll go back and try to document

00:26:05.200 --> 00:26:05.360
this in the etherpad. Thank you everybody for

00:26:08.860 --> 00:26:09.160
[Speaker 1]: Thank you guys. Great work.

00:26:09.400 --> 00:26:09.900
[Speaker 0]: joining. Great. Thank you.

00:26:11.100 --> 00:26:11.600
Take care. Bye-bye.

00:26:15.060 --> 00:26:15.560
[Speaker 1]: Take care. Bye. Silence.