diff options
Diffstat (limited to '2023/talks/test.md')
| -rw-r--r-- | 2023/talks/test.md | 113 |
1 files changed, 113 insertions, 0 deletions
diff --git a/2023/talks/test.md b/2023/talks/test.md new file mode 100644 index 00000000..f704f599 --- /dev/null +++ b/2023/talks/test.md @@ -0,0 +1,113 @@ +[[!meta title="What I learned by writing test cases for GNU Hyperbole"]] +[[!meta copyright="Copyright © 2023 Mats Lidell"]] +[[!inline pages="internal(2023/info/test-nav)" raw="yes"]] + +<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing --> +<!-- You can manually edit this file to update the abstract, add links, etc. ---> + + +# What I learned by writing test cases for GNU Hyperbole +Mats Lidell (he, him, his) - IRC: matsl, @matsl@mastodon.acc.sunet.se, <mailto:matsl@gnu.org> + +[[!inline pages="internal(2023/info/test-before)" raw="yes"]] + +I'm maintaining GNU Hyperbole. I volunteered for that at a time when FSF was asking for one +since it was unmaintained. I did not have much elisp experience but a passion for the +package. Not much happened. + +To my great delight a few years ago the author of Hyperbole Bob Weiner joined the band and +we started together to actively develop Hyperbole again. + +One of my focus areas in that work has been to add test cases. We have now gone from no +tests to over 300 ert tests for the package. This talk is about my test case journey. What +I have learned by doing that. + + +# Discussion + +## Questions and answers + +- Q:How many tests do you have for Hyperbole and how wouild you rate + the test coverage compared to other packages? + - A: + - With all tests including the interactive we have 354 tests. + Havng said that I must point out that the size of the tests + can be very different. I tend to split tests so they are + logically (in some sense) different. So that if a test fails + it will more likely point you to what the error is. This + makes it become more tests. Codewise you could collect + similar tests to one ert-deftest making the name of the test + point out some group or collection of functions, but I + don't do that! + - I have not studied other packages so I don't know how our + test coverage compares to other packages. In fact I don't + know what code coverage we have. That is another thing to + look into. +- Q: One small suggestion, to me 'should' means optional, whereas + 'shall' or 'must' means required. Not sure if it is too late to + make a major grammar change like that :) Very nice presentation. (I + see :)) + - A: The assertions come from the ert package so any changes would + have to be suggested to that. I guess you could make your own + version of the assestions using aliases for should et al. +- Q: FYI, you may find this helpful for running Emacs tests/lints, + both from a command line and from within Emacs with a Transient + menu: <https://github.com/alphapapa/makem.sh> It also works on + remote CI. + - A: Thanks for the suggestion. I did have a look at makem.sh but + a long time ago so I don't remember why we did not try to apply + it. I might give it another look now when I have used plain ert + more. +- Q: Is it easy to run ad hoc tests inside of an Emacs session, given + the command line scripts you need to run to get a batch test session + running? In other words, can you tweak tests in an Emacs session and + run them right away? + - A: + - Yes, in principle you just load your tests and run them all + using `ert` and give it the test selector `t`. That runs + all loaded tests. + - If you want to modify a test you can do that. You change it, + evaluate it, and run it again. Just as you change any + function. +- Q: Did you have to change Hyperbole code and design to be more + readily testable as you were increasing your test coverage? + - A: + - Yes, we have done that to a small extent but we should do + more of that. Some Hyperbole functions are large and by that + complicated to test. Splitting them into smaller logical + parts can make testing easier. + - Also moving code into pure functions and avoid side effects + is a good thing. Pure functions are easier to test. Maybe + haveing the side effects separated out into fewer places. + This has not been applied but is something I have been + thinking about. With side effects I here mean things like + adding or modifying text in buffers. +- Q: What's the craziest bug you found when writing these tests? + - A: This is not a bug but I always assumed giving a prefix + argument to a cursor movement would give the same result as + hitting the key the same amount of times. So like C-u 2 C-f + would be the same as hitting the C-f key twise. It is not! When + moving over a hidden area, the three dots '...' at the end of + folded line in org-mode or outline-mode, you get different + behavior. Trying to write a test case for the kotl-mode and its + folded behavior teached me that. +- Q: Why do you prefer el-mock to mocking using cl-letf. (Question + asked in BBB) + - - With cl-letf you need to keep track if the mocked + functionality is being called or not. The el-mock package + does that for you which is what you normally want. Doing + this with cl-letf means the definition becomes longer and + more complicated. Sort of blurs the picture. el-mock is more + to the point. + - BUT since cl-letf does allow you do define a "new" + function it is more powerful and it can be the only option + in cases where el-mock is too limited. So it is good to know + of this possibility with cl-letf when el-mock does not + provide what you need. + + +[[!inline pages="internal(2023/info/test-after)" raw="yes"]] + +[[!inline pages="internal(2023/info/test-nav)" raw="yes"]] + + |