Since Andrew Moore has already done some great work on building up Koha’s test suite for C4 classes, I thought I’d talk about some of the ways that we as developers can add to his work.  I know in my recent development work, I have been trying to add tests where I can.

There’s a good overview on running the test suite, or specific test cases, on the Koha wiki.  What I’d like to talk about are a couple of reasons why we should be writing test cases whenever possible.

Adding more tests to our test suite can be beneficial in a number of ways.  As a new developer to Koha, writing test cases has really helped me understand what’s going on behind the scenes.  For me, writing the test case before I start working is the best way to figure out what the code is doing now, and also what it should be doing in the case of bug fixes, or what I want it to do in the case of adding new features.  That process, generally called Test Driven Developement, or TDD for short, can be awkward for many developers that have never done it before.  My main point though is not to get everyone drinking the TDD Kool-Aid, so the benefits of TDD can wait.  Though as someone that had to re-learn how to develop using TDD, I can attest to the changes one has to overcome. Test cases can be easily written both before and after code changes, so let’s get back to the topic at hand.

Another benefit of test cases is that it allows for a more rapid refactoring of the code base.  Even in my short time with LibLime and working on Koha, many of us on the Koha IRC or mailing lists have discussed ways to improve the code base, like converting to DBIx, mod_perl, or just adding warnings to classes. The more robust and complete our test cases become, the higher the confidence all of us can have in changing the code by running the test suite (or class) after we make changes.  Part of the problem in making these large-scale changes though, is the concern for what functionality we would break or alter.  Having test cases that validate behavior both before and after changes are made can be a huge benefit to successful deployment.

Of course, one of the problems in writing test cases is the fact that as a developer, often times you write as much (or more) in the test cases than you do in the actual code change itself!  While that is often true, most of the time this can be attributed to initial setup cost as once tests are written, they can be used over and over again.  An example is the work I am doing on enhancing the fines portion of Koha.  As I write test cases, I am also able to use the same test cases I wrote for the C4::Members class in another branch, making C4::Members work with DBIx::Class.  As these test cases get merged into the Koha public repository, others can also use these tests to validate still more changes to C4::Members.  So I spent some time initially doing more code than I planned, but in the long term, the benefits quickly repay the initial cost spent in code and time.

While it is often helpful to still test code changes through the browser, having test cases also allow us as developers to isolate problems faster. If you can verify that methods are inserting/updating records correctly via test cases, you save valuable time on support by then focusing on Javascript or HTML errors.  Code also tends to become more maintainable as ‘do-all’ methods get broken up into smaller methods that are easier to test.  In my past experience, while there can doubt as to the quality of the tests available, or concerns that existing test cases are not reliably structured, there can be no doubt that everyone agreed that the existence of tests provided a more stable end product. With the community’s help, hopefully we can all witness that first-hand here as well.

I recently put into place a few tools that should make automated testing of Koha a bit easier. The first is an installation of smolder that will graphically display the results of the test suite. The other is an addition to the test suite to make it easier to send reports to smolder. I have set up a cronjob to run the test suite twice a day and send the results into smolder. Some people call that a “smokebot,” since it’s an automated way to perform smoke tests.

I’m hopeful that this will help is in a few ways. One is that it may make it easier for us to catch those instances where a new patch will break some existing functionality. If there’s an automated regression test to verify some functionality and it stops passing at some point, we can use the smolder to tell when it stopped working. That may help us track down the cause of the failure.

The other way that this might help is is to increase the number of platforms that koha works on. If we start getting test results from other platforms or architectures, we can become more confident that koha works in other environments.

If you have questions about how to use the test suite in Koha or how to use smolder, please don’t hesitate to ask.