DevOps Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

I don't know how to test this

06.23.2011
| 8616 views |
  • submit to reddit
There is a maxim said by Misko Hevery which I share (and probably misquote) here:
The only acceptable excuse for lack of tests is that you don't know how to test: testing is an ability that has to be learnt.

Thus testing ability is just like being able to use version control effectively, or designing a schema and querying it with SQL (possibly more important). In some cases we don't write tests because we don't know how; Misko covers very well the part related to object-oriented design, and I think if you're reading here you are already familiar with the concepts:

  • don't spread new operators in your codebase, but ask for your dependencies to be injected via setters or constructors (Dependency Injection).
  • Avoid static classes and Singletons, which maintain global state and result in action-at-a-distance antipatterns.
  • Follow the law of Demeter instead of walking in the whole object graph in a 2-kilometer radius.

But what about technologies? A (web) application is not made only by in-memory objects. It is made of databases, web services, moving parts written entirely in different languages such as JavaScript...

How do we test these components? Here's what I have learned in the last year by working on applications, reading books and personal experimentation.

Libraries

Stubbing and mocking, and any other Test Double specialization, are your best friends when it comes to use an external library. It's also helpful to create a Facade object (or multiple ones) over the library in order to follow the Only mock types you own principle.

In any case, we are talking about testing your code mostly in isolation from the library, and targeting the integration with it in a few slower and noisier tests.

Frameworks

Framework-based code is somewhat more difficult to test than code that just calls a library, due to the lack of control flow: a framework is supposed to instance and call your classes, so there is nothing to mock and in some cases not even a way to easily instantiate your objects out of the framework.

The Humble Object pattern is a surefire solution to tackle this problem: it consists in transforming the framework-related classes in pure delegation components, leaving all the logic to test in a separate package that you can forge with design freedom.

Object-Relational Mapping

An ORM interfacing with a database is a specialization of the library case, a specialization so common that it has a name: Data Mapper pattern. An ORM implemented as a Data Mapper lets you free to work on in-memory objects and hand them for persistence at the end of a user request. The gate to the database is commonly implemented with the Repository pattern.

CouchApps

Couchapps are JavaScript-based applications which accesses CouchDB directly from the browser (and are served via HTTP from a CouchDB instance as well.)
There are two main components to test here:

  • the CouchDB instance views, which are just JavaScript functions: we can test them by passing in fake-data. For example, you can use a combination of Rhino and Qunit or jsTestDriver (which is my preferred choice) to test them independently. I would prefer to use SpiderMonkey since it's what CouchDB uses, but I still don't know if it's possible or simple enough to run a testing framework in it.
  • The browser's part, which makes use of Ajax and the DOM. See the related mini-chapter of this article.

Dealing with graphics

How to test that this result is correct?Functional and end-to-end testing requires you to specify all input data. In the case of web testing, the entirety of a request and a response; in case of graphic-related tests, giant matrices of pixels.

That's why, when dealing with graphic data such as images or video frames, I use simplified versions of these objects in tests. If I want to test-drive a graphic transformations that just rotate an image by 90°, I may use an image of 2x3 pixels generated in memory. And if I can test rotation and translation separately on the same class, I will be confident that they will work together in most cases.

Thus every time you want to test higher-level logic over a layer of heavy data, like a service or an image manager which composes our image rotator, isolating from the lower layer with Test Doubles or fake data is crucial.

Without xUnit

When there is not an xUnit framework for your language of choice (rarer and rarer) you have to build a simple version of such a tool yourself. If phpt does the job for the whole world of PHP native functions, it's not really complex to build a testing framework.

The important thing for this to be easy is being in an OO language. In Matlab, there is a quasi-official xUnit framework, but working on functions and nested functions instead on of objects. There is nothing that can fix the lack of object-orientation of a language, but many practices, such as the absolute elimination of duplication, will work in every Turing-complete language.

Ajax-intensive apps

A year ago I would have shout out Selenium, the famous tool for end-to-end testing of web applications that drive browsers via clicking and asserting on displayed text and events. But Selenium tests are very brittle and are only adequate for end-to-end tests, not for example in to test-drive our JavaScript code.

Unit testing of JavaScript has always existed (see for example JsUnit and its Jasmine evolution), but it's much easier now with Google's jsTestDriver, which capture browsers and can be run from the command line.

Thus the problem is not the tool anymore: even discounting jsTestDriver, there are many frameworks available on the web. The problem is isolation from global browser state like DOM elements or the creation of XmlHttpRequests, but that is a design problem that I'm facing.

JavaScript makes very easy to quickly create stubs, since it supports the on-the-fly substitution of functions:

oldAjax = jQuery.ajax;
jQuery.ajax = function () {}
// execute your test...
jQuery.ajax = oldAjax;

But composition of objects and functions is possible and preferable, just as in every other OO language.

These are only examples

These are lots of examples, and I hope some will be useful to you. But what I hope sticks today is this: when you avoid writing a test, demand yourself if it is because you really don't know how. Misko tackled this well with OO design problems, but as web developers we use many kinds of technologies and languages: our testing skills must be refined also in relation to them, since we aren't always writing Java (or PHP) in-memory objects that concatenate strings, but also JavaScript classes that start HTTP requests like crazy.

Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Artur Biesiadowski replied on Thu, 2011/06/23 - 8:38am

I still don't know how to test that fire generated my 3d engine particle system looks 'realistic' enough. I'm doing small corrections here and there to enable different effects, but how do I make a test which makes sure that there is no bonfire regression?

And no, making bonfire 2x3 pixels won't work. And it is animated and not deterministic system, so taking screenshots also won't work.

Giorgio Sironi replied on Thu, 2011/06/23 - 8:55am in response to: Artur Biesiadowski

Why do you have non-deterministic unit tests? If you seed the random generator with the same initial data, you can get a deterministic result and have regression (not for test-driving) tests via screenshots|frame comparison.

I think the test-driving part is the real issue.

darryl west replied on Thu, 2011/06/23 - 9:43am

thanks for promoting good programming habits. it's hard to believe, but a sad fact that unit and integration testing are still not in the mainstream, especially in the javascript world.

Ronald Miura replied on Thu, 2011/06/23 - 9:45am

It's just one case that TDD or even unit testing in general is not applicable, or not worth it. Making the fire look good is much more important than making it look exactly as specified. You should test all the internal structure and algorithms, though.

Giorgio Sironi replied on Thu, 2011/06/23 - 11:29am in response to: darryl west

In the world of JavaScript, it's also due to the lack of tools, that are still maturing (try integrating a browser into a CI process:).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.