[crossfire] Unit tests (was: Code cleanlieness and server stability)

[tchize]

Le Mardi 28 Février 2006 01:29, Alex Schultz a écrit :
>I was reading up a little bit on unit tests, and I was thinking, that it
>might be an interesting idea to integrate some unit testing into the
>server. The difficulty is, it would be difficult to create tests that
>could work in a useful way with something as complex as the crossfire

Not true, the difficult part is the work to do because nothing in server has 
been unit tested yet (ak lot's of unit tests to write). The complexity of 
code is not something that would prevent unit testing. There are 2 purpose of 
unit testing

- Test every piece of code against expected behaviour. Those unit test are 
supposed to be written at the same time piece of code is written (well 
ideally in XP you write the unit test before you write the tested code, this 
way you have a clean api, but i suspect not all programmer are ready work 
this way)
- Regression test. Each time a bug is discovered, a unit test must be written 
to reproduce the bug. When this unit test is written, fix code until unit 
test does not fail anymore. This way, you are sure if someone does something 
that could reactivate the bug, it will be visible at unit testing time.


>The big question is, is this worth it? Well, in my opinion, it would
>take a significant amount of effort to make, and useful results might
>not be seen very often with it, however I feel that it might be good to
>have some sort of 'stress test' like this, pushing game conditions to
>their limits, and use that as yet another way to find some of the more
>subtle bugs as them occur. I think that if this is implemented as a
>plugin of some sort that can be optionally built, it wouldn't cutter the
>server code too much either. This said, I'm not sure its the greatest
>idea, but right now I'm brainstorming a bit and it does seem plausible
>anyways and might have some benefits.

This is very worth it, not to push server to it's limit, but simply, despite 
additionnal work involved by unit tests writting, you get those benefits
- api stability (if a function change behaviour, unit tests success ensure the 
change did not break something else)
- improved server stability (if there are enough unit tests to cover the code, 
potential server instabilities would be highlighted by failing unit tests)
- better documentation. Because unit test describe the expected behaviour of a 
given function, we need to document the expected behaviour of a function to 
write it. Also a testcase is like a demo code telling other programmers how 
to use a given function)

But i must raise my shield against the idea to make a unit testing plugin. A 
unit test is by definition a unit. This mean an execution unit. Unit testing 
is something done at compile time. A unit testing framework gather all the 
unit test, and runs them one after each other. For isolation purposes, each 
test is generally run in it's own process and/or memory address space. After 
each run, it gather informations on success and failures.

This framework http://check.sourceforge.net/ looks very good to me, it has 
automake integration, a clean api and does fork to have each test on it's own 
address space. Unfortuately, it's not possible to run those unit test under 
visual studio (visual studio does not have fork()). But it should be possible 
to run it under cygwin or alike.

List of existing unit testing frameworks:
http://www.xprogramming.com/software.htm

I have a good experience in java unit testing and will be glad to provide help 
on unit test enabling crossfire.

Regards,
Tchize
>
>Alex Schultz
>
>_______________________________________________
>crossfire mailing list
>crossfire at metalforge.org
>http://mailman.metalforge.org/mailman/listinfo/crossfire

-- 
--
Tchize (David Delbecq)
tchize at myrealbox.com
Public PGP KEY FINGERPRINT:
    F4BC EF69 54CC F2B5 4621  8DAF 1C71 8E6B 5436 C17C
Public PGP KEY location:
    http://wwwkeys.pgp.net/pgpnet/wwwkeys.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mailman.metalforge.org/pipermail/crossfire/attachments/20060306/ca928fc1/attachment.pgp