[crossfire] Unit tests

[Tchize]

Alex Schultz a écrit :

>tchize wrote:
>
>  
>
>>But i must raise my shield against the idea to make a unit testing plugin. A 
>>unit test is by definition a unit. This mean an execution unit. Unit testing 
>>is something done at compile time. A unit testing framework gather all the 
>>unit test, and runs them one after each other. For isolation purposes, each 
>>test is generally run in it's own process and/or memory address space. After 
>>each run, it gather informations on success and failures.
>>
>>This framework http://check.sourceforge.net/ looks very good to me, it has 
>>automake integration, a clean api and does fork to have each test on it's own 
>>address space. Unfortuately, it's not possible to run those unit test under 
>>visual studio (visual studio does not have fork()). But it should be possible 
>>to run it under cygwin or alike.
>>
>>List of existing unit testing frameworks:
>>http://www.xprogramming.com/software.htm
>> 
>>
>>    
>>
>Actually, I was looking at those a bit, including that one at 
>http://check.sourceforge.net/ however I think that using one of those 
>frameworks might be somewhat bloated for our needs, and more flexibility 
>would be good. I believe that writing our own minimalistic framework, 
>tuned to crossfire, would be a better solution. Also, I'm not sure how 
>good those frameworks are at doing a core dump in the event that a test 
>segfaults, which given that a significant amount of the bugs that are 
>getting fixes are segfault ones, is an important feature for any test 
>framework that will be used for crossfire.
>  
>
check does fork(), it provide address space isolation, that mean a
testcase doing a segfault will be marked as 'failed' but will not
jeopardize running of other tests. check is just a mather of including a
.m4 and a .a in the project, i don't think it's over bloated

>In terms of isolation, I agree that some is needed. I was thinking that 
>forking for each test (perhaps starting a new server, with some argvs, 
>would be an alternative method for platforms with no support for fork). 
>One idea that Mark Wedel brought up on IRC a few days back, was possibly 
>first running each test in isolation, then afterwards, run them in 
>series without isolation, to make the test more robust.
>  
>
No you must isolate each of your fixture, provide each of them with a
clean environment, this is because cf uses lots of static variables.
Those must be reset before each test. If you have a few testcase that
can be safely run in same address space, they are gathered in a fixture.
A fixture is a setup methode, n testcases and a tearDown() method.

>In terms of as a plugin, what I meant, is that to test, it could be 
>loaded at startup time using some of the plugin code, and essentially 
>overrides the normal server operation and reads a text file defining the 
>tests. I'm not sure that is the best idea, but that was what I was 
>thinking at the time. One flaw with that anyways, is it would be good 
>for defining high level tests (see next paragraph), but it would be 
>rather poor for defining lower level tests of functions.
>
>I think both high and low level tests are important, perhaps some 
>'hybrid' ones too. High level being ones were it loads and runs a map, 
>and checks conditions after a certain number of ticks. Low level being 
>running an isolated server function. Hybrid ones being ones that load a 
>map and check a low level server function on that map.
>  
>
Map loading, network protocol checking and so on are not really problems
with a classical framework. unit testing framework are made easy to use,
are robust and are able, at some level, to provide with clean test
reports. Each project then add it's test toolkits around that to do
general setup operations. Those frameworks are made to be adaptable to
any project, what ever it's specificity. I can't see a specificity in
crossfire that would justify to reinvent the wheel.

>Also, thanks for clearing up a bit of that in the first paragraph of 
>your original reply. I already became aware that some of what I said was 
>not entirely accurate, but what you said clears it up further.
>
>  
>
No problem.

>    --Alex Schultz
>
>_______________________________________________
>crossfire mailing list
>crossfire at metalforge.org
>http://mailman.metalforge.org/mailman/listinfo/crossfire
>
>  
>