No subject

surprising bugs, less time to spend wandering in code to find out which
of your call in your code does something wrong, and so on)

>  In my ideal world, I'd love for there to be unit tests for everything.=
  I'm=20
>just not sure what level we can really achieve.  If we create/enforce dr=
aconian=20
>policies, I think the more likely scenario is people just stop writing n=
ew code,=20
>and not that they write unit tests (and/or they fork the code to a versi=
on such=20
>that they don't have to write unit tests).
>
>  I'm certainly not saying unit tests are a bad thing.  But I think we h=
ave to=20
>keep in mind the people who are likely to write the tests.
>
>  All that said, quick thoughts:
>
>1) Basic function/API tests should be easy to write in most cases (if I =
call=20
>xyz(a,b) does it do the right thing).  Those shouldn't be much an issue.
>
>2) A lot of the code operates on extensive external state (map data, arc=
hetypes,=20
>etc).  So to test the function, you need to have a map set up in the rig=
ht way=20
>(right objects on the map, etc).
>
>  In this case, you want something that makes such a test easy.  What ma=
y be=20
>easiest is in fact just a large set of test maps, and the unit test coul=
d=20
>effectively hard code those, with known coordinates (test object being o=
nly=20
>object at 0,0 and I know I want to insert that at 5,5)
> =20
>
That's the idea. If you want to test map behaviour X (like, let's say,
the 'connected' behaviour in code), you write a map (with a few
connected object), and write a test that check specific object status
(the connection) after loading. Of course, as part of unit testing
framework you need to write some helper methods like
init_test_using_map(path_to_map).

So along with the framework we need to provide a toolkit that will help
writting testcase in specific area (eg, a network toolkit, map based
test toolkit, a server initialisation toolkit and so on, that's the part
that will take much of the time, but it will also be written only once)

>  My thinking here is that if I can write the basic test to reproduce th=
e bug in=20
><10 minutes, I'll probably write the test.  IF the framework is such tha=
t it=20
>will take me an hour, then I probably wouldn't.
> =20
>
If it take an hour to write a testcase for a specific bug that mean
either it took you this time to find in which conditions a bug arise (i
could even argue in some cases here writting a unit test helped us find
those conditions faster) or the framework is so difficult to use that
noone can use it (which is not the case for most framework). basically a
unittest is just this

- init needed server code (with toolkits, take 2 or 3 lines)
- do some methods calls
- check_for_some_condition(boolean condition, char*
error_message_if_condition_not_met)
- do some methods calls
- check_for_some_condition(boolean condition, char*
error_message_if_condition_not_met)
- do some methods calls
- check_for_some_condition(boolean condition, char*
error_message_if_condition_not_met)
- do some methods calls
- check_for_some_condition(boolean condition, char*
error_message_if_condition_not_met)
- ...
- test finished

The main and the fixture creating code can nearly be cut and paste from
any existing unit test.

>3) If the number of tests grow, it is probably desirable to be able to r=
un some=20
>specific subset of tests, eg, I don't want to have to wait 15 minutes fo=
r the=20
>server to run all the tests when I'm trying to fix some specific bug - I=
 just=20
>want it to run my test.  After all is done, then perhaps running all tes=
ts=20
>before commit makes sense.
>
> =20
>
Of course, you should be able to run a specific fixture, not all
fixtures. Here at work, running all unit tests can take a few hours
(well that's mainly because we also use a code test coverage tool which
run all unit tests in debug mode). When we are doing changes, we only
test them on a specific fixture or set of fixture, we do not run all
testcase. Running all test cases is done at night from a cvs checkout an
we get the report online. This way we can keep an eye on what's wrong.

It is a good thing, for example, to run all those tests before a release.

>_______________________________________________
>crossfire mailing list
>crossfire at metalforge.org
>http://mailman.metalforge.org/mailman/listinfo/crossfire
>
> =20
>