On 10/13/06, Ross Walker <ross.rosswalker.co.uk> wrote:
> Scott's script is nice and we should probably resurrect it and start using
> it on more machines. However, I would like to try setting up a system
> whereby the test suite is run and the results are written to a web page that
> we can all refer to. This way we can easily see what test cases are failing,
> when they started failing etc. I will look into scripting things to do this.
> The problem we have at the moment is that the entire test suite does not run
> without human intervention. I.e. some of the test cases crash and in so
> doing bring down the testing mechanism. For things to work properly we will
> need to get these fixed so that at least 'make test' and 'make
> test.parallel' will run to completion without having to go in and comment
> out certain test cases. Thus even if one of the tests segfaults the testing
> should just report and error and continue to the next test case.
>
Hi,
The way I do here is I have a MPI job script that runs the tests
separately. Instead of just 'make test.sander.MPI', I run each
individual sander.* test. That means having lots of 'make
test.sander.***' in the script. (Note that, for each block, the tests
are still run together.)
This script is useful for me because it gives me a more readable
printout. Also, it doesn't kill the whole test if one part fails, only
that specific module.
If we could use something along these lines, the script could also
easily verify a 'TEST_FAILURES' file for each bloch, and automatically
e-mail the respective author in case any of the tests fail. (Of
course, that would be ONLY for your nightly build, NOT for everyone's
make test)
Just an idea.
Gustavo.
Received on Sun Oct 15 2006 - 06:07:11 PDT
