Hi Ben,
On Fri, Feb 26, 2010 at 9:48 PM, Ben Roberts <roberts.qtp.ufl.edu> wrote:
> OK. Steps I've taken so far in my working copy:
>
> 1. Make sure that any "make" command in a Makefile is actually executed as
> "make -k". This change affects two or three statements that I've found.
Great. Is that a change to the Makefile or is it in the script? That
alone solves most of the problems. If it works fine, I'd say check
that in.
> 2. Prepare the aforementioned scripts. At the moment, I've also created a
> separate "logs" directory, with subdirectories for each test
> (test_at_serial, test_at_parallel, test_amber_serial and
> test_amber_parallel). I figure this approach avoids very long file names in
> the top level directory.
>
> Regarding the counters, I used this approach:
>
> var=`grep TEXT | wc -l`
>
> which assumes that all successes produce the same greppable pattern (even if
> surrounded by other things), and ditto for failures and fatal errors. I'm
> not sure to what extent this assumption is valid; some parts of the test
> suite may need to be standardised. A thankless task if ever there was one,
> though (cf. Dave's remark about the nab tests, most of which don't actually
> report program errors at all). Gustavo: is this better than, worse than, or
> as good as using awk?
Myself, I'd still recommend awk. With awk, we can do more, like
finding separators (like the "-------" lines, and then looking for the
"PASSED" betwen them, etc. It gives us a lot more flexibility then
trying to fix every test. Also, tests that fail with segfaults, for
example, don't always print the "POSSIBLE FAILURE" statement, so we
need to look for other stuff. Awk allows you to make a easier to use
counter, save data about the failed tests, etc. If you want, I can
take a look at it later, after you check in what you already have.
> I haven't committed any of these changes yet. Is it appropriate to do so?
> I'm not quite sure how much boiler-busting will be involved, or whether this
> change counts as a new and poorly tested feature.
Most of the changes don't change anything for other users, except the
Makefile changes (if that's how you have done it) that will make sure
they don't get the annoying errors they have been getting, so I'd say
check them in.
This is great stuff Ben, thanks a lot!
Gustavo.
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Fri Feb 26 2010 - 17:30:05 PST
