>> My solution: Put the make command in a shell script and advise the
>> user to
>> run the shell script rather than "make" directly.
>>
>> Like so:
>>
>> #!/bin/bash
>>
>> date_string=`date %Y%m%d%H%M%S`
>>
>> (make -k -f Makefile_at 2>&1) | tee Makefile_at.serial.$
>> {date_string}.log
>>
>>
>>
>> That way, there's automatically a log file, timestamped so the user
>> can, if
>> he or she so wishes, compare the output of different tests. Also,
>> though I
>> haven't implemented this yet, one could use awk (I guess...) to
>> count the
>> number of passes/failures/errors at the end of the shell script;
>> provided
>> the output strings are consistent enough to be greppable.
>>
>> Ben
>>
>
> That should work as well. The use of that script will also avoid the
> need to modify the Makefile. And, it is indeed possible to use awk to
> count the failures, as long as we have the reports loosely consistent.
>
OK. Steps I've taken so far in my working copy:
1. Make sure that any "make" command in a Makefile is actually
executed as "make -k". This change affects two or three statements
that I've found.
2. Prepare the aforementioned scripts. At the moment, I've also
created a separate "logs" directory, with subdirectories for each test
(test_at_serial, test_at_parallel, test_amber_serial and
test_amber_parallel). I figure this approach avoids very long file
names in the top level directory.
Regarding the counters, I used this approach:
var=`grep TEXT | wc -l`
which assumes that all successes produce the same greppable pattern
(even if surrounded by other things), and ditto for failures and fatal
errors. I'm not sure to what extent this assumption is valid; some
parts of the test suite may need to be standardised. A thankless task
if ever there was one, though (cf. Dave's remark about the nab tests,
most of which don't actually report program errors at all). Gustavo:
is this better than, worse than, or as good as using awk?
I haven't committed any of these changes yet. Is it appropriate to do
so? I'm not quite sure how much boiler-busting will be involved, or
whether this change counts as a new and poorly tested feature.
Ben
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Fri Feb 26 2010 - 17:00:04 PST
