On Sat, Feb 27, 2010 at 2:41 AM, Ben Roberts <roberts.qtp.ufl.edu> wrote:
>
> On 26/02/2010, at 8:05 PM, Gustavo Seabra wrote:
>
>> Hi Ben,
>>
>> On Fri, Feb 26, 2010 at 9:48 PM, Ben Roberts <roberts.qtp.ufl.edu> wrote:
>>>
>>> OK. Steps I've taken so far in my working copy:
>>>
>>> 1. Make sure that any "make" command in a Makefile is actually executed
>>> as
>>> "make -k". This change affects two or three statements that I've found.
>>
>> Great. Is that a change to the Makefile or is it in the script? That
>> alone solves most of the problems. If it works fine, I'd say check
>> that in.
>
> I made a handful of changes to Makefiles (where one Makefile calls another).
> In general, however, I put the "make -k" in the scripts. Changes to
> Makefiles have been uploaded, as have the scripts themselves.
That should work, thanks.
>>
>>> 2. Prepare the aforementioned scripts.
>>
>> Myself, I'd still recommend awk. With awk, we can do more, like
>> finding separators (like the "-------" lines, and then looking for the
>> "PASSED" betwen them, etc. It gives us a lot more flexibility then
>> trying to fix every test. Also, tests that fail with segfaults, for
>> example, don't always print the "POSSIBLE FAILURE" statement, so we
>> need to look for other stuff. Awk allows you to make a easier to use
>> counter, save data about the failed tests, etc. If you want, I can
>> take a look at it later, after you check in what you already have.
>
> That would be appreciated. Since I don't know my way around awk very well, I
> would like to leave it to those (such as yourself) who do.
I'm checking out a new tree now, and will look into that. Cheking out
here in Brazil takes hours, literally :-(
>>> I haven't committed any of these changes yet. Is it appropriate to do so?
>>> I'm not quite sure how much boiler-busting will be involved, or whether
>>> this
>>> change counts as a new and poorly tested feature.
>>
>> Most of the changes don't change anything for other users, except the
>> Makefile changes (if that's how you have done it) that will make sure
>> they don't get the annoying errors they have been getting, so I'd say
>> check them in.
>>
>> This is great stuff Ben, thanks a lot!
>
> No problem. I hope it all works properly.
>
> As an aside, in my personal opinion, it would be helpful if all tests behave
> consistently - so that the same result generates the same report. But even
> apart from the feature freeze, I agree that sorting them out definitely
> falls into the "rainy day project" category. Having said that, enforcing
> consistency across the tests may be easier than trying to write awk commands
> to cover every contingency. What are your thoughts?
I totally agree with you. The only reason I'm voting on the awk script
at this point is that I think this is the "quick-and-dirty" way to do
it now. It would be nicer to have all tests working this consistently,
but I guess we'll have to wait for that rainy day to come... (and here
at least, those days are *not* all that common :-)
Cheers,
Gustavo.
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Sat Feb 27 2010 - 14:30:07 PST
