Re: [AMBER-Developers] Looking for another volunteer--make a "fast" test suite?

From: Hai Nguyen <nhai.qn.gmail.com>
Date: Thu, 7 Sep 2017 01:00:46 -0400

On Thu, Sep 7, 2017 at 12:32 AM, David Cerutti <dscerutti.gmail.com> wrote:

> This sounds like a great idea to me--the only thing, I think, that could
> prevent serial tests from running properly in parallel is that some of them
> write to the same files. Could we use make dependencies to ensure that the
> tests are all run safely, then just let people do the old make -j8?
>
>
>
I've run my local script with several cores in casegroup and got below
summary. Most of the failures are from FEW test.
Given the number of small failures, we can fix it (hopefully easily).

2098 file comparisons passed

14 file comparisons failed

3 tests experienced errors

Finished testing AmberTools in 41.56 (minutes)


Hai


>
> On Wed, Sep 6, 2017 at 4:46 PM, Hai Nguyen <nhai.qn.gmail.com> wrote:
>
> > On Wed, Sep 6, 2017 at 1:33 PM, David Cerutti <dscerutti.gmail.com>
> wrote:
> >
> > > One other thing we might do is have each of us go into our respective
> > > projects and ensure that the tests are running as efficiently as
> > possible.
> > > For example, rather than 50 steps of MD printing every ten iterations,
> > can
> > > the same quality assurance be got in ten steps printing every two
> > > iterations? For sander/pmemd, startup time is still a significant
> > > overhead, but shave 40% off many of the test cases and that'll take the
> > > edge off the problem. (Another thing to mention is that if your test
> > needs
> > > 50 steps to monitor numbers with four places after the decimal and
> ensure
> > > the code is not subtly corrupted, a more sensitive metric needs to be
> > > devised to get at lower significant figures.) I think that part of the
> > > problem here is like pollution: each test contributing an extra few
> > seconds
> > > goes a long way to making the suite as a whole bloated.
> > >
> > >
> > I am also working on split the serial tests to run them in parallel.
> >
> > Hai
> >
> >
> > > Dave
> > >
> > >
> > > On Wed, Sep 6, 2017 at 12:14 PM, Daniel Roe <daniel.r.roe.gmail.com>
> > > wrote:
> > >
> > > > On Wed, Sep 6, 2017 at 10:30 AM, Jason Swails <
> jason.swails.gmail.com>
> > > > wrote:
> > > > > My suggestion is to move from gitosis to a tool that implements a
> > > > > PR/CI-gating workflow like GitLab (which can be self-hosted).
> > Disable
> > > > > pushing directly to master and make every change pass through a
> gated
> > > > pull
> > > > > request that enforces some level of quality before merging is
> > > permitted.
> > > >
> > > > Yes, let's do this! But only once we've come up with a far more
> > > > compact test suite per DAC's previous request. The full test suite
> can
> > > > still be run nightly.
> > > >
> > > > -Dan
> > > >
> > > > --
> > > > -------------------------
> > > > Daniel R. Roe
> > > > Laboratory of Computational Biology
> > > > National Institutes of Health, NHLBI
> > > > 5635 Fishers Ln, Rm T900
> > > > Rockville MD, 20852
> > > > https://www.lobos.nih.gov/lcb
> > > >
> > > > _______________________________________________
> > > > AMBER-Developers mailing list
> > > > AMBER-Developers.ambermd.org
> > > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > >
> > > _______________________________________________
> > > AMBER-Developers mailing list
> > > AMBER-Developers.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > >
> > _______________________________________________
> > AMBER-Developers mailing list
> > AMBER-Developers.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber-developers
> >
> _______________________________________________
> AMBER-Developers mailing list
> AMBER-Developers.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber-developers
>
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Wed Sep 06 2017 - 22:30:03 PDT
Custom Search