On Thu, Feb 25, 2016 at 8:06 AM, David A Case <david.case.rutgers.edu>
wrote:
> On Wed, Feb 24, 2016, Jason Swails wrote:
> >
> > git submodules can help with that -- that is *precisely* for managing a
> > superstructure that is composed of git repos (so that git knows where to
> go
> > off and clone all of its components). What I'd like to see is each
> > component in a separate repo with only pmemd on our own servers, all
> > enveloped within the Amber-MD GitHub organization. Code reviews, proper
> > issue tracking, ...
>
> Sounds good to me, but how do we get there from where we are? Could we
> generate a submodule framework on github.com/amber-md with what is there
> now,
> then slowly add more programs?
>
We could do that, or we could do some script-based approach that would
populate the tree through git (and/or something along the lines of Tim's
mrgit tool). There are lots of ways we could track projects in separate
repos and bring them together so developers can interact with the whole
suite if they want to. We'd need to experiment to find the best way for
us. (In any case, this is a post-Amber 16 move).
About testing before commits to a master branch: we have never had a set of
> tests for (say) sander that all PASS on more than about one platform. And
> it's not just laziness -- there are some real difficulties involved for
> chaotic MD simulations. Even cpptraj shows a number of test failures on my
> machines, just to illustrate the difficulties involved here. travis-CI is
> not
> a magic bullet.
>
My experience has been that, at release, we have a set of tests that pass
on a wide range of platforms. Mac, Ubuntu, recent CentOS with standard
compilers (GNU and Intel compilers released within the last 5 years).
There are often problems with exotic, poorly-tested configurations (like
GNU 4.1.2, Cray, clang, etc.), but the CI here we only propose to test on
common platforms -- Ubuntu LTS and Macs for the time being (maybe if we can
provision a Jenkins-CI bot with the Intel compilers we can do that as
well). Our test suite passes every test on those platforms each time we
release -- we make sure it does. Travis and Jenkins are not magic bullets,
but it will prevent a majority of the master branch pollution that happens.
As Dan pointed out, the failures in cpptraj arose because OS X was not
tested via CI. Once we added it, we saw the errors and those should now be
fixed (and CI will prevent it from happening again).
Finally, we should keep in mind that there can be a downside to making it
> more
> onerous to complete a commit: people will continue to work in their own
> branches, and we don't get as much effective testing because fewer people
> are
> using the code. Just saying that the test cases pass is generally not
> enough--having the code actually being used a lot in real world
> applications
> is also needed. We've had plenty of cases where code gets committed to
> meet a
> release deadline, and passes tests, but has unidentified flaws or
> limitations
> because it has only been used by one or two people, and on a very limited
> number of applications.
>
I agree, but I think what the GH workflow will do is make it *less*
onerous to make a commit. It would cost nothing to commit and submit a
pull request, since you're not negatively affecting anybody by doing so.
It lives in your own branch, and you will be sure that the tests and
compile works on popular setups before your commit will even be considered
for merging. What we have now is a situation where if the tests fail in a
concerning way, you have to effectively undo your changes to see whether
the failing tests are your fault, or a previous commit that was not
thoroughly-enough tested. In my opinion, and experience, this fosters the
practice of holding on to everything in an outdated branch that "works",
and then making a mad dash to dump it into the tree for the deadline. And
then you have multiple people breaking stuff in different ways at the same
time, and unraveling that mess is time-consuming. We already have a system
where too many people dump everything into the tree right before the
deadline and not enough man-hours are put into testing this code; I have a
hard time believing we could do anything that would make this worse than it
already is.
I've had to set aside a couple weeks of dedicated time to prepare for the
releases the last handful of years, and a lot of that time was spent fixing
problems that CI would catch *easily*. e.g., Amber doesn't compile in
parallel, a test file was not committed, all the tests for a component
failed horrendously, ... it's all happened. They're simple to fix, but
collectively add up to a lot of time I wanted to spend doing other
organizational stuff (like building the upgrade and preparing branches and
update/patching scripts, etc.). We would also turn on the CI for the
release branches, meaning we'd be able to run proposed updates through the
full test suite on popular hardware (far more quality control checking than
we have now for updates, which is almost none).
One thing I'll point out is that the development pace of pytraj, cpptraj,
and ParmEd have jumped significantly since moving to GitHub. The code
reviews also offer a great place to transfer knowledge and for new
contributors to learn from reviews of seasoned ones. I have personally
learned a great deal from the other GH projects I contribute to through the
issue tracking and PR code reviews, and I'm a better coder because of it.
In short, GitHub increases both the safety and "collaborativity" of coding
while making it more fun. And that experience has held for myself, Hai,
and Dan (they can correct me if I've misinterpreted our past conversations).
All the best,
Jason
--
Jason M. Swails
BioMaPS,
Rutgers University
Postdoctoral Researcher
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Thu Feb 25 2016 - 19:00:03 PST
