Hi Dave
What Ross said is absolutely correct. When we were implementing ips we
decided that, to avoid evaluating an if statement continuously inside the
non-bonding loop, we would only support ips=1. Originally we did have an
implementation that supported the extra options, however, this decision
improved the performance significantly and we thought people using ips in
pmemd would benefit more from this than from having the extra flexibility.
As Ross, I also don't think having ips for VdW should be a problem,
specially since vdw decays faster than the electrostatic interactions,
which again was our reasoning for implementing it this way. I agree the
manual does not reflect that information and I will correct it. The code
however does exit with the proper error if ips is set to other than 0 or 1
so users should never be able to run into a problem.
Here I am copying an extract of an email I sent Ross at that time with the
timings for ips vs pme. I ran the bemchmark calculations on Trestles up to
64 processors for JAC using the JAC_production_NVT files and here is a
little table of the Master total CPU time. I also checked ips in pmemd vs
sander (there is a test case in Amber that is used for both). I tested
that my addition of IPS wouldn't affect a regular PME run as well as shows
in columns PME-orig and PME-new.
_______begin message_________
"... I added the changes to pairs_calc_novec.i and pme_direct.fpp to avoid
having a bunch of if's for each pair of atoms..." "The only thing to keep
in mind with this implementation is that we can't have "mixed" situations
in which we only calculate the electrostatics or the vdw with ips, it is
an all or nothing implementation, so if you choose ips you have to go with
both as opposed to how I had it before that you could use ips only for one
of them (it is like that in sander)."
#procs NO-longrange PME-orig PME-new IPS-new
4 409.21 553.72 550.47 475.03
8 219.93 290.28 291.92 250.88
16 127.92 159.21 159.51 144.96
32 75.23 91.02 90.19 79.76
64 73.58 79.80 79.22 73.35
______end message______
Hope this helps! Please do let me know what you think and if you believe
we should revert the implementation allowing for the extra cases. As I
said before the only thing to correct is that the manual should mention,
and I will correct it to do so, that pmemd only supports the most useful
case, ips=1, for performance reasons.
Best,
Romelia
> Hi Dave,
>
>> 1. Thinking behind allowing only IPS=1? IPS=2 would seem to be the
>> "sweet
>> spot", but maybe performance is really not much different between these
>> two.
>> Have people checked on this?
>
> Romelia is in charge of the ips implementation so will be able to answer
> you
> in more detail. I believe the choice of only having ips=1 was to avoid
> having an if statement inside the nonbond loop. It should be pretty easy
> to
> support ips=2 or 3, I think the main issue at the time was just lack of
> resources (= people) to actually test the impact of having that on various
> different architectures, with different compilers etc. I didn't think
> there
> was any down side to having ips for VDW but if you think there is an
> appreciable advantage to having ips=2 we could add that. Certainly the
> individual cutoff for vdw and eel are in the code already so it should be
> possible to have this without too much of a hit.
>
>> 2. How well does it scale, and how much has it been tested? I'm seeing
>> failures on BlueGene at 128 threads (not crazy results, but marked lack
>> of
>> energy conservation in NVE), compared to identical input (JAC
>> benchmark) but
>> using 32 or 64 threads. I'll debug as I have time (although am very
>> busy for
>> the next month), but reports from others on experience at high
>> processor
>> counts would be helpful.
>
> Romelia is going to have to provide the details here. My understanding was
> she'd tested this on large processor counts and looked at scaling etc as
> well as making sure the results matched. It has definitely NOT been tested
> on any bluegene system. Do you know if it shows this same issue on other
> systems? - Is it just isolated to ips? - I don't think anybody has been
> testing high processor counts for pmemd in general for a while. :-S
>
> I note the manual does not list the IPS restrictions for pmemd. :-( I'll
> get
> that updated.
>
> All the best
> Ross
>
> /\
> \/
> |\oss Walker
>
> ---------------------------------------------------------
> | Assistant Research Professor |
> | San Diego Supercomputer Center |
> | Adjunct Assistant Professor |
> | Dept. of Chemistry and Biochemistry |
> | University of California San Diego |
> | NVIDIA Fellow |
> | http://www.rosswalker.co.uk | http://www.wmd-lab.org/ |
> | Tel: +1 858 822 0854 | EMail:- ross.rosswalker.co.uk |
> ---------------------------------------------------------
>
> Note: Electronic Mail is not secure, has no guarantee of delivery, may not
> be read every day, and should not be used for urgent or sensitive issues.
>
>
>
>
> _______________________________________________
> AMBER-Developers mailing list
> AMBER-Developers.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber-developers
>
--
****************************************
Romelia Salomon
Walker Group
398 San Diego Supercomputing Center
UC San Diego
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Thu Jul 19 2012 - 23:30:03 PDT
