Got sidetracked, back on this full-time...
So OK, if there are no extra points, I'm doing it in the beginning of the
update kernel.
But if extra points, I need to do it before the OrientForcesKernel?
So, what about Softcore TI? Do I need to do separate corrections for each
TI region? Are we doing separate gradsums there as well? Because I'm
thinking separate update regions too get that playing nice.
The first cut which is good for neither special case is checked into the
branch sl_netfrc. It's 7% faster for JAC NVE, and 1-3% faster for
everything else I benchmarked.
This is still a WIP only for single GPU ATM. I will fix that shortly. I
want to get all the special cases running single GPU before tackling that.
Scott
On Fri, May 22, 2020 at 3:06 PM David Cerutti <dscerutti.gmail.com> wrote:
> I agree with Taisung's idea--doing the netfrc correction immediately after
> (or even in the same kernel as) the PME gradient sum will have it happen
> before any spreading occurs, so any further netfrc accumulation after that
> will be solely a function of 1.) conserved forces that automagically... or
> 2.) fp32 roundoff error, if there is any, in the kOrientForces() kernel.
> (In my new frame types there are some places where I revert to fp32 after
> getting displacements down to the level of local coordinate frames, as many
> more registers may be needed for several of the new frame styles.)
>
> Dave
>
>
> On Fri, May 22, 2020 at 5:41 PM Scott Le Grand <varelse2005.gmail.com>
> wrote:
>
> > Yep. there will be FPRE differences. We'll need to validate. I just don't
> > want to break the science.
> >
> > On Fri, May 22, 2020 at 2:16 PM <taisung.gmail.com> wrote:
> >
> > > Actually, I don't think you need to deal w/ the extra points at all if
> > you
> > > do netfrc right after PME gradsum. Just do the total accumulate and
> get
> > > rid
> > > of the resulting net forces for all atoms (including extra points).
> The
> > > forces on extra points will be transferred to nearby heavy atoms (hence
> > > become zero) in a later stage.
> > >
> > > The results could be tiny different though. But theoretically I
> believe
> > it
> > > is OK.
> > >
> > > Taisung
> > >
> > > -----Original Message-----
> > > From: Scott Le Grand [mailto:varelse2005.gmail.com]
> > > Sent: Friday, May 22, 2020 4:31 PM
> > > To: AMBER Developers Mailing List <amber-developers.ambermd.org>
> > > Subject: Re: [AMBER-Developers] Faster NetFrc
> > >
> > > So if I accumulate them from the extra points in gradsum, but then only
> > > apply to atoms that are not extra points, we are good?
> > >
> > > On Fri, May 22, 2020 at 1:29 PM David Cerutti <dscerutti.gmail.com>
> > wrote:
> > >
> > > > Our emails may have crossed in writing, but it appears that in both
> > > > CPU and GPU codes the netfrc is being subtracted after extra points
> > > > have had their forces transmitted to their massive frame atoms. The
> > > > solutions to dealing with the pollution that netfrc correction causes
> > > > with extra points differ in each code.
> > > >
> > > > Dave
> > > >
> > > >
> > > > On Fri, May 22, 2020 at 4:27 PM Scott Le Grand <
> varelse2005.gmail.com>
> > > > wrote:
> > > >
> > > > > Cool, and if it's an extra point should I or should I not subtract
> > > > > the netfrc? Dumb questions asked upfront to save time later?
> > > > >
> > > > > On Fri, May 22, 2020 at 1:20 PM <cancersimulation.gmail.com>
> wrote:
> > > > >
> > > > > > It is static. Usually, it should be the same as "numextra" here
> > > > > >
> > > > > > use prmtop_dat_mod, only : numextra
> > > > > >
> > > > > > I believe Dave Cerutti has implemented some extra-point stuff and
> > > > > > I
> > > > don't
> > > > > > know if the above statement still holds with his stuff.
> > > > > >
> > > > > > Taisung
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Scott Le Grand [mailto:varelse2005.gmail.com]
> > > > > > Sent: Friday, May 22, 2020 3:02 PM
> > > > > > To: AMBER Developers Mailing List <amber-developers.ambermd.org>
> > > > > > Subject: Re: [AMBER-Developers] Faster NetFrc
> > > > > >
> > > > > > So also...
> > > > > >
> > > > > > I would assume the value of "ignored" is static. How do I
> > > > > > calculate it
> > > > a
> > > > > > priori? And it looks like I don't apply it to the extra points.
> > > > > > Feel
> > > > free
> > > > > > to
> > > > > > describe off-list, but I want to drill down deep and get this
> > > > > > right on
> > > > > the
> > > > > > first swing...
> > > > > >
> > > > > > On Thu, May 21, 2020 at 6:08 PM Scott Le Grand
> > > > > > <varelse2005.gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Excellent, that makes this much more straightforward. Should
> > > > > > > have something by early next week.
> > > > > > >
> > > > > > > On Thu, May 21, 2020 at 6:04 PM <taisung.gmail.com> wrote:
> > > > > > >
> > > > > > >> The force thresholds are for those "dummies" (not the
> > > > > > >> alchemical dummies but something like lone-pair points of
> water
> > > > > > >> models). The forces of those atoms are "transferred" to other
> > > > > > >> real atoms at the final force collection stage--and hence need
> > > > > > >> to be kept to zero during the netfrc stage. Of course, you
> may
> > > > > > >> find better ways to do things. For example, as Scott
> > > > > > >> mentioned, the only non-conserved force part is PME reciprocal
> > > > > > >> part. If the netfrc is done in the PME reciprocal part, there
> > is
> > > no need to have such force thresholds.
> > > > > > >>
> > > > > > >> Taisung
> > > > > > >>
> > > > > > >> -----Original Message-----
> > > > > > >> From: David Cerutti [mailto:dscerutti.gmail.com]
> > > > > > >> Sent: Thursday, May 21, 2020 8:37 PM
> > > > > > >> To: AMBER Developers Mailing List
> > > > > > >> <amber-developers.ambermd.org>
> > > > > > >> Subject: Re: [AMBER-Developers] Faster NetFrc
> > > > > > >>
> > > > > > >> As implied, the GTI code is the revision that introduced this.
> > > > > > >> Taisung can comment more on his logic, but the presence of
> this
> > > > > > >> "small" term reminds me of something he's got in the
> non-bonded
> > > > inner
> > > > > > >> loop as well. I'm not sure we ever determined why these
> > > > conditionals
> > > > > > >> were needed; I think the one in the non-bonded loop should
> just
> > > > > > >> go away after some other revisions I made, but I'll wait for
> > > > > > >> more
> > > > input.
> > > > > > >>
> > > > > > >> Dave
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, May 21, 2020 at 8:29 PM Scott Le Grand
> > > > > > >> <varelse2005.gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hey guys,
> > > > > > >> > Back as an AMBER developer and one of the first things I'd
> > > > > > >> > like to do is optimize netfrc.
> > > > > > >> >
> > > > > > >> > Fixed point conservative forces have no net force. They
> > > > > > >> > automagically cancel out 100%. So we can ignore everything
> > > > > > >> > except non-conserved
> > > > > > >> forces.
> > > > > > >> >
> > > > > > >> > The only non-conserved force I know of is the Ewald Gradient
> > > Sum.
> > > > > > >> > So if I add up the net force there, and handle it upon
> either
> > > > force
> > > > > > >> > reduction or update, I can eliminate two passes on memory
> and
> > > > > > >> > the two kernels currently dedicated to adding and then
> > > > > > >> > subtracting
> > > > them.
> > > > > > >> >
> > > > > > >> > But... Are there any other non-conserved forces?
> > > > > > >> >
> > > > > > >> > And... It looks like the FORTRAN and the CUDA code do
> > > > > > >> > different
> > > > > > things.
> > > > > > >> > FORTRAN simply averages the forces and subtracts. But CUDA
> > > > > > >> > only averages over atoms with force thresholds above a
> > > > > > >> > predefined small amount. What's up with that?
> > > > > > >> >
> > > > > > >> > pme_ene.F90:
> > > > > > >> > if (netfrc .gt. 0 .and. onstep) then
> > > > > > >> >
> > > > > > >> > if (ti_mode .eq. 0) then
> > > > > > >> > do i = 1, atm_cnt
> > > > > > >> > net_frcs(:) = net_frcs(:) + frc(:, i)
> > > > > > >> > end do
> > > > > > >> >
> > > > > > >> > ! Now do the correction:
> > > > > > >> >
> > > > > > >> > net_frcs(:) = net_frcs(:) / dble(atm_cnt - numextra)
> > > > > > >> >
> > > > > > >> > do i = 1, atm_cnt
> > > > > > >> > frc(:, i) = frc(:, i) - net_frcs(:)
> > > > > > >> > end do
> > > > > > >> > else
> > > > > > >> > do i = 1, atm_cnt
> > > > > > >> > ti_net_frcs(1, :) = ti_net_frcs(1, :) +
> > > > > > >> > ti_nb_frc(1, :,
> > > > i)
> > > > > > >> > ti_net_frcs(2, :) = ti_net_frcs(2, :) +
> > > > > > >> > ti_nb_frc(2, :,
> > > > i)
> > > > > > >> > end do
> > > > > > >> >
> > > > > > >> > ti_net_frcs(1,:) =
> > > > > > >> > ti_net_frcs(1,:)/dble(ti_atm_cnt(1)-ti_numextra_pts(1))
> > > > > > >> > ti_net_frcs(2,:) =
> > > > > > >> > ti_net_frcs(2,:)/dble(ti_atm_cnt(2)-ti_numextra_pts(2))
> > > > > > >> > net_frcs(:) = ti_net_frcs(1,:) + ti_net_frcs(2,:)
> > > > > > >> >
> > > > > > >> > do i = 1, atm_cnt
> > > > > > >> > ! This matches how sander removes netfrcs in TI
> runs
> > > > > > >> > if (ti_lst(1,i) .ne. 0) then
> > > > > > >> > frc(:, i) = frc(:, i) - ti_net_frcs(1,:)
> > > > > > >> > else if (ti_lst(2,i) .ne. 0) then
> > > > > > >> > frc(:, i) = frc(:, i) - ti_net_frcs(2,:)
> > > > > > >> > else
> > > > > > >> > frc(:, i) = frc(:, i) - net_frcs(:)
> > > > > > >> > end if
> > > > > > >> > end do
> > > > > > >> > end if
> > > > > > >> > ! Any extra points must have their 0.d0 forces
> reset...
> > > > > > >> >
> > > > > > >> > if (numextra .gt. 0 .and. frameon .ne. 0) &
> > > > > > >> > call zero_extra_pnts_vec(frc, ep_frames,
> > > > > > >> > gbl_frame_cnt)
> > > > > > >> >
> > > > > > >> > end if
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > GTI path:
> > > > > > >> > while (pos < cSim.atoms) {
> > > > > > >> > PMEFloat fx = converter(pX[pos], ONEOVERFORCESCALE);
> > > > > > >> > PMEFloat fy = converter(pY[pos], ONEOVERFORCESCALE);
> > > > > > >> > PMEFloat fz = converter(pZ[pos], ONEOVERFORCESCALE);
> > > > > > >> > if (abs(fx) > small || abs(fy) > small || abs(fz) >
> > small) {
> > > > > > >> > pX[pos] -= nfX;
> > > > > > >> > pY[pos] -= nfY;
> > > > > > >> > pZ[pos] -= nfZ;
> > > > > > >> > }
> > > > > > >> > pos += increment;
> > > > > > >> > }
> > > > > > >> >
> > > > > > >> > Scott
> > > > > > >> > _______________________________________________
> > > > > > >> > AMBER-Developers mailing list AMBER-Developers.ambermd.org
> > > > > > >> > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > > > >> >
> > > > > > >> _______________________________________________
> > > > > > >> AMBER-Developers mailing list
> > > > > > >> AMBER-Developers.ambermd.org
> > > > > > >> http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > > > >>
> > > > > > >>
> > > > > > >> _______________________________________________
> > > > > > >> AMBER-Developers mailing list
> > > > > > >> AMBER-Developers.ambermd.org
> > > > > > >> http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > > > >>
> > > > > > >
> > > > > > _______________________________________________
> > > > > > AMBER-Developers mailing list
> > > > > > AMBER-Developers.ambermd.org
> > > > > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > AMBER-Developers mailing list
> > > > > > AMBER-Developers.ambermd.org
> > > > > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > > >
> > > > > _______________________________________________
> > > > > AMBER-Developers mailing list
> > > > > AMBER-Developers.ambermd.org
> > > > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > > >
> > > > _______________________________________________
> > > > AMBER-Developers mailing list
> > > > AMBER-Developers.ambermd.org
> > > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > > >
> > > _______________________________________________
> > > AMBER-Developers mailing list
> > > AMBER-Developers.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > >
> > >
> > > _______________________________________________
> > > AMBER-Developers mailing list
> > > AMBER-Developers.ambermd.org
> > > http://lists.ambermd.org/mailman/listinfo/amber-developers
> > >
> > _______________________________________________
> > AMBER-Developers mailing list
> > AMBER-Developers.ambermd.org
> > http://lists.ambermd.org/mailman/listinfo/amber-developers
> >
> _______________________________________________
> AMBER-Developers mailing list
> AMBER-Developers.ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber-developers
>
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Fri Jun 26 2020 - 17:30:03 PDT
