I'll look into this right now.
Ross Walker wrote:
>Hi Wei, Francesco and others,
>
>As a follow-up to what I sent you earlier (Wei) I just ran some tests and
>have uncovered a problem with NEB and I likely suspect PIMD as well.
>
>I ran 50 steps of alanine phi-psi rotation based on the heating phase of the
>Amber 9 NEB tutorial but using Amber 10. This has 30 replicas. Running with
>30 MPI tasks I get what looks like a reasonable output file. The energies
>are reasonable and the energy of replica 1 and 30 remain fixed as they
>should do. So after 50 steps we have:
>
> NSTEP = 50 TIME(PS) = 0.02500 TEMP(K) = 0.97 PRESS =
>0.0
> Etot = 0.0000 EKtot = 0.0000 EPtot =
>-923.8469
> BOND = 23.8927 ANGLE = 57.5721 DIHED =
>270.9344
> 1-4 NB = 88.2706 1-4 EEL = 1323.4884 VDWAALS =
>-49.3114
> EELEC = -2086.8697 EGB = -551.8240 RESTRAINT =
>0.0000
>NEB replicate breakdown:
>Energy for replicate 1 = -32.7646
>Energy for replicate 2 = -32.7582
>Energy for replicate 3 = -32.7582
>Energy for replicate 4 = -32.7582
>Energy for replicate 5 = -32.7582
>Energy for replicate 6 = -32.7582
>Energy for replicate 7 = -32.7582
>Energy for replicate 8 = -32.7582
>Energy for replicate 9 = -32.7582
>Energy for replicate 10 = -32.7582
>Energy for replicate 11 = -32.7582
>Energy for replicate 12 = -32.7582
>Energy for replicate 13 = -32.7580
>Energy for replicate 14 = -32.7476
>Energy for replicate 15 = -26.6800
>Energy for replicate 16 = -26.1991
>Energy for replicate 17 = -29.4524
>Energy for replicate 18 = -29.4535
>Energy for replicate 19 = -29.4535
>Energy for replicate 20 = -29.4535
>Energy for replicate 21 = -29.4535
>Energy for replicate 22 = -29.4535
>Energy for replicate 23 = -29.4535
>Energy for replicate 24 = -29.4535
>Energy for replicate 25 = -29.4535
>Energy for replicate 26 = -29.4535
>Energy for replicate 27 = -29.4535
>Energy for replicate 28 = -29.4535
>Energy for replicate 29 = -29.4535
>Energy for replicate 30 = -29.4626
>Total Energy of replicates = -923.8469
>NEB RMS = 0.000000
>
>Note the NEB RMS is still not working though and always shows 0.000000.
>
>Now if we repeat the exact same calculation with 60 cpus (i.e. 2 per
>replica) then we get garbage after 50 steps:
>
>
> NSTEP = 50 TIME(PS) = 0.02500 TEMP(K) = 1188.21 PRESS =
>0.0
> Etot = 0.0000 EKtot = 0.0000 EPtot =
>31975.5541
> BOND = 25627.3226 ANGLE = 5938.0900 DIHED =
>880.9965
> 1-4 NB = 690.5816 1-4 EEL = 1400.3019 VDWAALS =
>119.5234
> EELEC = -2041.5006 EGB = -639.7612 RESTRAINT =
>0.0000
>NEB replicate breakdown:
>Energy for replicate 1 = -32.7626
>Energy for replicate 2 = -32.7329
>Energy for replicate 3 = -32.7329
>Energy for replicate 4 = -32.7329
>Energy for replicate 5 = -32.7329
>Energy for replicate 6 = -32.7329
>Energy for replicate 7 = -32.7329
>Energy for replicate 8 = -32.7329
>Energy for replicate 9 = -32.7329
>Energy for replicate 10 = -32.7329
>Energy for replicate 11 = -32.7329
>Energy for replicate 12 = -32.7329
>Energy for replicate 13 = -32.7323
>Energy for replicate 14 = -32.7201
>Energy for replicate 15 = -27.0427
>Energy for replicate 16 = 1051.8587
>Energy for replicate 17 = 1696.9930
>Energy for replicate 18 = 1687.6905
>Energy for replicate 19 = 1740.1162
>Energy for replicate 20 = 1682.0969
>Energy for replicate 21 = 1678.2029
>Energy for replicate 22 = 1486.9083
>Energy for replicate 23 = 2720.0971
>Energy for replicate 24 = 2061.0010
>Energy for replicate 25 = 2684.6440
>Energy for replicate 26 = 4699.5947
>Energy for replicate 27 = 1992.0437
>Energy for replicate 28 = 1240.5178
>Energy for replicate 29 = 2154.1431
>Energy for replicate 30 = 3884.9658
>Total Energy of replicates = 31975.5541
>NEB RMS = 0.000000
>
>The complete pimdout files are attached.
>
>And it would appear that the problem here is a function of us not having
>sufficiently robust test cases. For example the PIMD test cases (and the NEB
>ones) override whatever the user sets for DO_PARALLEL. E.g.:
>
>else
> set MY_DO_PARALLEL="$DO_PARALLEL"
> set numprocs=`echo $DO_PARALLEL | awk -f numprocs.awk `
> if ( $numprocs != 4 ) then
> echo "this test is set up for 4 nodes only, changing node number to
>4..."
> set MY_DO_PARALLEL=`echo $DO_PARALLEL | awk -f chgprocs.awk`
> endif
>
>Thus they ALWAYS use 4 cpus - and incidentally they all have 4 beads or
>replicas and so the situation where we have multiple cpus per bead or
>replica is NEVER TESTED :-(. Hence how such a problem arrises...
>
>Wei, since you know the NEB code better than me can you see if you can track
>down this problem...
>
>In addition I suggest we have a rethink of the way the PIMD test cases work
>so that the above problem does not occur.
>
>All the best
>Ross
>
>/\
>\/
>|\oss Walker
>
>| HPC Consultant and Staff Scientist |
>| San Diego Supercomputer Center |
>| Tel: +1 858 822 0854 | EMail:- ross.rosswalker.co.uk |
>| http://www.rosswalker.co.uk | PGP Key available on request |
>
>Note: Electronic Mail is not secure, has no guarantee of delivery, may not
>be read every day, and should not be used for urgent or sensitive issues.
>
>
Received on Wed May 02 2007 - 06:07:22 PDT
