Hi Amber Devs,
I've been making progress on the MPI-based parallel implementation of
mdgx, but I'm having a problem when it comes to sending structs and I'm
not sure how to proceed. There is a very convenient method for
user-defined data types in MPI, MPI_Type_struct, and I'd like to use this
to define my own data types and pass them in messages.
http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Type_struct.html
However, it seems that a struct like
struct iddd {
int it1;
double db1;
double db2;
double db3;
};
typedef struct iddd id3;
will show up as 32 bytes if I do sizeof(id3) even though
sizeof(int) + 3* sizeof(double) = 28
Similarly, with a struct like
struct iiddd {
int it1;
int it2;
double db1;
double db2;
double db3;
};
typedef struct iiddd i2d3;
the result of sizeof(i2d3) is 32, which fortunately is in agreement with
2*sizeof(int) + 3* sizeof(double) = 32
What's going on here, and is there any way I can work with MPI_Type_struct
and be assured that what I'm doing is safe? The main types that I'd like
to define for the bulk of communications have two integers followed by 3,
6, or 18 doubles, so in that sense it seems like I might be OK on
architectures where 2*sizeof(int) = sizeof(double) if sizeof(iNdK) (where
iNdK is a struct with N integers followed K doubles) is rounding up to the
nearest increment of sizeof(double).
It seems that a struct like
struct ididd {
int it1;
double db1;
int it2;
double db2;
double db3;
};
typedef struct ididd idid2;
would show up as sizeof(idid2) = 40 (I've been testing this), so that
jibes with other things I've heard about packing the smallest elements of
a struct together to conserve memory. But, I'm still left with my
fundamental question about how to send structs efficiently.
It seems that I could send them all as type MPI_BYTE using
N*sizeof(mystruct) to send N elements. That would work in most cases, it
seems, but it might get broken if someone had a heterogeneous cluster.
One other solution, suggested by the website above, might be to determine
the offsets for various fields in the struct by pointer arithmetic:
int offset1, offset2, offset3;
idid2 mystruct;
offset1 = &mystruct.db1 - &mystruct;
offset2 = &mystruct.it2 - &mystruct;
offset3 = &mystruct.db2 - &mystruct;
However when I test that, I get offset1=1, offset2=2, and offset3=3.
Consider the 64-bit architecture, that agrees with doubles being 8 bytes,
but it still seems that there's a lot of stuff I'm not able to control as
I'd like to. It may be that there isn't in fact a safe way to send
messages with heterogeneous data types in a heterogeneous cluster. Can
anyone suggest a strong solution?
Thanks,
Dave
_______________________________________________
AMBER-Developers mailing list
AMBER-Developers.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber-developers
Received on Sun Sep 25 2011 - 10:00:02 PDT
