Case 3. One process sends and receives; the other receives and sends It is always safe to order the calls of MPI_(I)SEND and MPI_(I)RECV so that a
2.5 Derived Data Types
2.5.2 Subroutines to Define Useful Derived Data Types
In parallelizing programs, you often need to send and receive elements of a submatrix that are not contiguous in memory. This section provides four utility subroutines to define the most typical derived data types that you might want to use in your programs. These subroutines are not part of the MPI library and are not supported by IBM.
Figure 23. A Submatrix for Transmission
Suppose you want to send data in the shaded region in Figure 23 on page 30. By using the utility subroutine para_type_block2a or para_type_block2 described below, the code becomes very neat:
CALL para_type_block2a(2, 7, 2, 5, MPI_REAL, itype)
CALL MPI_SEND(a(3,1), 1, itype, idst, itag, MPI_COMM_WORLD, ierr)
or
CALL para_type_block2(2, 7, 0, 3, 4, 1, 5, MPI_REAL, itype2) CALL MPI_SEND(a, 1, itype2, idst, itag, MPI_COMM_WORLD, ierr)
The meanings of parameters are clarified in the following sections. The source code for these subroutines is also included, as well as three dimensional
versions. When you use the data type defined by para_type_block2a, specify the initial address of the submatrix,a(3,1), as the address of the send buffer. On the other hand, in using para_type_block2, specify the initial address of the matrix,a ora(2,0), as the address of the send buffer.
2.5.2.1 Utility Subroutine: para_type_block2a
Parameters
INTEGER imin The minimum coordinate value of the first dimension INTEGER imax The maximum coordinate value of the first dimension INTEGER ilen The length of the rectangle in the first dimension INTEGER jlen The length of the rectangle in the second dimension INTEGER ioldtype The data type of the elements in the rectangular area INTEGER inewtype The newly defined derived data type
Figure 24. Utility Subroutine para_type_block2a
Source code
SUBROUTINE para_type_block2a(imin, imax, ilen, jlen,
& ioldtype, inewtype)
INCLUDE ’mpif.h’
CALL MPI_TYPE_VECTOR(jlen, ilen, imax - imin + 1, CALL para_type_block2a(imin, imax, ilen, jlen, ioldtype,
inewtype) Usage
& ioldtype, inewtype, ierr) CALL MPI_TYPE_COMMIT(inewtype, ierr)
END
2.5.2.2 Utility Subroutine: para_type_block2
Parameters
INTEGER imin The minimum coordinate value of the first dimension INTEGER imax The maximum coordinate value of the first dimension INTEGER jmin The minimum coordinate value of the second dimension INTEGER ista The minimum coordinate value of the rectangular area in
the first dimension
INTEGER iend The maximum coordinate value of the rectangular area in the first dimension
INTEGER jsta The minimum coordinate value of the rectangular area in the second dimension
INTEGER jend The maximum coordinate value of the rectangular area in the second dimension
INTEGER ioldtype The data type of the elements in the rectangular area INTEGER inewtype The newly defined derived data type
Figure 25. Utility Subroutine para_type_block2
Source code
SUBROUTINE para_type_block2(imin, imax, jmin,
& ista, iend, jsta, jend,
& ioldtype, inewtype)
INCLUDE ’mpif.h’
INTEGER iblock(2), idisp(2), itype(2)
CALL para_type_block2(imin, imax, jmin, ista, iend, jsta, jend, ioldtype, inewtype)
Usage
CALL MPI_TYPE_EXTENT(ioldtype, isize, ierr) ilen = iend - ista + 1
jlen = jend - jsta + 1
CALL MPI_TYPE_VECTOR(jlen, ilen, imax - imin + 1,
& ioldtype, itemp, ierr)
iblock(1) = 1 iblock(2) = 1 idisp(1) = 0
idisp(2) = ((imax-imin+1) * (jsta-jmin) + (ista-imin)) * isize itype(1) = MPI_LB
itype(2) = itemp
CALL MPI_TYPE_STRUCT(2, iblock, idisp, itype, inewtype, ierr) CALL MPI_TYPE_COMMIT(inewtype, ierr)
END
2.5.2.3 Utility Subroutine: para_type_block3a
Parameters
INTEGER imin The minimum coordinate value of the first dimension INTEGER imax The maximum coordinate value of the first dimension INTEGER jmin The minimum coordinate value of the second dimension INTEGER jmax The maximum coordinate value of the second dimension INTEGER ilen The length of the rectangular solid in the first dimension INTEGER jlen The length of the rectangular solid in the second dimension INTEGER klen The length of the rectangular solid in the third dimension INTEGER ioldtype The data type of the elements in the rectangular solid INTEGER inewtype The newly defined derived data type
CALL para_type_block3a(imin, imax, jmin, jmax, ilen, jlen, klen, ioldtype, inewtype)
Usage
Figure 26. Utility Subroutine para_type_block3a
Source code
SUBROUTINE para_type_block3a(imin, imax, jmin, jmax,
& ilen, jlen, klen,
& ioldtype,inewtype)
INCLUDE ’mpif.h’
CALL MPI_TYPE_EXTENT(ioldtype, isize, ierr) CALL MPI_TYPE_VECTOR(jlen, ilen, imax - imin + 1,
& ioldtype, itemp, ierr)
idist = (imax - imin + 1) * (jmax - jmin + 1) * isize
CALL MPI_TYPE_HVECTOR(klen, 1, idist, itemp, inewtype, ierr) CALL MPI_TYPE_COMMIT(inewtype, ierr)
END
2.5.2.4 Utility Subroutine: para_type_block3
Parameters
INTEGER imin The minimum coordinate value of the first dimension INTEGER imax The maximum coordinate value of the first dimension
CALL para_type_block3(imin, imax, jmin, jmax, kmin, ista, iend, jsta, jend, ksta, kend, ioldtype, inewtype) Usage
INTEGER jmin The minimum coordinate value of the second dimension INTEGER jmax The maximum coordinate value of the second dimension INTEGER kmin The minimum coordinate value of the third dimension INTEGER ista The minimum coordinate value of the rectangular solid in
the first dimension
INTEGER iend The maximum coordinate value of the rectangular solid in the first dimension
INTEGER jsta The minimum coordinate value of the rectangular solid in the second dimension
INTEGER jend The maximum coordinate value of the rectangular solid in the second dimension
INTEGER ksta The minimum coordinate value of the rectangular solid in the third dimension
INTEGER kend The maximum coordinate value of the rectangular solid in the third dimension
INTEGER ioldtype The data type of the elements in the rectangular solid INTEGER inewtype The newly defined derived data type
Figure 27. Utility Subroutine para_type_block3
Source code
SUBROUTINE para_type_block3(imin, imax, jmin, jmax, kmin,
& ista, iend, jsta, jend, ksta, kend,
& ioldtype, inewtype) INCLUDE ’mpif.h’
INTEGER iblock(2), idisp(2), itype(2) CALL MPI_TYPE_EXTENT(ioldtype, isize, ierr) ilen = iend - ista + 1
jlen = jend - jsta + 1 klen = kend - ksta + 1
CALL MPI_TYPE_VECTOR(jlen, ilen, imax - imin + 1,
& ioldtype, itemp, ierr)
idist = (imax-imin+1) * (jmax-jmin+1) * isize
CALL MPI_TYPE_HVECTOR(klen, 1, idist, itemp, itemp2, ierr) iblock(1) = 1
iblock(2) = 1 idisp(1) = 0
idisp(2) = ((imax-imin+1) * (jmax-jmin+1) * (ksta-kmin)
& + (imax-imin+1) * (jsta-jmin) + (ista-imin)) * isize itype(1) = MPI_LB
itype(2) = itemp2
CALL MPI_TYPE_STRUCT(2, iblock, idisp, itype, inewtype, ierr) CALL MPI_TYPE_COMMIT(inewtype, ierr)
END