|
HWRF
trunk@4391
|
This is the abstract superclass of all classes that represent one or more MPI ranks, including MPI ranks that are actually serial programs. More...
This is the abstract superclass of all classes that represent one or more MPI ranks, including MPI ranks that are actually serial programs.
Subclasses of MPIRanksBase allow an MPI program to be represented as a tree of MPIRanksBase objects, in such a way that they can be easily converted to a produtil.prog.Runner object for execution. The actual conversion to a Runner is done in the produtil.mpi_impl package (see produtil/mpi_impl/__init__.py)
Definition at line 68 of file mpiprog.py.
Public Member Functions | |
| def | to_arglist |
| This is the underlying implementation of most of the mpi_impl modules, and hence make_runner as well. More... | |
| def | make_runners_immutable (self) |
| Returns a copy of this object where all child produtil.prog.Runner objects have been replaced with produtil.prog.ImmutableRunner objects. More... | |
| def | get_logger (self) |
| Returns a logger.Logger object for this MPIRanksBase or one from its child MPIRanksBase objects (if it has any). More... | |
| def | check_serial (self) |
| Returns a tuple (s,p) where s=True if there are serial ranks in this part of the MPI program, and p=True if there are parallel ranks. More... | |
| def | nranks (self) |
| Returns the number of ranks in this part of the MPI program. More... | |
| def | ranks (self) |
| Iterates over all MPIRank objects in this part of the MPI program. More... | |
| def | ngroups (self) |
| Returns the number of groups of repeated MPI ranks in the MPI program. More... | |
| def | groups |
| Iterates over all groups of repeating MPI ranks in the MPI program returning tuples (r,c) containing a rank r and the count (number) of that rank c. More... | |
| def | getthreads (self) |
| Returns the number of threads requested by this MPI rank, or by each MPI rank in this group of MPI ranks. More... | |
| def | setthreads (self, nthreads) |
| Sets the number of threads requested by each MPI rank within this group of MPI ranks. More... | |
| def | delthreads (self) |
| Removes the request for threads. More... | |
| def | __eq__ (self, other) |
| Returns True if this part of the MPI program is the same as another part. More... | |
| def | __mul__ (self, factor) |
| Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times. More... | |
| def | __rmul__ (self, other) |
| Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times. More... | |
| def | __add__ (self, other) |
| Returns a new set of MPI ranks that consist of this set of ranks with the "other" set appended. More... | |
| def | __radd__ (self, other) |
| Returns a new set of MPI ranks that consist of the "other" set of ranks with this set appended. More... | |
| def | isplainexe (self) |
| Determines if this set of MPI ranks can be represented by a single serial executable with a single set of arguments run without MPI. More... | |
| def | to_shell (self) |
| Returns a POSIX sh command that will execute the serial program, if possible, or raise a subclass of NotValidPosixSh otherwise. More... | |
| def | expand_iter |
| This is a wrapper around ranks() and groups() which will call self.groups() if expand=False. More... | |
| def | __repr__ (self) |
| Returns a string representation of this object intended for debugging. More... | |
| def produtil.mpiprog.MPIRanksBase.__add__ | ( | self, | |
| other | |||
| ) |
Returns a new set of MPI ranks that consist of this set of ranks with the "other" set appended.
| other | the data to append |
Definition at line 228 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.__eq__ | ( | self, | |
| other | |||
| ) |
Returns True if this part of the MPI program is the same as another part.
| other | the other object |
Definition at line 213 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.__mul__ | ( | self, | |
| factor | |||
| ) |
Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times.
| factor | how many times to duplicate |
Definition at line 218 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.__radd__ | ( | self, | |
| other | |||
| ) |
Returns a new set of MPI ranks that consist of the "other" set of ranks with this set appended.
| other | the data to prepend |
Definition at line 233 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.__repr__ | ( | self | ) |
Returns a string representation of this object intended for debugging.
Definition at line 271 of file mpiprog.py.
Referenced by produtil.prog.Runner.__str__().
| def produtil.mpiprog.MPIRanksBase.__rmul__ | ( | self, | |
| other | |||
| ) |
Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times.
| other | how many times to duplicate |
Definition at line 223 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.check_serial | ( | self | ) |
Returns a tuple (s,p) where s=True if there are serial ranks in this part of the MPI program, and p=True if there are parallel ranks.
Note that it is possible that both could be True, which is an error. It is also possible that neither are True if there are zero ranks.
Definition at line 161 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.delthreads | ( | self | ) |
Removes the request for threads.
Definition at line 207 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.expand_iter | ( | self, | |
| expand, | |||
threads = False |
|||
| ) |
This is a wrapper around ranks() and groups() which will call self.groups() if expand=False.
If expand=True, this will call ranks() returning a tuple (rank,1) for each rank.
| expand | If True, expand groups of identical ranks into one rank of each member |
| threads | If True, then a third element will be in each tuple: the number of requested threads per MPI rank. |
Definition at line 250 of file mpiprog.py.
Referenced by produtil.mpiprog.MPIRanksBase.to_arglist().
| def produtil.mpiprog.MPIRanksBase.get_logger | ( | self | ) |
Returns a logger.Logger object for this MPIRanksBase or one from its child MPIRanksBase objects (if it has any).
If no logger is found, None is returned.
Definition at line 156 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.getthreads | ( | self | ) |
Returns the number of threads requested by this MPI rank, or by each MPI rank in this group of MPI ranks.
If different ranks have different numbers of threads, returns the maximum requested. Returns None if no threads are requested.
Definition at line 189 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.groups | ( | self, | |
threads = False |
|||
| ) |
Iterates over all groups of repeating MPI ranks in the MPI program returning tuples (r,c) containing a rank r and the count (number) of that rank c.
| threads | If True, then a three-element tuple is iterated, (r,c,t) where the third element is the number of threads. |
Definition at line 180 of file mpiprog.py.
Referenced by produtil.mpiprog.MPIRanksBase.delthreads(), produtil.mpiprog.MPIRanksBase.expand_iter(), and produtil.mpiprog.MPIRanksBase.setthreads().
| def produtil.mpiprog.MPIRanksBase.isplainexe | ( | self | ) |
Determines if this set of MPI ranks can be represented by a single serial executable with a single set of arguments run without MPI.
Returns false by default: this function can only return true for MPISerial.
Definition at line 238 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.make_runners_immutable | ( | self | ) |
Returns a copy of this object where all child produtil.prog.Runner objects have been replaced with produtil.prog.ImmutableRunner objects.
Definition at line 152 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.ngroups | ( | self | ) |
Returns the number of groups of repeated MPI ranks in the MPI program.
Definition at line 176 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.nranks | ( | self | ) |
Returns the number of ranks in this part of the MPI program.
Definition at line 168 of file mpiprog.py.
Referenced by produtil.mpiprog.MPIRanksSPMD.__add__(), and produtil.mpiprog.MPIRanksBase.to_arglist().
| def produtil.mpiprog.MPIRanksBase.ranks | ( | self | ) |
Iterates over all MPIRank objects in this part of the MPI program.
Definition at line 172 of file mpiprog.py.
Referenced by produtil.mpiprog.MPIRanksBase.expand_iter().
| def produtil.mpiprog.MPIRanksBase.setthreads | ( | self, | |
| nthreads | |||
| ) |
Sets the number of threads requested by each MPI rank within this group of MPI ranks.
Definition at line 201 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.to_arglist | ( | self, | |
to_shell = False, |
|||
expand = False, |
|||
shell_validate = None, |
|||
pre = [], |
|||
before = [], |
|||
between = [], |
|||
after = [], |
|||
post = [], |
|||
extra = {} |
|||
| ) |
This is the underlying implementation of most of the mpi_impl modules, and hence make_runner as well.
It converts this group of MPI ranks into a set of arguments suitable for sending to a Runner object or for writing to a command file. This is done by iterating over either all ranks (if expand=True) or groups of repeated ranks (if expand=False), converting their arguments to a list. It prepends an executable, and can insert other arguments in specified locations (given in the pre, before, between, after, and post arguments). It can also use the to_shell argument to convert programs to POSIX sh commands, and it performs simple string interpolation via the "extra" hash.
If to_shell=False then the executable and arguments are inserted directly to the output list. Otherwise (when to_shell=True) the to_shell subroutine is called on the MPIRank object to produce a single argument that contains a shell command. That single argument is then used in place of the executable and arguments. Note that may raise NotValidPosixSh (or a subclass thereof) if the command cannot be expressed as a shell command. In addition, if shell_validate is not None, then it is called on each post-conversion shell argument, and the return value is used instead.
You can specify additional argument lists to be inserted in certain locations. Each argument in those lists will be processed through the % operator, specifying "extra" as the keyword list with two new keywords added: nworld is the number of ranks in the MPI program, and "n" is the number in the current group of repeated ranks if expand=False (n=1 if expand=True). Those argument lists are: pre, before, between, after and post.
| to_shell | If True, convert executable and arguments to a POSIX sh command instead of inserting them directly. |
| expand | If True, groups of repeated ranks are expanded. |
| shell_validate | A function to convert each argument to some "shell-acceptable" version. |
| pre | Inserted before everything else. This is where you would put the "mpiexec" and any global settings. |
| before | Inserted before each rank (if expand=True) or group (if expand=False) |
| between | Inserted between each rank (if expand=True) or group (if expand=False) |
| after | Inserted after each rank (if expand=True) or group (if expand=False) |
| post | Appended at the end of the list of arguments. |
| extra | used for string expansion |
Definition at line 78 of file mpiprog.py.
| def produtil.mpiprog.MPIRanksBase.to_shell | ( | self | ) |
Returns a POSIX sh command that will execute the serial program, if possible, or raise a subclass of NotValidPosixSh otherwise.
Works only on single MPI ranks that are actually MPI wrappers around a serial program (ie.: from mpiserial).
Definition at line 244 of file mpiprog.py.
1.8.10
