|
HWRF
trunk@4391
|
This subclass of HWRFTask represents a WPS Task. More...
This subclass of HWRFTask represents a WPS Task.
Multiple WPS jobs run in the same work directory. This class allows the jobs to be represented together as a set, with one work directory specified at the top level. This class exists only to reduce code duplication by letting Metgrid, Geogrid and Ungrib share a superclass.
Public Member Functions | |
| def | __init__ (self, dstore, conf, section, sim, domains, taskname=None, wpsdir=None, starttime=None, increment=None, endtime=None, parent_atime=None, geogrid_from=None, kwargs) |
| Create a new WPSTask. More... | |
| def | io_suffix (self, stream) |
| Guesses the file suffix (.nc, etc.) for the specified stream. More... | |
| def | getsim (self) |
| The WRFSimulation object for which we are preparing input. More... | |
| def | check_geogrid (self) |
| Checks to see if the geogrid MOAD output file is present and non-empty in the current working directory. More... | |
| def | link_geogrid (self) |
| Links geogrid output from the specified Geogrid task to the current working directory. More... | |
| def | times (self) |
| Iterates over all output times. More... | |
| def | domains (self) |
| Iterates over the domains in this WPSTask. | |
| def | link_fix |
| Links all fix files for ungrib to the current working directory. More... | |
| def | make_product_structure (self, stream) |
| Generates the self._products data structure used by the products method. More... | |
| def | make_products (self) |
| This subroutine should be implemented in subclasses. More... | |
| def | products |
| Iterates over all products. More... | |
| def | make_namelist (self) |
| Returns the namelist.wps contents as a string. More... | |
| def | undeliver_products |
| Deletes all delivered products and marks them as unavailable. More... | |
| def | deliver_products |
| This is called from self.run to deliver all products to the intercom directory. More... | |
 Public Member Functions inherited from hwrf.hwrftask.HWRFTask | |
| def | __init__ (self, dstore, conf, section, taskname=None, workdir=None, outdir=None, storminfo=UNSPECIFIED, taskvars=UNSPECIFIED, kwargs) |
| Creates an HWRFTask. More... | |
| def | isfakestorm (self) |
| def | ismultistorm (self) |
| def | get_workdir (self) |
| Returns the directory the class should work in, as set by the "workdir" metadata value. More... | |
| def | set_workdir (self, val) |
| Sets the directory the class should work in. More... | |
| def | get_outdir (self) |
| Gets the directory that should receive output data. More... | |
| def | set_outdir (self, val) |
| Sets the directory that should receive output data. More... | |
| def | realtime (self) |
| Is this job a real-time forecast job? More... | |
| def | redirect (self) |
| Should subprograms' outputs be redirected to separate files? | |
| def | scrub (self) |
| Should temporary files be deleted as soon as they are not needed? | |
| def | tvset (self, opt, val) |
| Sets a taskvar option's value. More... | |
| def | tvdel (self, opt) |
| Deletes an object-local value set by tvset. More... | |
| def | tvget (self, opt) |
| Gets a taskvar's value. More... | |
| def | tvhave |
| Is a taskvar set? More... | |
| def | taskvars (self) |
| The dict of object-local values used for string substitution. More... | |
| def | confint |
| Alias for self.conf.getint for section self.section. More... | |
| def | confstr |
| Alias for self.conf.getstr for section self.section. More... | |
| def | conffloat |
| Alias for self.conf.getfloat for section self.section. More... | |
| def | confbool |
| Alias for self.conf.getbool for section self.section. More... | |
| def | confget |
| Alias for self.conf.get for section self.section. More... | |
| def | confitems |
| Alias for self.conf.items for section self.section. More... | |
| def | confstrinterp (self, string, section=None, kwargs) |
| Alias for self.icstr for backward compatibility. More... | |
| def | conftimestrinterp (self, string, ftime, atime=None, section=None, kwargs) |
| Alias for self.timestr for backward comaptibility. More... | |
| def | confraw |
| Get a raw configuration value before string expansion. More... | |
| def | icstr (self, string, section=None, kwargs) |
| Expands a string in the given conf section. More... | |
| def | timestr (self, string, ftime, atime=None, section=None, kwargs) |
| Expands a string in the given conf section, including time vars. More... | |
| def | getdir |
| Alias for hwrf.config.HWRFConfig.get() for the "dir" section. More... | |
| def | getexe |
| Alias for hwrf.config.HWRFConfig.get() for the "exe" section. More... | |
| def | getconf (self) |
| Returns this HWRFTask's hwrf.config.HWRFConfig object. More... | |
| def | getsection (self) |
| Returns this HWRFTask's section name in the HWRFConfig. More... | |
| def | log |
| Obtain a logging domain. More... | |
| def | inputiter (self) |
| Iterates over all inputs required by this task. More... | |
| Public Member Functions inherited from produtil.datastore.Task | |
| def | __init__ (self, dstore, taskname, logger=None, kwargs) |
| Task constructor. More... | |
| def | jlogfile (self) |
| returns the jlogfile logger. More... | |
| def | postmsg (self, message, args, kwargs) |
| same as produtil.log.jlogger.info() More... | |
| def | setstate (self, val) |
| Sets the state of this job. More... | |
| def | getstate (self) |
| Returns the job state. More... | |
| def | strstate (self) |
| A string representation of the job state. More... | |
| def | gettaskname (self) |
| Returns the task name part of the database ID, which is the same as the prodname. More... | |
| def | products (self, args, kwargs) |
| Iterate over the products this task produces. More... | |
| def | log (self) |
| Returns the logger object for this task. More... | |
| def | clean (self) |
| Cleans up any unneeded data used by this task. More... | |
| def | unrun (self) |
| Undoes the effect of run(). More... | |
| def | run (self) |
| Performs the work this Task should do and generates all products. More... | |
| def | is_completed (self) |
| Is this task complete? More... | |
| def | completed (self) |
| Read-only property: is this task completed? Same as is_completed() More... | |
| def | runpart (self) |
| Run some of this task's work, deliver some products. More... | |
| Public Member Functions inherited from produtil.datastore.Datum | |
| def | __init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs) |
| Datum constructor. More... | |
| def | __enter__ (self) |
| Acquires this object's thread lock. More... | |
| def | __exit__ (self, etype, evalue, traceback) |
| Releases this object's thread lock. More... | |
| def | validate (self) |
| Validates this object's Datastore, prodname and category. More... | |
| def | getid (self) |
| Returns the database ID of this datum. More... | |
| def | getdatastore (self) |
| Returns the datastore of this datum. More... | |
| def | transaction (self) |
| Creates, but does not lock, a Transaction for this datum's datastore. More... | |
| def | getprodtype (self) |
| Returns the product type of this Datum. More... | |
| def | getprodname (self) |
| Returns the product name part of the database ID. More... | |
| def | getcategory (self) |
| Returns the product category part of the database ID. More... | |
| def | getlocation (self) |
| Returns the "location" field of this Datum's database entry. More... | |
| def | setlocation (self, v) |
| Sets the "location" field of this Datum's database entry. More... | |
| def | __hash__ (self) |
| Integer hash function. More... | |
| def | __str__ (self) |
| Human-readable description of this Datum. More... | |
| def | __repr__ (self) |
| Python code-like description of this Datum. More... | |
| def | __cmp__ (self, other) |
| Compares two Datums' prodnames and categories. More... | |
| def | set_loc_avail (self, loc, avail) |
| Sets the location and availability of this Datum in a single transaction. More... | |
| def | update (self) |
| Discards all cached metadata and refreshes it from the Datastore. More... | |
| def | __getitem__ (self, k) |
| Returns the value of the specified metadata key or raises KeyError. More... | |
| def | meta |
| Return the value of a metadata key. More... | |
| def | get |
| Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More... | |
| def | __setitem__ (self, k, v) |
| Sets the value of the specified metadata key. More... | |
| def | __delitem__ (self, k) |
| Deletes the specified metadata key, which must not be "available" or "location". More... | |
| def | __contains__ (self, k) |
| Determines if a metadata key is set. More... | |
| def | iteritems (self) |
| Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More... | |
Public Attributes | |
| location | |
| nl | |
| an hwrf.namelist.Conf2Namelist with the namelist.wps information | |
| starttime | |
| Simulation start time. More... | |
| endtime | |
| Simulation end time. More... | |
| increment | |
| Simulation boundary input interval. | |
| parent_atime | |
| Parent model analysis time. | |
| Public Attributes inherited from hwrf.hwrftask.HWRFTask | |
| storminfo | |
| The hwrf.storminfo.StormInfo describing the vitals information for the storm processed by this HWRFTask. More... | |
Properties | |
| nocolons | |
| Read-only property that guesses whether colons should be omitted from filenames (True) or not (False) More... | |
| maxdom = property(_maxdom,None,None,"The number of WRF domains.") | |
| The number of WRF domains. More... | |
| MOAD = property(_MOAD,None,None,"Returns the Mother of All Domains (MOAD)") | |
| The Mother Of All Domains (MOAD) as an hwrf.wrf.WRFDomain. | |
| sim | |
| Read-only property: the hwrf.wrf.WRFSimulation for this WPSTask. More... | |
| Properties inherited from hwrf.hwrftask.HWRFTask | |
| workdir | |
| The directory in which this task should be run. More... | |
| outdir | |
| The directory in which this task should deliver its final output. More... | |
| conf | |
| This HWRFTask's hwrf.config.HWRFConfig object. More... | |
| section | |
| The confsection in self.section for this HWRFTask (read-only) More... | |
| Properties inherited from produtil.datastore.Task | |
| state | |
| Read-write property: the job state. More... | |
| taskname | |
| Read-only property: the name of this task. More... | |
| Properties inherited from produtil.datastore.Datum | |
| prodname = property(getprodname,None,None,) | |
| Read-only property, an alias for getprodname(): the product name part of the database ID. More... | |
| category = property(getcategory,None,None,) | |
| Read-only property, an alias for getcategory(), the category name part of the database ID. More... | |
| prodtype | |
| Read-only property, an alias for getprodtype(), the product type. More... | |
| did | |
| Read-only property, an alias for getid(). More... | |
| dstore | |
| Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More... | |
| location | |
| Read-write property, an alias for getlocation() and setlocation(). More... | |
| def hwrf.wps.WPSTask.__init__ | ( | self, | |
| dstore, | |||
| conf, | |||
| section, | |||
| sim, | |||
| domains, | |||
taskname = None, |
|||
wpsdir = None, |
|||
starttime = None, |
|||
increment = None, |
|||
endtime = None, |
|||
parent_atime = None, |
|||
geogrid_from = None, |
|||
| kwargs | |||
| ) |
Create a new WPSTask.
| dstore | The produtil.datastore.Datastore for database information |
| conf | the hwrf.config.HWRFConfig for configuration info |
| section | the section to use in conf |
| sim | The "sim" is the WRFSimulation for which we are preparing input. |
| domains | The domains must be a list of WRFDomain objects. |
| taskname | Optional: the name of this Task. Default: config section name. |
| wpsdir | Optional: the directory in which to run WPS. Default: taskname subdirectory of conf.getdir("WORKhwrf") |
| starttime,endtime | Optional: simulation length. Default: same as sim. |
| increment | Optional: boundary input interval. Default: get from sim. |
| parent_atime | Optional: analysis time of parent model. Default: starttime |
| geogrid_from | Optional: a Geogrid object whose output should be linked before running this WPS step. This is used when the WPS Ungrib and Metgrid are run more than once on the same domain. For example, one may run multiple analysis times or multiple forecast lengths off of the same geogrid output. |
| kwargs | Other options are sent to the hwrf.hwrftask.HWRFTask.__init__ constructor. |
| def hwrf.wps.WPSTask.check_geogrid | ( | self | ) |
Checks to see if the geogrid MOAD output file is present and non-empty in the current working directory.
Raises GeogridNoOutput if the output is missing or empty.
Definition at line 227 of file wps.py.
Referenced by hwrf.wps.Metgrid.run().
| def hwrf.wps.WPSTask.deliver_products | ( | self, | |
time = None, |
|||
domain = None, |
|||
fromloc = None, |
|||
keep = True, |
|||
relink = False |
|||
| ) |
This is called from self.run to deliver all products to the intercom directory.
| time,domain | The optional time and domain arguments are sent to self.products. |
| fromloc | By default, this routine assumes that the file to be delivered is in the current working directory with the same name as the destination file. To change that, give a lambda function in "fromloc", which converts the destination filename (the only argument) to a local filename. |
| keep | The "keep" argument has the same meaning as in deliver_file: if False, the file may be moved to the destination. |
| relink | If True, and the file is moved, then a symlink will be made from the original file location to the destination. |
Definition at line 392 of file wps.py.
Referenced by hwrf.ww3.WW3Init.run(), hwrf.wps.Geogrid.run(), hwrf.wps.Ungrib.run(), hwrf.wps.Metgrid.run(), and hwrf.wps.WPSTask.undeliver_products().
| def hwrf.wps.WPSTask.getsim | ( | self | ) |
| def hwrf.wps.WPSTask.io_suffix | ( | self, | |
| stream | |||
| ) |
Guesses the file suffix (.nc, etc.) for the specified stream.
Tries to guess what io suffix WPS will use for a stream. If the stream is not one of the known streams, this function logs a warning and tries to continue, but will likely fail.
| stream | the stream of interest. |
Definition at line 189 of file wps.py.
Referenced by hwrf.wps.WPSTask.check_geogrid(), hwrf.wps.Geogrid.make_products(), and hwrf.wps.Metgrid.make_products().
| def hwrf.wps.WPSTask.link_fix | ( | self, | |
geog_data = False, |
|||
table = None |
|||
| ) |
Links all fix files for ungrib to the current working directory.
| geog_data | if True, also link the geog-data directory |
| table | name of the table file symbolic link |
Definition at line 305 of file wps.py.
Referenced by hwrf.wps.Geogrid.run(), hwrf.wps.Ungrib.run(), and hwrf.wps.Metgrid.run().
| def hwrf.wps.WPSTask.link_geogrid | ( | self | ) |
Links geogrid output from the specified Geogrid task to the current working directory.
Will raise an exception if Geogrid has not produced its outputs yet. This is used when multiple Metgrid executions are used with the same Geogrid. The geogrid only needs to be run once since its outputs do not change with time, just with domain configuration.
Specifically, this finds all Product objects in task self._geogrid_from whose prodname contains "geo" and links from the product's location to its prodname in the current working directory. This logic must match the logic in Geogrid.make_products and WPSTask.check_geogrid.
Definition at line 237 of file wps.py.
Referenced by hwrf.wps.Metgrid.run().
| def hwrf.wps.WPSTask.make_namelist | ( | self | ) |
Returns the namelist.wps contents as a string.
Definition at line 377 of file wps.py.
Referenced by hwrf.wps.Geogrid.run(), hwrf.wps.Ungrib.run(), and hwrf.wps.Metgrid.run().
| def hwrf.wps.WPSTask.make_product_structure | ( | self, | |
| stream | |||
| ) |
Generates the self._products data structure used by the products method.
Should be called by make_products in subclasses.
Definition at line 323 of file wps.py.
Referenced by hwrf.wps.Geogrid.make_products(), and hwrf.wps.Metgrid.make_products().
| def hwrf.wps.WPSTask.make_products | ( | self | ) |
This subroutine should be implemented in subclasses.
It should call self.make_product_structure(), and then add products by doing:
where the domain is a WRFDomainBase subclass, the time is anything accepted by to_datetime_rel's first argument, and the product is any subclass of Product.
| def hwrf.wps.WPSTask.products | ( | self, | |
time = None, |
|||
domain = None |
|||
| ) |
Iterates over all products.
| domain | the hwrf.wrf.WRFDomain |
| time | the time |
Definition at line 345 of file wps.py.
Referenced by hwrf.wrf.ExternalWRFTask.__init__(), hwrf.wps.WPSTask.deliver_products(), hwrf.wps.WPSTask.undeliver_products(), hwrf.wrf.ExternalWRFTask.unrun(), and hwrf.wrf.ExternalWRFTask.wrf_check().
| def hwrf.wps.WPSTask.times | ( | self | ) |
Iterates over all output times.
Definition at line 289 of file wps.py.
Referenced by hwrf.wps.Ungrib.check_outfiles(), hwrf.wps.Ungrib.get_grib(), hwrf.wps.Ungrib.inputiter(), hwrf.wps.Metgrid.make_products(), and hwrf.wps.Metgrid.run().
| def hwrf.wps.WPSTask.undeliver_products | ( | self, | |
time = None, |
|||
domain = None, |
|||
fromloc = None |
|||
| ) |
Deletes all delivered products and marks them as unavailable.
| time,domain | passed to products() to get the list of products |
| fromloc | Ignored. |
| hwrf.wps.WPSTask.endtime |
Simulation end time.
Definition at line 143 of file wps.py.
Referenced by hwrf.wps.WPSTask.deliver_products(), hwrf.wps.WPSTask.make_product_structure(), hwrf.wps.Metgrid.make_products(), hwrf.wps.Metgrid.run(), and hwrf.wps.WPSTask.times().
| hwrf.wps.WPSTask.starttime |
Simulation start time.
Definition at line 141 of file wps.py.
Referenced by hwrf.wps.WPSTask.deliver_products(), hwrf.wps.Geogrid.geodat(), hwrf.wps.Ungrib.get_grib(), hwrf.wps.Ungrib.inputiter(), hwrf.wps.WPSTask.make_product_structure(), hwrf.wps.Geogrid.make_products(), hwrf.wps.Metgrid.make_products(), hwrf.wps.Metgrid.run(), hwrf.wps.WPSTask.times(), and hwrf.wps.WPSTask.undeliver_products().
|
static |
The number of WRF domains.
Definition at line 214 of file wps.py.
Referenced by hwrf.wrfbase.WRFDomains.__len__(), hwrf.wrfbase.WRFDomains.add(), hwrf.wps.WPSTask.deliver_products(), and hwrf.wrfbase.WRFDomains.fill_auto_starts_multistorm().
|
static |
Read-only property that guesses whether colons should be omitted from filenames (True) or not (False)
Definition at line 185 of file wps.py.
Referenced by hwrf.wps.WPSTask.deliver_products(), hwrf.wps.Metgrid.make_products(), and hwrf.wrf.WRFDomain.trackpatcf_file().
|
static |
Read-only property: the hwrf.wrf.WRFSimulation for this WPSTask.
Definition at line 300 of file wps.py.
Referenced by hwrf.fcsttask.WRFGhost.get_ghost(), hwrf.fcsttask.WRFGhostForPost.get_ghost(), hwrf.fcsttask.WRFAnl.get_wrfanl(), hwrf.ensda.FromGFSENKF.make_fcst(), hwrf.fcsttask.WRFAnl4Trak.make_products(), hwrf.fcsttask.WRFAnl4Trak.products(), and hwrf.fcsttask.WRFAnl4Trak.track_domains().
1.8.10
