|
HWRF
trunk@4391
|
Represents a process or actor that makes a Product. More...
Represents a process or actor that makes a Product.
A Task represents some work that needs to be done to produce Products. A task has a state (stored in the "available" metadata attribute), a location, whose meaning is up to the implementer to decide, and a logger.Logger. As with all Datum subclasses, a Task also has arbitrary metadata.
Definition at line 1052 of file datastore.py.
Public Member Functions | |
| def | __init__ (self, dstore, taskname, logger=None, kwargs) |
| Task constructor. More... | |
| def | jlogfile (self) |
| returns the jlogfile logger. More... | |
| def | postmsg (self, message, args, kwargs) |
| same as produtil.log.jlogger.info() More... | |
| def | setstate (self, val) |
| Sets the state of this job. More... | |
| def | getstate (self) |
| Returns the job state. More... | |
| def | strstate (self) |
| A string representation of the job state. More... | |
| def | gettaskname (self) |
| Returns the task name part of the database ID, which is the same as the prodname. More... | |
| def | products (self, args, kwargs) |
| Iterate over the products this task produces. More... | |
| def | log (self) |
| Returns the logger object for this task. More... | |
| def | clean (self) |
| Cleans up any unneeded data used by this task. More... | |
| def | unrun (self) |
| Undoes the effect of run(). More... | |
| def | run (self) |
| Performs the work this Task should do and generates all products. More... | |
| def | is_completed (self) |
| Is this task complete? More... | |
| def | completed (self) |
| Read-only property: is this task completed? Same as is_completed() More... | |
| def | runpart (self) |
| Run some of this task's work, deliver some products. More... | |
 Public Member Functions inherited from produtil.datastore.Datum | |
| def | __init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs) |
| Datum constructor. More... | |
| def | __enter__ (self) |
| Acquires this object's thread lock. More... | |
| def | __exit__ (self, etype, evalue, traceback) |
| Releases this object's thread lock. More... | |
| def | validate (self) |
| Validates this object's Datastore, prodname and category. More... | |
| def | getid (self) |
| Returns the database ID of this datum. More... | |
| def | getdatastore (self) |
| Returns the datastore of this datum. More... | |
| def | transaction (self) |
| Creates, but does not lock, a Transaction for this datum's datastore. More... | |
| def | getprodtype (self) |
| Returns the product type of this Datum. More... | |
| def | getprodname (self) |
| Returns the product name part of the database ID. More... | |
| def | getcategory (self) |
| Returns the product category part of the database ID. More... | |
| def | getlocation (self) |
| Returns the "location" field of this Datum's database entry. More... | |
| def | setlocation (self, v) |
| Sets the "location" field of this Datum's database entry. More... | |
| def | __hash__ (self) |
| Integer hash function. More... | |
| def | __str__ (self) |
| Human-readable description of this Datum. More... | |
| def | __repr__ (self) |
| Python code-like description of this Datum. More... | |
| def | __cmp__ (self, other) |
| Compares two Datums' prodnames and categories. More... | |
| def | set_loc_avail (self, loc, avail) |
| Sets the location and availability of this Datum in a single transaction. More... | |
| def | update (self) |
| Discards all cached metadata and refreshes it from the Datastore. More... | |
| def | __getitem__ (self, k) |
| Returns the value of the specified metadata key or raises KeyError. More... | |
| def | meta |
| Return the value of a metadata key. More... | |
| def | get |
| Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More... | |
| def | __setitem__ (self, k, v) |
| Sets the value of the specified metadata key. More... | |
| def | __delitem__ (self, k) |
| Deletes the specified metadata key, which must not be "available" or "location". More... | |
| def | __contains__ (self, k) |
| Determines if a metadata key is set. More... | |
| def | iteritems (self) |
| Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More... | |
Properties | |
| state | |
| Read-write property: the job state. More... | |
| taskname | |
| Read-only property: the name of this task. More... | |
| Properties inherited from produtil.datastore.Datum | |
| prodname = property(getprodname,None,None,) | |
| Read-only property, an alias for getprodname(): the product name part of the database ID. More... | |
| category = property(getcategory,None,None,) | |
| Read-only property, an alias for getcategory(), the category name part of the database ID. More... | |
| prodtype | |
| Read-only property, an alias for getprodtype(), the product type. More... | |
| did | |
| Read-only property, an alias for getid(). More... | |
| dstore | |
| Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More... | |
| location | |
| Read-write property, an alias for getlocation() and setlocation(). More... | |
| def produtil.datastore.Task.__init__ | ( | self, | |
| dstore, | |||
| taskname, | |||
logger = None, |
|||
| kwargs | |||
| ) |
Task constructor.
Creates a new Task from the given dataset and with the given task name.
| dstore | the Datastore where this Task will live |
| taskname | the task name, passed to the Datum as prodname |
| logger | a logging.Logger for this task to use for log messages |
| kwargs | other keyword arguments are passed to Datum.__init__ |
Definition at line 1060 of file datastore.py.
| def produtil.datastore.Task.clean | ( | self | ) |
Cleans up any unneeded data used by this task.
Subclasses should override this function to clean up any unneeded temporary files or other unused resources consumed by the run() function. This default implementation does nothing.
Definition at line 1152 of file datastore.py.
| def produtil.datastore.Task.completed | ( | self | ) |
Read-only property: is this task completed? Same as is_completed()
True if self.state==COMPLETED, False otherwise.
Definition at line 1189 of file datastore.py.
| def produtil.datastore.Task.getstate | ( | self | ) |
Returns the job state.
Returns the "available" attribute as an integer. This is used as the state of the Task. Typically, the return value should be one of: FAILED, UNSTARTED, RUNNING, PARTIAL, or COMPLETED.
Definition at line 1103 of file datastore.py.
Referenced by hwrf.post.PostOneWRF.run(), and hwrf.post.PostManyWRF.wrftask().
| def produtil.datastore.Task.gettaskname | ( | self | ) |
Returns the task name part of the database ID, which is the same as the prodname.
Definition at line 1127 of file datastore.py.
| def produtil.datastore.Task.is_completed | ( | self | ) |
Is this task complete?
Returns True if this task's state is COMPLETED, and False otherwise.
Definition at line 1180 of file datastore.py.
Referenced by hwrf.gribtask.GRIBTask.run_helper().
| def produtil.datastore.Task.jlogfile | ( | self | ) |
returns the jlogfile logger.
Returns a logging.Logger for the jlogfile. The jlogfile is intended to receive only major errors, and per-job start and completion information. This is equivalent to simply accessing produtil.log.jlogger.
Definition at line 1075 of file datastore.py.
| def produtil.datastore.Task.log | ( | self | ) |
Returns the logger object for this task.
Definition at line 1149 of file datastore.py.
Referenced by scripts.exhwrf_output.Deliverer.deliver_file(), hwrf.tracker.TrackerView.run(), and hwrf.coupling.CouplingStatus.set().
| def produtil.datastore.Task.postmsg | ( | self, | |
| message, | |||
| args, | |||
| kwargs | |||
| ) |
same as produtil.log.jlogger.info()
Sends a message to the multi-job shared log file at level INFO.
| message | the message |
| args | positional arguments for string replacement |
| kwargs | keyword arguments for string replacement. |
Definition at line 1084 of file datastore.py.
Referenced by hwrf.gsi.GSIBase.grab_wrf_enkf(), hwrf.finalmergetask.FinalMergeTask.run(), hwrf.prep.PrepHybrid.run(), hwrf.tracker.TrackerView.run(), hwrf.wps.Geogrid.run(), hwrf.relocate.Stage1.run(), hwrf.wps.Ungrib.run(), hwrf.wps.Metgrid.run(), hwrf.tracker.TrackerTask.run(), hwrf.relocate.Stage2.run(), hwrf.relocate.Stage3.run(), hwrf.relocate.Merge.run(), hwrf.gsi.GSIBase.run_gsi_exe(), and hwrf.post.PostManyWRF.wrftask().
| def produtil.datastore.Task.products | ( | self, | |
| args, | |||
| kwargs | |||
| ) |
Iterate over the products this task produces.
Iterates over some or all of the products produced by this task. The arguments are used to select subsets of the total set of products. Provide no arguments to get the full list of products. All subclasses should re-implement this method, and interpret the arguments in a way that makes sense to that class. The default implementation returns immediately without doing anything.
| args,kwargs | Implementation-defined, used by subclasses. |
Definition at line 1136 of file datastore.py.
| def produtil.datastore.Task.run | ( | self | ) |
Performs the work this Task should do and generates all products.
Performs the work that this task is supposed to do. All subclasses should re-implement this method, and should set the state to COMPLETED the end. This implementation simply calls self.setstate(COMPLETED)
Definition at line 1171 of file datastore.py.
Referenced by produtil.datastore.Task.runpart().
| def produtil.datastore.Task.runpart | ( | self | ) |
Run some of this task's work, deliver some products.
Performs a subset of the work that this task is supposed to do and returns. This is intended to be used for tasks that can be broken up into small pieces, such as post-processing all output files from a NWP simulation one by one. The default implementation simply calls self.run()
Definition at line 1193 of file datastore.py.
| def produtil.datastore.Task.setstate | ( | self, | |
| val | |||
| ) |
Sets the state of this job.
Sets the job stat to the specified value. This works by setting the "available" attribute to the specified integer. For compatibility with other scripts, this should be FAILED, UNSTARTED, RUNNING, PARTIAL or COMPLETED.
| val | the new job state, an int |
Definition at line 1094 of file datastore.py.
Referenced by hwrf.post.PostOneWRF.run(), produtil.datastore.Task.run(), and hwrf.post.PostManyWRF.wrftask().
| def produtil.datastore.Task.strstate | ( | self | ) |
A string representation of the job state.
Definition at line 1118 of file datastore.py.
| def produtil.datastore.Task.unrun | ( | self | ) |
Undoes the effect of run().
Cleans up this Task's work areas, "undelivers" all deliverables, and makes it look like the task has never been run. All subclasses should re-implement this method, and must also "unrun" everything their parent class runs. The default implementation simply calls self.clean() and sets the state to UNSTARTED.
Definition at line 1159 of file datastore.py.
|
static |
Read-write property: the job state.
Can be FAILED, UNSTARTED, RUNNING, PARTIAL or COMPLETED.
Definition at line 1114 of file datastore.py.
Referenced by produtil.datastore.Task.completed(), and produtil.datastore.Task.is_completed().
|
static |
Read-only property: the name of this task.
Same as self.prodname.
Definition at line 1134 of file datastore.py.
Referenced by hwrf.bufrprep.Bufrprep.__init__(), hwrf.ww3.WW3Init.__init__(), hwrf.nhc_products.NHCProducts.__init__(), hwrf.gsi.GSIBase.__init__(), hwrf.post.PostOneWRF.__init__(), hwrf.wrf.ExternalWRFTask.__init__(), hwrf.fcsttask.WRFAtmos.__init__(), hwrf.relocate.Stage3.__init__(), hwrf.relocate.Merge.__init__(), hwrf.gsipost.GSIPost.add_case(), hwrf.multistorm.ForecastWatcher.as_product(), hwrf.wrf.ExternalWRFTask.as_product(), hwrf.fcsttask.AnalysisCycle.as_product(), hwrf.post.PostOneWRF.can_run(), hwrf.multistorm.WRFCopyTaskMS.comfile(), hwrf.copywrf.WRFCopyTask.d_final(), hwrf.copywrf.WRFCopyTask.d_wrfprod(), hwrf.copywrf.WRFCopyTask.deliver_group(), hwrf.finalmergetask.FinalMergeTask.deliver_products(), hwrf.relocate.RelocationTask.deliver_products(), hwrf.init.FGATInit.fhr_and_init(), hwrf.mpipomtc.POMInit.get_inputs(), hwrf.hwrftask.HWRFTask.get_outdir(), hwrf.ensda.FromPriorCycle.get_product(), hwrf.hwrftask.HWRFTask.get_workdir(), hwrf.wps.WPSTask.link_geogrid(), hwrf.hwrftask.HWRFTask.log(), hwrf.hycom.HYCOMInit.make_products(), hwrf.gribtask.GRIBTask.make_products(), hwrf.wps.Geogrid.make_products(), hwrf.wps.Metgrid.make_products(), hwrf.fcsttask.WRFAnl.make_products(), hwrf.post.PostOneWRF.product_name(), hwrf.gribtask.GRIBTask.products(), hwrf.mpipomtc.POMInit.run(), hwrf.finalmergetask.FinalMergeTask.run(), hwrf.prep.PrepHybrid.run(), hwrf.ensda.FromGFSENKF.run(), hwrf.tracker.TrackerView.run(), hwrf.wps.Geogrid.run(), hwrf.wps.Ungrib.run(), hwrf.wps.Metgrid.run(), hwrf.tracker.TrackerTask.run(), hwrf.gribtask.GRIBTask.run_helper(), hwrf.tracker.TrackerTask.send_atcfunix_subset(), hwrf.tracker.TrackerTask.send_raw_atcfunix(), hwrf.multistorm.FakeInit.sim(), hwrf.post.PostManyWRF.taskname_for(), and hwrf.post.PostManyWRF.wrftask().
1.8.10
