|
HWRF
trunk@4391
|
A piece of data produced by a Task. More...
A piece of data produced by a Task.
A Product is a piece of data that can be produced by a Task. Once the product is available, self.available or self.is_available() will be True, and the self.location will be valid. The meaning of self.location is up to the implementer to decide, but it should be a full path to a location on disk for file products. As with all Datum objects, a Product also has arbitrary metadata.
Definition at line 716 of file datastore.py.
Public Member Functions | |
| def | add_callback |
| Adds a delivery callback function. More... | |
| def | has_callbacks (self) |
| Returns True if this Product has any callback functions and False otherwise. | |
| def | call_callbacks |
| Calls all delivery callback functions. More... | |
| def | check (self, kwargs) |
| Asks the product to check its own availability and update the database. More... | |
| def | deliver (self, kwargs) |
| Asks the Product to deliver itself. More... | |
| def | undeliver (self, kwargs) |
| "Undelivers" a product. More... | |
| def | setavailable (self, val) |
| Sets the availability to the specified value. More... | |
| def | is_available (self) |
| Is the product available? More... | |
| def | validate (self) |
| Validates this object's Datastore, prodname and category. More... | |
 Public Member Functions inherited from produtil.datastore.Datum | |
| def | __init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs) |
| Datum constructor. More... | |
| def | __enter__ (self) |
| Acquires this object's thread lock. More... | |
| def | __exit__ (self, etype, evalue, traceback) |
| Releases this object's thread lock. More... | |
| def | validate (self) |
| Validates this object's Datastore, prodname and category. More... | |
| def | getid (self) |
| Returns the database ID of this datum. More... | |
| def | getdatastore (self) |
| Returns the datastore of this datum. More... | |
| def | transaction (self) |
| Creates, but does not lock, a Transaction for this datum's datastore. More... | |
| def | getprodtype (self) |
| Returns the product type of this Datum. More... | |
| def | getprodname (self) |
| Returns the product name part of the database ID. More... | |
| def | getcategory (self) |
| Returns the product category part of the database ID. More... | |
| def | getlocation (self) |
| Returns the "location" field of this Datum's database entry. More... | |
| def | setlocation (self, v) |
| Sets the "location" field of this Datum's database entry. More... | |
| def | __hash__ (self) |
| Integer hash function. More... | |
| def | __str__ (self) |
| Human-readable description of this Datum. More... | |
| def | __repr__ (self) |
| Python code-like description of this Datum. More... | |
| def | __cmp__ (self, other) |
| Compares two Datums' prodnames and categories. More... | |
| def | set_loc_avail (self, loc, avail) |
| Sets the location and availability of this Datum in a single transaction. More... | |
| def | update (self) |
| Discards all cached metadata and refreshes it from the Datastore. More... | |
| def | __getitem__ (self, k) |
| Returns the value of the specified metadata key or raises KeyError. More... | |
| def | meta |
| Return the value of a metadata key. More... | |
| def | get |
| Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More... | |
| def | __setitem__ (self, k, v) |
| Sets the value of the specified metadata key. More... | |
| def | __delitem__ (self, k) |
| Deletes the specified metadata key, which must not be "available" or "location". More... | |
| def | __contains__ (self, k) |
| Determines if a metadata key is set. More... | |
| def | iteritems (self) |
| Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More... | |
Properties | |
| available | |
| Read-write property: is the product available? More... | |
| Properties inherited from produtil.datastore.Datum | |
| prodname = property(getprodname,None,None,) | |
| Read-only property, an alias for getprodname(): the product name part of the database ID. More... | |
| category = property(getcategory,None,None,) | |
| Read-only property, an alias for getcategory(), the category name part of the database ID. More... | |
| prodtype | |
| Read-only property, an alias for getprodtype(), the product type. More... | |
| did | |
| Read-only property, an alias for getid(). More... | |
| dstore | |
| Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More... | |
| location | |
| Read-write property, an alias for getlocation() and setlocation(). More... | |
| def produtil.datastore.Product.add_callback | ( | self, | |
| callback, | |||
args = None, |
|||
states = None |
|||
| ) |
Adds a delivery callback function.
Adds a delivery callback that is called when the product is delivered. This is intended to do such tasks as running an NCO dbn_alert, or copying to a website, or emailing someone. This function is only added in this local Python instance, not in the database file. Also, it is the responsibility of the subclasses to call self.call_callbacks() from self.deliver() to ensure the callbacks are run.
Example:
| callback | The callback function, which must be able to take any keyword or indexed arguments. |
| args | The indexed arguments to send. |
| states | Presently unused. |
Definition at line 725 of file datastore.py.
| def produtil.datastore.Product.call_callbacks | ( | self, | |
logger = None |
|||
| ) |
Calls all delivery callback functions.
Calls all data delivery callbacks for this Product. Collects any raised Exception subclasses until after all callbacks are called. Will raise CallbackExceptions if any exceptions are caught.
Subclasses should call this from either check, or deliver, as appropriate for the product type.
| logger | Optional: the logging.Logger for logging messages. |
Definition at line 759 of file datastore.py.
Referenced by produtil.datastore.UpstreamFile.check(), hwrf.post.EGRIB1Product.deliver(), hwrf.tracker.RawATCFProduct.deliver(), hwrf.tracker.CleanATCFSubsetProduct.deliver(), produtil.datastore.FileProduct.deliver(), hwrf.regrib.GRIB2Product.deliver(), and hwrf.regrib.GRIB1Product.deliver().
| def produtil.datastore.Product.check | ( | self, | |
| kwargs | |||
| ) |
Asks the product to check its own availability and update the database.
Checks to see if this product is available. This is generally not a cheap operation, as it can take seconds or minutes and may fail. One should call "available" instead if cached information is sufficient.
| kwargs | Additional keyword arguments are unused. This is for use by subclasses. |
Definition at line 786 of file datastore.py.
Referenced by produtil.fileop.FileWaiter.checkfiles(), hwrf.regrib.UpstreamGRIB1.is_ready(), hwrf.regrib.GRIB2Product.is_ready(), and hwrf.regrib.GRIB1Product.is_ready().
| def produtil.datastore.Product.deliver | ( | self, | |
| kwargs | |||
| ) |
Asks the Product to deliver itself.
Delivers a product to its destination. This is not implemented by the base class. Note that this is generally an expensive operation which may take seconds or minutes, and may fail. It may involve copying many files, network access, or even pulling tapes from a silo. In the end, the location and availability are expected to be updated in the database.
| kwargs | Unused, to be used by subclasses. |
Definition at line 798 of file datastore.py.
| def produtil.datastore.Product.is_available | ( | self | ) |
Is the product available?
Returns the "available" attribute of this Product in the database, converted to a boolean value via bool()
Definition at line 831 of file datastore.py.
| def produtil.datastore.Product.setavailable | ( | self, | |
| val | |||
| ) |
Sets the availability to the specified value.
Sets the "available" attribute of this Product in the database after converting the given value to a bool and then int (int(bool(val))).
| val | the new availability |
Definition at line 823 of file datastore.py.
| def produtil.datastore.Product.undeliver | ( | self, | |
| kwargs | |||
| ) |
"Undelivers" a product.
The meaning of this function is implementation-dependent: it could mean deleting an output file, or any number of other actions. Regardless, it should result in self.available=False or an exception being thrown. Note that this is generally an expensive operation that could take seconds or minutes, and may fail. The default implementation simply sets available to False.
Definition at line 810 of file datastore.py.
| def produtil.datastore.Product.validate | ( | self | ) |
Validates this object's Datastore, prodname and category.
Validates the Datastore, prodname and category of this Product. In addition to the requirements made by Datum, this function requires that the category not contain any double stars ("**").
Definition at line 843 of file datastore.py.
Referenced by produtil.mpiprog.MPIRank.__init__().
|
static |
Read-write property: is the product available?
Definition at line 840 of file datastore.py.
Referenced by produtil.datastore.Product.check(), produtil.datastore.UpstreamFile.check(), produtil.datastore.FileProduct.deliver(), produtil.datastore.Product.undeliver(), produtil.datastore.FileProduct.undeliver(), and produtil.datastore.UpstreamFile.undeliver().
1.8.10
