Superclass of anything that can be stored in a Datastore. More...

Detailed Description

Superclass of anything that can be stored in a Datastore.

This is the superclass of anything that can be placed in a datastore. It has a category, a product name (prodname for short), a location, availability (an int) and arbitrary metadata (key,value) pairs. It caches database metadata in self._meta, which is directly accessed by the Datastore class. Cache data will be discarded once its age is older than self._cacheage.

Definition at line 426 of file datastore.py.

Inheritance diagram for produtil.datastore.Datum:

Public Member Functions
def	__init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs)
	Datum constructor. More...

def	__enter__ (self)
	Acquires this object's thread lock. More...

def	__exit__ (self, etype, evalue, traceback)
	Releases this object's thread lock. More...

def	validate (self)
	Validates this object's Datastore, prodname and category. More...

def	getid (self)
	Returns the database ID of this datum. More...

def	getdatastore (self)
	Returns the datastore of this datum. More...

def	transaction (self)
	Creates, but does not lock, a Transaction for this datum's datastore. More...

def	getprodtype (self)
	Returns the product type of this Datum. More...

def	getprodname (self)
	Returns the product name part of the database ID. More...

def	getcategory (self)
	Returns the product category part of the database ID. More...

def	getlocation (self)
	Returns the "location" field of this Datum's database entry. More...

def	setlocation (self, v)
	Sets the "location" field of this Datum's database entry. More...

def	__hash__ (self)
	Integer hash function. More...

def	__str__ (self)
	Human-readable description of this Datum. More...

def	__repr__ (self)
	Python code-like description of this Datum. More...

def	__cmp__ (self, other)
	Compares two Datums' prodnames and categories. More...

def	set_loc_avail (self, loc, avail)
	Sets the location and availability of this Datum in a single transaction. More...

def	update (self)
	Discards all cached metadata and refreshes it from the Datastore. More...

def	__getitem__ (self, k)
	Returns the value of the specified metadata key or raises KeyError. More...

def	meta
	Return the value of a metadata key. More...

def	get
	Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More...

def	__setitem__ (self, k, v)
	Sets the value of the specified metadata key. More...

def	__delitem__ (self, k)
	Deletes the specified metadata key, which must not be "available" or "location". More...

def	__contains__ (self, k)
	Determines if a metadata key is set. More...

def	iteritems (self)
	Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More...

Properties
	prodname = property(getprodname,None,None,)
	Read-only property, an alias for getprodname(): the product name part of the database ID. More...

	category = property(getcategory,None,None,)
	Read-only property, an alias for getcategory(), the category name part of the database ID. More...

	prodtype
	Read-only property, an alias for getprodtype(), the product type. More...

	did
	Read-only property, an alias for getid(). More...

	dstore
	Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More...

	location
	Read-write property, an alias for getlocation() and setlocation(). More...

Constructor & Destructor Documentation

def produtil.datastore.Datum.__init__	(	self,
		dstore,
		prodname,
		category,
		meta = `None`,
		cache = `30`,
		location = `None`,
		kwargs
	)

Datum constructor.

Creates a Datum in the given Datastore dstore, under the specified category and product name prodname. The datastore id used is "category::prodname". The value for "cache" is the number of seconds to retain cached metadata before going back to disk to reread it. That only applies to data "get" operations: setting a data or metadata value will cause an immediate write to the database. Also, contains ("var" in self) will force a fetch from the database if the requested metadata variable is not in the cached copy of the database.

Values for location and meta are only the DEFAULT values, and will be ignored if other values are already set in the database. The location is only used if the product is not already in the database: its location will be set to the provided values upon entry. Similarly, the metadata is only set in this call if there isn't already metadata for the product with the given metadata keys.

Parameters

dstore	The Datastore for this Datum.
prodname	The product name portion of the Datum ID
category	The category part of the Datum ID
meta	A dict of metadata values.
cache	Metadata cache lifetime in seconds.
location	The initial value for location, if it is not set already in the database.
kwargs	Ignored.

Definition at line 435 of file datastore.py.

Member Function Documentation

def produtil.datastore.Datum.__cmp__	(	self,
		other
	)

Compares two Datums' prodnames and categories.

Parameters

other the other datum to compare against

Definition at line 576 of file datastore.py.

def produtil.datastore.Datum.__contains__	(	self,
		k
	)

Determines if a metadata key is set.

Returns: True if the specified metadata key is set, and False otherwise. Immediately returns True for 'available' and 'location' without checking the metadata cache.

Parameters

k	the key of interest

Definition at line 667 of file datastore.py.

def produtil.datastore.Datum.__delitem__	(	self,
		k
	)

Deletes the specified metadata key, which must not be "available" or "location".

Parameters

k	the key to delete

Definition at line 657 of file datastore.py.

def produtil.datastore.Datum.__enter__ ( self )

Acquires this object's thread lock.

This is used to manage cached data.

Definition at line 484 of file datastore.py.

def produtil.datastore.Datum.__exit__	(	self,
		etype,
		evalue,
		traceback
	)

Releases this object's thread lock.

This is used to manage cached data.

Parameters

etype,evalue,traceback Exception information.

Definition at line 488 of file datastore.py.

def produtil.datastore.Datum.__getitem__	(	self,
		k
	)

Returns the value of the specified metadata key or raises KeyError.

Forces a cache update if k is not in the cache.

Definition at line 619 of file datastore.py.

def produtil.datastore.Datum.__hash__ ( self )

Integer hash function.

Definition at line 566 of file datastore.py.

def produtil.datastore.Datum.__repr__ ( self )

Python code-like description of this Datum.

Definition at line 572 of file datastore.py.

Referenced by produtil.prog.Runner.__str__().

def produtil.datastore.Datum.__setitem__	(	self,
		k,
		v
	)

Sets the value of the specified metadata key.

Parameters

k	the key
v	the value

Definition at line 649 of file datastore.py.

def produtil.datastore.Datum.__str__ ( self )

Human-readable description of this Datum.

Definition at line 569 of file datastore.py.

def produtil.datastore.Datum.get	(	self,
		k,
		default = `None`
	)

Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset.

Does NOT force a cache update if k is missing from the cache. To force a cache update, use getitem instead.

Parameters

k	The key of interest.
default	The value to return if no value is seen.

Returns: The metadata value or the default.

Definition at line 637 of file datastore.py.

Referenced by produtil.datastore.UpstreamFile.check(), hwrf.tracker.CleanATCFSubsetProduct.deliver(), hwrf.regrib.GRIB1Product.getgrib1grbindex(), hwrf.regrib.GRIB1Product.getgrib1grid(), hwrf.regrib.GRIB1Product.getgrib1index(), hwrf.regrib.GRIB2Product.getgrib2grid(), hwrf.regrib.GRIB2Product.getgrib2index(), produtil.datastore.Datum.meta(), and hwrf.gribtask.GRIBTask.products().

def produtil.datastore.Datum.getcategory ( self )

Returns the product category part of the database ID.

Definition at line 520 of file datastore.py.

def produtil.datastore.Datum.getdatastore ( self )

Returns the datastore of this datum.

Definition at line 504 of file datastore.py.

def produtil.datastore.Datum.getid ( self )

Returns the database ID of this datum.

Definition at line 501 of file datastore.py.

def produtil.datastore.Datum.getlocation ( self )

Returns the "location" field of this Datum's database entry.

Definition at line 524 of file datastore.py.

def produtil.datastore.Datum.getprodname ( self )

Returns the product name part of the database ID.

Definition at line 517 of file datastore.py.

def produtil.datastore.Datum.getprodtype ( self )

Returns the product type of this Datum.

Returns the product type of this Datum. This is generally the name of the Python class that created the entry in the database.

Definition at line 510 of file datastore.py.

def produtil.datastore.Datum.iteritems ( self )

Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location".

Definition at line 677 of file datastore.py.

Referenced by hwrf.ensda.DAEnsemble.members_at_time(), and hwrf.gribtask.GRIBTask.products().

def produtil.datastore.Datum.meta	(	self,
		k,
		default = `None`
	)

Return the value of a metadata key.

Returns the value of the specified metadata key or returns default if it is unset. Does NOT force a cache update if k is missing from the cache. To force a cache update, use getitem instead.

Parameters

k	The key of interest.
default	The value to return if no value is seen.

Returns: The metadata value or the default.

Definition at line 625 of file datastore.py.

Referenced by hwrf.hwrftask.HWRFTask.__init__(), hwrf.hwrftask.HWRFTask.get_outdir(), and hwrf.hwrftask.HWRFTask.get_workdir().

def produtil.datastore.Datum.set_loc_avail	(	self,
		loc,
		avail
	)

Sets the location and availability of this Datum in a single transaction.

Parameters

loc	the new location, a string
avail	the new availability, an int

Definition at line 583 of file datastore.py.

Referenced by produtil.datastore.UpstreamFile.check(), hwrf.post.EGRIB1Product.deliver(), and produtil.datastore.FileProduct.deliver().

def produtil.datastore.Datum.setlocation	(	self,
		v
	)

Sets the "location" field of this Datum's database entry.

Parameters

v	the new location

Definition at line 527 of file datastore.py.

def produtil.datastore.Datum.transaction ( self )

Creates, but does not lock, a Transaction for this datum's datastore.

Definition at line 507 of file datastore.py.

Referenced by produtil.datastore.Datum.__delitem__(), produtil.datastore.Datum.__setitem__(), and produtil.datastore.Datum.set_loc_avail().

def produtil.datastore.Datum.update ( self )

Discards all cached metadata and refreshes it from the Datastore.

Definition at line 614 of file datastore.py.

Referenced by produtil.datastore.Product.check(), and hwrf.post.EGRIB1Product.deliver().

def produtil.datastore.Datum.validate ( self )

Validates this object's Datastore, prodname and category.

Definition at line 493 of file datastore.py.

Referenced by produtil.datastore.Datum.__init__(), and produtil.mpiprog.MPIRank.__init__().

Property Documentation

produtil.datastore.Datum.category = property(getcategory,None,None,)

static

Read-only property, an alias for getcategory(), the category name part of the database ID.

Definition at line 540 of file datastore.py.

Referenced by hwrf.post.EGRIB1Product.make_location().

produtil.datastore.Datum.did

static

Initial value:

1 = property(getid,None,None,

2 )

Read-only property, an alias for getid().

The full database ID.

Definition at line 551 of file datastore.py.

Referenced by produtil.datastore.Datum.__str__(), produtil.datastore.Product.call_callbacks(), hwrf.tracker.RawATCFProduct.deliver(), hwrf.tracker.CleanATCFSubsetProduct.deliver(), produtil.datastore.FileProduct.deliver(), hwrf.regrib.GRIB2Product.deliver(), hwrf.regrib.GRIB1Product.deliver(), and produtil.datastore.Datum.set_loc_avail().

produtil.datastore.Datum.dstore

static

Initial value:

1 = property(getdatastore,None,None,

2 )

Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides.

Definition at line 557 of file datastore.py.

Referenced by hwrf.ww3.WW3Init.__init__(), hwrf.nhc_products.NHCProducts.__init__(), hwrf.post.PostOneWRF.__init__(), hwrf.fcsttask.WRFAtmos.__init__(), produtil.datastore.Datum.__repr__(), hwrf.gsipost.GSIPost.add_case(), hwrf.multistorm.ForecastWatcher.as_product(), hwrf.wrf.ExternalWRFTask.as_product(), hwrf.fcsttask.AnalysisCycle.as_product(), hwrf.multistorm.WRFCopyTaskMS.comfile(), hwrf.copywrf.WRFCopyTask.d_final(), hwrf.copywrf.WRFCopyTask.d_wrfprod(), hwrf.init.FGATInit.fhr_and_init(), hwrf.mpipomtc.POMInit.get_inputs(), hwrf.ensda.FromPriorCycle.get_product(), hwrf.hwrfsystem.HWRFForecastPostProcessing.make_extra_trackers(), hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker(), hwrf.hwrfsystem.HWRFGSIPostProcessing.make_gsi_post(), hwrf.hycom.HYCOMInit.make_products(), hwrf.prep.PrepHybrid.make_products(), hwrf.wps.Metgrid.make_products(), hwrf.fcsttask.WRFAnl.make_products(), hwrf.hwrfsystem.HWRFForecastPostProcessing.make_wrfcopier(), hwrf.gribtask.GRIBTask.products(), hwrf.mpipomtc.POMInit.run(), hwrf.ensda.FromGFSENKF.run(), hwrf.tracker.TrackerView.run(), hwrf.tracker.TrackerTask.send_atcfunix_subset(), hwrf.tracker.TrackerTask.send_raw_atcfunix(), and hwrf.multistorm.FakeInit.sim().

produtil.datastore.Datum.location

static

Initial value:

1 = property(getlocation,setlocation,None,

2 )

Read-write property, an alias for getlocation() and setlocation().

The location on disk of this Datum.

Definition at line 563 of file datastore.py.

Referenced by hwrf.relocate.Stage3.__init__(), hwrf.relocate.Merge.__init__(), produtil.datastore.UpstreamFile.check(), hwrf.tracker.RawATCFProduct.deliver(), hwrf.tracker.CleanATCFSubsetProduct.deliver(), produtil.datastore.FileProduct.deliver(), hwrf.relocate.RelocationTask.deliver_products(), hwrf.post.EGRIB1Product.make(), hwrf.regrib.UpstreamGRIB1.make(), produtil.datastore.FileProduct.undeliver(), and hwrf.wrf.ExternalWRFTask.update_state().

produtil.datastore.Datum.prodname = property(getprodname,None,None,)

static

Read-only property, an alias for getprodname(): the product name part of the database ID.

Definition at line 535 of file datastore.py.

Referenced by hwrf.post.EGRIB1Product.make_location().

produtil.datastore.Datum.prodtype

static

Initial value:

1 = property(getprodtype,None,None,

2 )

Read-only property, an alias for getprodtype(), the product type.

This is generally the name of the Python class that created the entry in the database.

Definition at line 546 of file datastore.py.

Referenced by produtil.datastore.Datum.__repr__(), and produtil.datastore.Datum.__str__().

The documentation for this class was generated from the following file:

/mnt/pan2/projects/hwrfv3/Samuel.Trahan/trunk/ush/produtil/datastore.py

Detailed Description

Public Member Functions

Properties

Constructor & Destructor Documentation

Member Function Documentation

Property Documentation