hwrf.gsi.GSIBase Class Reference

Base class of anything that runs the GSI. More...

Detailed Description

Base class of anything that runs the GSI.

Do not use directly.

Definition at line 46 of file gsi.py.

Inheritance diagram for hwrf.gsi.GSIBase:

Public Member Functions
def	__init__ (self, dstore, conf, section, domain, wrf_in_prod, sim, taskname=None, atime=None, parent_atime=None, enkf_domains=None, ensda=None, kwargs)
	The GSIBase constructor: More...
def	wrf_top_Pa (self)
	The WRF model top in pascals.
def	atime (self)
	The analysis time of this GSI. More...
def	parent_atime (self)
	Parent model analysis time. More...
def	set_ensda (self, ensda, enkf_domains)
	Sets the hwrf.ensda.DAEnsemble to use, and the enkf domains. More...
def	inputiter (self)
	Iterate over needed inputs. More...
def	grab_enkf_input (self)
	Link or copy ensemble inputs. More...
def	grab_bufr
	Link bufr files. More...
def	grab_obstype_section
	Copies or links observations. More...
def	grab_prepbufr (self, atime=None, kwargs)
	Links or copies the prepbufr file to the local directory. More...
def	write_vitals
	Writes the tcvitals (from self.storminfo) to the specified file. More...
def	wrfout_copier (self, file)
	Generate the wrfout file converter. More...
def	grab_wrf_enkf (self, ensda)
	Links the WRF ENKF files to this directory. More...
def	grab_gfs_enkf (self, atime=None, kwargs)
	Links the GFS ENKF files to this directory. More...
def	copy_wrf_inout
	Copies the WRF analysis or input file to the specified filename. More...
def	get_ghost (self, domain)
	Obtain output ghost product for the specified domain. More...
def	get_wrfanl (self, domain)
	Obtain output ghost product for the specified domain. More...
def	get_wrfinput (self)
	Obtain output wrfinput data for the outermost WRF domain. More...
def	domain (self)
	The WRF domain for which GSI is being run. More...
def	products
	Iterates over all output products of this Task. More...
def	make_gsi_namelist
	Creates the GSI namelist in the specified file. More...
def	after_gsi (self)
	Called by run() after the gsi executable completes. More...
def	before_gsi (self)
	Called by run() just before running the gsi program. More...
def	grab_more_inputs (self)
	Called by run() to obtain additional inputs before before_gsi() More...
def	grab_bias_satang (self)
	Copies or links bias correction and satellite angle files.
def	run_gsi_exe (self)
	Runs the actual GSI executable. More...
def	deliver_products (self)
	Delivers output products. More...
def	make_diag_files (self, tgtpre, nthreads)
	Creates GSI diagnostic files. More...
def	run (self)
	Runs the GSI and delivers the results. More...
def	grab_fix_parm (self)
	Links or copies to the local directory any fix or parm files needed by GSI. More...
Public Member Functions inherited from hwrf.hwrftask.HWRFTask
def	__init__ (self, dstore, conf, section, taskname=None, workdir=None, outdir=None, storminfo=UNSPECIFIED, taskvars=UNSPECIFIED, kwargs)
	Creates an HWRFTask. More...
def	isfakestorm (self)
def	ismultistorm (self)
def	get_workdir (self)
	Returns the directory the class should work in, as set by the "workdir" metadata value. More...
def	set_workdir (self, val)
	Sets the directory the class should work in. More...
def	get_outdir (self)
	Gets the directory that should receive output data. More...
def	set_outdir (self, val)
	Sets the directory that should receive output data. More...
def	realtime (self)
	Is this job a real-time forecast job? More...
def	redirect (self)
	Should subprograms' outputs be redirected to separate files?
def	scrub (self)
	Should temporary files be deleted as soon as they are not needed?
def	tvset (self, opt, val)
	Sets a taskvar option's value. More...
def	tvdel (self, opt)
	Deletes an object-local value set by tvset. More...
def	tvget (self, opt)
	Gets a taskvar's value. More...
def	tvhave
	Is a taskvar set? More...
def	taskvars (self)
	The dict of object-local values used for string substitution. More...
def	confint
	Alias for self.conf.getint for section self.section. More...
def	confstr
	Alias for self.conf.getstr for section self.section. More...
def	conffloat
	Alias for self.conf.getfloat for section self.section. More...
def	confbool
	Alias for self.conf.getbool for section self.section. More...
def	confget
	Alias for self.conf.get for section self.section. More...
def	confitems
	Alias for self.conf.items for section self.section. More...
def	confstrinterp (self, string, section=None, kwargs)
	Alias for self.icstr for backward compatibility. More...
def	conftimestrinterp (self, string, ftime, atime=None, section=None, kwargs)
	Alias for self.timestr for backward comaptibility. More...
def	confraw
	Get a raw configuration value before string expansion. More...
def	icstr (self, string, section=None, kwargs)
	Expands a string in the given conf section. More...
def	timestr (self, string, ftime, atime=None, section=None, kwargs)
	Expands a string in the given conf section, including time vars. More...
def	getdir
	Alias for hwrf.config.HWRFConfig.get() for the "dir" section. More...
def	getexe
	Alias for hwrf.config.HWRFConfig.get() for the "exe" section. More...
def	getconf (self)
	Returns this HWRFTask's hwrf.config.HWRFConfig object. More...
def	getsection (self)
	Returns this HWRFTask's section name in the HWRFConfig. More...
def	log
	Obtain a logging domain. More...
def	inputiter (self)
	Iterates over all inputs required by this task. More...
Public Member Functions inherited from produtil.datastore.Task
def	__init__ (self, dstore, taskname, logger=None, kwargs)
	Task constructor. More...
def	jlogfile (self)
	returns the jlogfile logger. More...
def	postmsg (self, message, args, kwargs)
	same as produtil.log.jlogger.info() More...
def	setstate (self, val)
	Sets the state of this job. More...
def	getstate (self)
	Returns the job state. More...
def	strstate (self)
	A string representation of the job state. More...
def	gettaskname (self)
	Returns the task name part of the database ID, which is the same as the prodname. More...
def	products (self, args, kwargs)
	Iterate over the products this task produces. More...
def	log (self)
	Returns the logger object for this task. More...
def	clean (self)
	Cleans up any unneeded data used by this task. More...
def	unrun (self)
	Undoes the effect of run(). More...
def	run (self)
	Performs the work this Task should do and generates all products. More...
def	is_completed (self)
	Is this task complete? More...
def	completed (self)
	Read-only property: is this task completed? Same as is_completed() More...
def	runpart (self)
	Run some of this task's work, deliver some products. More...
Public Member Functions inherited from produtil.datastore.Datum
def	__init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs)
	Datum constructor. More...
def	__enter__ (self)
	Acquires this object's thread lock. More...
def	__exit__ (self, etype, evalue, traceback)
	Releases this object's thread lock. More...
def	validate (self)
	Validates this object's Datastore, prodname and category. More...
def	getid (self)
	Returns the database ID of this datum. More...
def	getdatastore (self)
	Returns the datastore of this datum. More...
def	transaction (self)
	Creates, but does not lock, a Transaction for this datum's datastore. More...
def	getprodtype (self)
	Returns the product type of this Datum. More...
def	getprodname (self)
	Returns the product name part of the database ID. More...
def	getcategory (self)
	Returns the product category part of the database ID. More...
def	getlocation (self)
	Returns the "location" field of this Datum's database entry. More...
def	setlocation (self, v)
	Sets the "location" field of this Datum's database entry. More...
def	__hash__ (self)
	Integer hash function. More...
def	__str__ (self)
	Human-readable description of this Datum. More...
def	__repr__ (self)
	Python code-like description of this Datum. More...
def	__cmp__ (self, other)
	Compares two Datums' prodnames and categories. More...
def	set_loc_avail (self, loc, avail)
	Sets the location and availability of this Datum in a single transaction. More...
def	update (self)
	Discards all cached metadata and refreshes it from the Datastore. More...
def	__getitem__ (self, k)
	Returns the value of the specified metadata key or raises KeyError. More...
def	meta
	Return the value of a metadata key. More...
def	get
	Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More...
def	__setitem__ (self, k, v)
	Sets the value of the specified metadata key. More...
def	__delitem__ (self, k)
	Deletes the specified metadata key, which must not be "available" or "location". More...
def	__contains__ (self, k)
	Determines if a metadata key is set. More...
def	iteritems (self)
	Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More...

Public Attributes
	used_regional_ensemble
	Was the regional ensemble used for forecast error covariances?
	global_ensemble_size
	The number of members in the global (parent model) ensemble.
	hybrid_da
	Was hybrid 3DVAR-ENKF data assimilation used?
	state
Public Attributes inherited from hwrf.hwrftask.HWRFTask
	storminfo
	The hwrf.storminfo.StormInfo describing the vitals information for the storm processed by this HWRFTask. More...

Additional Inherited Members
Properties inherited from hwrf.hwrftask.HWRFTask
	workdir
	The directory in which this task should be run. More...
	outdir
	The directory in which this task should deliver its final output. More...
	conf
	This HWRFTask's hwrf.config.HWRFConfig object. More...
	section
	The confsection in self.section for this HWRFTask (read-only) More...
Properties inherited from produtil.datastore.Task
	state
	Read-write property: the job state. More...
	taskname
	Read-only property: the name of this task. More...
Properties inherited from produtil.datastore.Datum
	prodname = property(getprodname,None,None,)
	Read-only property, an alias for getprodname(): the product name part of the database ID. More...
	category = property(getcategory,None,None,)
	Read-only property, an alias for getcategory(), the category name part of the database ID. More...
	prodtype
	Read-only property, an alias for getprodtype(), the product type. More...
	did
	Read-only property, an alias for getid(). More...
	dstore
	Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More...
	location
	Read-write property, an alias for getlocation() and setlocation(). More...

Constructor & Destructor Documentation

def hwrf.gsi.GSIBase.__init__	(		self,
			dstore,
			conf,
			section,
			domain,
			wrf_in_prod,
			sim,
			taskname = None,
			atime = None,
			parent_atime = None,
			enkf_domains = None,
			ensda = None,
			kwargs
	)

The GSIBase constructor:

Parameters

dstore	passed to Datum: the Datastore object for this Task
conf	the conf object for this task (passed to HWRFTask)
section	the conf section for this task (passed to HWRFTask)
domain	the WRFDomain for this GSI. Must have been initialized by a WRFSimulation
wrf_in_prod	the Product for the wrfinput_d01 or ghost_d0* file for that domain
sim	the hwrf.wrf.WRFSimulation that will be run as the forecast
taskname	Optional: the taskname for this product in the datastore
atime	the analysis time as a datetime.datetime. Default: conf.cycle
parent_atime	the analysis time of the parent model. This is relevant if the parent model's forecast is used as the background.
enkf_domains	a list of WRF domains that should be copied from the hwrf.ensda.EnsembleDA.
ensda	a subclass of hwrf.ensda.DAEnsemble that provides regional ensemble forecasts to generate the forecast error covariance
kwargs	ignored; passed to HWRFTask

Definition at line 60 of file gsi.py.

Member Function Documentation

def hwrf.gsi.GSIBase.after_gsi

(

self

)

Called by run() after the gsi executable completes.

This is intended to be overridden by subclasses to perform some action after gsi is complete, but before products are delivered. The default implementation does nothing.

Definition at line 756 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.atime

(

self

)

The analysis time of this GSI.

Definition at line 166 of file gsi.py.

Referenced by hwrf.gsi.FGATGSI.copy_wrf_inout(), hwrf.gsi.FGATGSI.fgat_fhrs(), hwrf.gsi.GSIBase.grab_gfs_enkf(), hwrf.gsi.GSIBase.grab_obstype_section(), hwrf.gsi.GSIBase.grab_prepbufr(), hwrf.gsi.GSIBase.grab_wrf_enkf(), hwrf.gsi.GSIBase.inputiter(), and hwrf.gsi.GSIBase.make_gsi_namelist().

def hwrf.gsi.GSIBase.before_gsi

(

self

)

Called by run() just before running the gsi program.

This is intended to be overridden by subclasses to perform some action after all inputs needed for gsi are available, but before gsi starts. The default implementation does nothing.

Definition at line 763 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.copy_wrf_inout	(		self,
			filename = 'wrf_inout'
	)

Copies the WRF analysis or input file to the specified filename.

Parameters

filename

the file to receive the data

Definition at line 630 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.deliver_products

(

self

)

Delivers output products.

This function is called by run() to deliver output files to the intercom or com directory and record in the database that they are delivered.

Definition at line 834 of file gsi.py.

Referenced by hwrf.mpipomtc.POMInit.run(), hwrf.ww3.WW3Init.run(), hwrf.wps.Geogrid.run(), hwrf.gsi.GSIBase.run(), hwrf.wps.Ungrib.run(), hwrf.wps.Metgrid.run(), and hwrf.relocate.Merge.run().

def hwrf.gsi.GSIBase.domain

(

self

)

The WRF domain for which GSI is being run.

Definition at line 672 of file gsi.py.

Referenced by hwrf.wrfbase.WRFOutput.__repr__(), hwrf.wrfbase.WRFOutput.__str__(), hwrf.gsi.GSIBase.get_ghost(), hwrf.gsi.GSIBase.get_wrfanl(), hwrf.gsi.GSIBase.products(), and hwrf.gsi.GSIBase.run_gsi_exe().

def hwrf.gsi.GSIBase.get_ghost	(		self,
			domain
	)

Obtain output ghost product for the specified domain.

If this GSI is being run on the specified domain, returns the output product of GSI, otherwise returns None.

Parameters

domain

the WRFDomain of interest

Definition at line 641 of file gsi.py.

def hwrf.gsi.GSIBase.get_wrfanl	(		self,
			domain
	)

Obtain output ghost product for the specified domain.

If this GSI is being run on the specified domain, returns the output product of GSI, otherwise returns None.

Parameters

domain

the WRFDomain of interest

Definition at line 651 of file gsi.py.

Referenced by hwrf.relocate.Stage3.wrfanl_at_time(), and hwrf.relocate.Merge.wrfanl_at_time().

def hwrf.gsi.GSIBase.get_wrfinput

(

self

)

Obtain output wrfinput data for the outermost WRF domain.

If this GSI is being run on the WRF outermost domain (Mother Of All Domains, or MOAD), returns the output product of GSI. Otherwise, returns None.

Definition at line 661 of file gsi.py.

Referenced by hwrf.relocate.RelocationTask.wrfinput_at_time(), and hwrf.relocate.Stage3.wrfinput_at_time().

def hwrf.gsi.GSIBase.grab_bufr	(		self,
			atime = None,
			morevars = None
	)

Link bufr files.

Links or copies all needed bufr files to the local directory. If sat_da is False, satellite obs will be omitted.

Parameters

atime	the analysis time to use when specifying the required bufr files
morevars	additional variables to pass for string replacement when expanding bufr filenames in configuration (hwrf.config.HWRFConfig) sections.

Definition at line 286 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.grab_enkf_input

(

self

)

Link or copy ensemble inputs.

Links or copies the output of the last ENKF or hwrf.ensda cycle. Calls grab_wrf_enkf() if any enkf_domains were given to the constructor. Otherwise, calls grab_gfs_enkf() to get the global enkf. Will also revert to grab_gfs_enkf() if the ensda should be used, but was unavailable.

Definition at line 268 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.grab_fix_parm

(

self

)

Links or copies to the local directory any fix or parm files needed by GSI.

Definition at line 1015 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.grab_gfs_enkf	(		self,
			atime = None,
			kwargs
	)

Links the GFS ENKF files to this directory.

Parameters

atime	the analysis time, or time relative to self.atime. Used for string expansion in the hwrf.config.HWRFConfig.
kwargs	also passed to the hwrf.config.HWRFConfig for string expansion

Definition at line 580 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.grab_enkf_input().

def hwrf.gsi.GSIBase.grab_more_inputs

(

self

)

Called by run() to obtain additional inputs before before_gsi()

This is intended to be overridden by subclasses to copy or link more inputs for GSI. The default implementation does nothing.

Definition at line 770 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.grab_obstype_section	(		self,
			section,
			atime = None,
			morevars = None
	)

Copies or links observations.

Parameters

section	the obstype section to read
atime	the atime for string expansion when finding bufr files
morevars	more variables for string expansion when finding bufr files

Copies or links observations specified in the obstype sections of the configuration file to the current working directory.

The section listed in self.section should contain an "obstype" option, whose value is a comma separated list of section names. This method reads every section in that list. For example,

[gsi_d02]
catalog = {input_catalog}
...
obstypes = hdob_obstype,sat_obstypes,tdr_new_obstype
...
[sat_obstypes]
type=satellite
dataset=gfs
item=gfs_bufr
gsnd1bufr=goesfv
amsuabufr=1bamua
satwndbufr=satwnd
gpsrobufr=gpsro
...
[hdob_obstype]
...
[tdr_new_obstype]
...

For each section, the option keys are the local directory filenames expected by GSI, while the values are the data type part of the operational filename (ie.: the satwind in gfs.t12z.tm00.satwind.bufr_d). There are a few special keys:

dataset - the name of the dataset for hwrf.input purposes item - the name of the item for hwrf.input purposes type - the type of observation: satellite, or anything else. At present, only "satellite" has any special meaning.

If the type is "satellite" then the entire section will be skipped if sat_radiance_da=False in this task's config section.

If the type is "satwnd" then the entire section will be skipped if sat_wnd_da=False in this task's config section.

Once the section is parsed, the files are all linked to this directory.

Definition at line 303 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.grab_bufr().

def hwrf.gsi.GSIBase.grab_prepbufr	(		self,
			atime = None,
			kwargs
	)

Links or copies the prepbufr file to the local directory.

Parameters

atime	the analysis time, or time relative to self.atime. Used for string expansion in the hwrf.config.HWRFConfig.
kwargs	also passed to the hwrf.config.HWRFConfig for string expansion

Definition at line 437 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.grab_wrf_enkf	(		self,
			ensda
	)

Links the WRF ENKF files to this directory.

Parameters

ensda

the hwrf.ensda.DAEnsemble that provides the files

Definition at line 508 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.grab_enkf_input().

def hwrf.gsi.GSIBase.inputiter

(

self

)

Iterate over needed inputs.

Iterates over all files external to this workflow that are required to run the GSI. This may include the GFS ENKF, prepbufr, bufr, and other files. This is used by the hwrf.input module to obtain those inputs.

Definition at line 194 of file gsi.py.

def hwrf.gsi.GSIBase.make_diag_files	(		self,
			tgtpre,
			nthreads
	)

Creates GSI diagnostic files.

Makes some diagnostic files and copies them to the specified delivery location. Part of this routine is threaded: specify the number of worker threads in nthreads. Minimum is 1.

Parameters

tgtpre	the prefix to the output names, including the full path
nthreads	the maximum number of threads to use

Definition at line 849 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.make_gsi_namelist	(		self,
			filename = 'gsiparm.anl'
	)

Creates the GSI namelist in the specified file.

Parameters

filename

the destination filename

Definition at line 688 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.parent_atime

(

self

)

Parent model analysis time.

The analysis time of the parent data assimilation models (ie.: GDAS, GFS ENKF). This is usually six hours before self.atime.

Definition at line 171 of file gsi.py.

Referenced by hwrf.gsi.FGATGSI.grab_more_inputs(), hwrf.gsi.GSIBase.inputiter(), and hwrf.gsi.FGATGSI.parent_fhrs().

def hwrf.gsi.GSIBase.products	(		self,
			domains = None,
			prodtype = 'wrf_out_prod'
	)

Iterates over all output products of this Task.

Parameters

domains	the domains of interest
prodtype	ignored

Definition at line 676 of file gsi.py.

Referenced by hwrf.wrf.ExternalWRFTask.__init__(), hwrf.tracker.TrackerTask.call_completed_callbacks(), hwrf.wps.WPSTask.deliver_products(), hwrf.relocate.RelocationTask.deliver_products(), hwrf.wps.WPSTask.undeliver_products(), hwrf.prep.PrepHybrid.unrun(), hwrf.post.PostManyWRF.unrun(), hwrf.wrf.ExternalWRFTask.unrun(), and hwrf.wrf.ExternalWRFTask.wrf_check().

def hwrf.gsi.GSIBase.run

(

self

)

Runs the GSI and delivers the results.

Executes the GSI in a temporary scrub directory, deleting it afterwards if self.scrub is False. Follows this overall pattern:

1. Make temporary area and cd there
2. Copy inputs
3. Run the grab_more_inputs(), which subclasses should override
4. Calls make_gsi_namelist() to generate the namelist
5. Calls before_gsi() which subclasses should override
6. Calls run_gsi_exe() to run the actual GSI program
7. Calls after_gsi() which subclasses should override
8. Calls deliver_products() to copy files to COM
9. Generates diagnostic files.
10. Deletes the temporary directory if self.scrub=False

Definition at line 964 of file gsi.py.

Referenced by produtil.datastore.Task.runpart().

def hwrf.gsi.GSIBase.run_gsi_exe

(

self

)

Runs the actual GSI executable.

Definition at line 796 of file gsi.py.

Referenced by hwrf.gsi.GSIBase.run().

def hwrf.gsi.GSIBase.set_ensda	(		self,
			ensda,
			enkf_domains
	)

Sets the hwrf.ensda.DAEnsemble to use, and the enkf domains.

Specifies the ensemble to use for forecast error covariances, and the list of hwrf.wrf.WRFDomain domains that should be input to the GSI.

Parameters

ensda	the hwrf.ensda.DAEnsemble that provides short simulation output, for the forecast error covariances
enkf_domains	an iterable of enkf domains, WRFDomain objects, identifying which domains in ensda should be used.

Definition at line 179 of file gsi.py.

def hwrf.gsi.GSIBase.wrfout_copier	(		self,
			file
	)

Generate the wrfout file converter.

Returns the "copier" argument to deliver_file to use to copy the specified file. Will be None, unless the file is HDF5, in which case it will be "ncks -6 source target" to decompress and convert to NetCDF3 style (uncompressed) with 64-bit indexing. If ncks is missing, such a conversion is impossible, so None is returned.

Parameters

file	the file that is to be copied

Definition at line 481 of file gsi.py.

def hwrf.gsi.GSIBase.write_vitals	(		self,
			filename = 'tcvitl'
	)

Writes the tcvitals (from self.storminfo) to the specified file.

Parameters

filename

the file to receive the tcvitals

Definition at line 471 of file gsi.py.

Referenced by hwrf.relocate.RelocationTask.copy_inputs(), hwrf.relocate.Stage1.run(), hwrf.gsi.GSIBase.run(), and hwrf.relocate.Stage2.run().

---

The documentation for this class was generated from the following file:

• /mnt/pan2/projects/hwrfv3/Samuel.Trahan/trunk/ush/hwrf/gsi.py