hwrf.hwrfsystem.HWRFForecastPostProcessing Class Reference

Creates the post-processing for the HWRF forecast job. More...

Detailed Description

Creates the post-processing for the HWRF forecast job.

Creates and configures all tasks related to the post-processing system. This includes the post, regribber, trackers, and any NHC-specific product generation, as well as some data delivery settings. Everything is made configurable via the hwrf.config.HWRFConfig system.

This class is configurable through config file options:

[.conf]
   [forecast_products]
   # Post-processing start, end and step for various components:
   tracker_step=1      # hours between tracker inputs
   nonsatpost_step=1   # hours between non-satellite post inputs
   satpost_step=6      # hours between satellite post inputs
   wrfcopier_start=0   # first WRF native output time to copy to COM
   wrfcopier_end=9     # last native output time to copy to COM
   wrfcopier_step=3    # hours between WRF native outputs copied to COM
   combinetrack_fhr=12 # length of cycling track in hours
   
   # Settings for GRIB1 grid 255 (native lat-lon) for each grid:
   d23_grid=0.02,0.02,12.,15.,136,751,601
   d123low_grid=0.20,0.20,90.,110.,136,551,451
   d123high_grid=0.06,0.06,90.,110.,136,1834,1500
   d2_grid=0.06,0.06,12.,14.,136,234,201
   d3_grid=0.02,0.02,7.5,9.0,136,450,375
   d2t_grid=0.06,0.06,30.,30.,128,500,500
   d1t_grid=0.10,0.10,30.,30.,128,301,301
   
   # Tracker grid expansion settings:
   trk_expand=2.5,2.5,4.0,4.0,1000,1000,128,0.02,0.02
   
   # GRIB2 compression method:
   grib2_compression=32  ; complex packing with second-order differences
   # grib2_compression=40   ; alternative: "lossless" jpeg 2000
   
   # Output filenames:
   hwrftrk%com={out_prefix}.hwrftrk.f{fahr:03d}.grb
   hwrftrk%intercom={out_prefix}.hwrftrk.grbf{fahr:02d}
   
   anl_outer={out_prefix}.wrfanl_d02
   anl_inner={out_prefix}.wrfanl_d03
   
   hwrfprs_m%intercom={out_prefix}.hwrfprs.d23.0p02.f{fahr:03d}.grb
   hwrfprs_n%intercom={out_prefix}.hwrfprs.d3.0p02.f{fahr:03d}.grb
   hwrfprs_i%intercom={out_prefix}.hwrfprs.d2.0p06.f{fahr:03d}.grb
   hwrfprs_p%intercom={out_prefix}.hwrfprs.d1.0p20.f{fahr:03d}.grb
   hwrfprs_c%intercom={out_prefix}.hwrfprs.d123.0p06.f{fahr:03d}.grb
   hwrfprs_g%intercom={out_prefix}.hwrfprs.d123.0p25.f{fahr:03d}.grb
   
   hwrfsat_m%intercom={out_prefix}.hwrfsat.d23.0p02.f{fahr:03d}.grb
   hwrfsat_n%intercom={out_prefix}.hwrfsat.d3.0p02.f{fahr:03d}.grb
   hwrfsat_i%intercom={out_prefix}.hwrfsat.d2.0p06.f{fahr:03d}.grb
   hwrfsat_p%intercom={out_prefix}.hwrfsat.d1.0p20.f{fahr:03d}.grb
   hwrfsat_c%intercom={out_prefix}.hwrfsat.d12.0p06.f{fahr:03d}.grb
   hwrfsat_g%intercom={out_prefix}.hwrfsat.d123.0p25.f{fahr:03d}.grb
   
   hwrf2prs_m%com={out_prefix}.hwrfprs.d23.0p02.f{fahr:03d}.grb2
   hwrf2prs_n%com={out_prefix}.hwrfprs.d3.0p02.f{fahr:03d}.grb2
   hwrf2prs_i%com={out_prefix}.hwrfprs.d2.0p06.f{fahr:03d}.grb2
   hwrf2prs_p%com={out_prefix}.hwrfprs.d1.0p20.f{fahr:03d}.grb2
   hwrf2prs_c%com={out_prefix}.hwrfprs.d123.0p06.f{fahr:03d}.grb2
   hwrf2prs_g%com={out_prefix}.hwrfprs.d123.0p25.f{fahr:03d}.grb2
   
   hwrf2sat_m%com={out_prefix}.hwrfsat.d23.0p02.f{fahr:03d}.grb2
   hwrf2sat_n%com={out_prefix}.hwrfsat.d3.0p02.f{fahr:03d}.grb2
   hwrf2sat_i%com={out_prefix}.hwrfsat.d2.0p06.f{fahr:03d}.grb2
   hwrf2sat_p%com={out_prefix}.hwrfsat.d1.0p20.f{fahr:03d}.grb2
   hwrf2sat_c%com={out_prefix}.hwrfsat.d12.0p06.f{fahr:03d}.grb2
   hwrf2sat_g%com={out_prefix}.hwrfsat.d123.0p25.f{fahr:03d}.grb2

Config options have certain formats:

• productcom and productintercom — delivery location in com and intercom for each regribber output product
• *_grid — GRIB1 grid 255 grid descriptions sent to add_clatlon_grid()
• *_expand — GRIB1 grid 255 grid expansions sent to add_expandlatlon_grid()
• *_step and *_fhr — times or timespans in hours. All delivery filenames are relative to {com} and are subject to string expansion by hwrf.config.HWRFConfig.timestrinterp(). Hence everything in the [config] and [dir] sections are available, as well as standard time values.

The product names in productcom and productintercom follow a certain pattern:

hwrfprs_m
hwrf2sat_g

In the second example, here is what each component means:

• hwrf or hwrf2 – GRIB1 or GRIB2 file? The GRIB1 files are created first, and are later converted to GRIB2
• prs or sat – non-satellite post (prs) or satellite post (sat)
• m, n, i, p, c or g – the grid Here are the grid meanings:
• n – innermost domain only at its resolution
• i – intermediate domain only at its resolution
• p – parent domain only at its resolution
• m – innermost and intermediate domains, at innermost domain resolution
• c – if prs, parent and intermediate domains; if sat, all three domains. This is at the intermediate domain resolution.
• g – global quarter degree grid, GRIB1 grid 193. That is the output grid for the GFS master GRIB files.

Definition at line 258 of file hwrfsystem.py.

Inheritance diagram for hwrf.hwrfsystem.HWRFForecastPostProcessing:

Public Member Functions
def	__init__ (self, ds, conf, section, runwrf, wrf, postdoms, wrfdoms, moad, storm1inner, storm1outer, kwargs)
	Constructor for HWRFForecastPostProcessing. More...
def	make_nonsatpost
	Creates the non-satellite post-processor. More...
def	make_satpost
	Creates the satellite post-processor. More...
def	make_wrfcopier
	Generates the WRF input/output copier task. More...
def	make_gribber_tracker
	Generate the regribbing objects and the main tracker. More...
def	make_extra_trackers (self)
	Generates intermediate and outermost domain trackers. More...
Public Member Functions inherited from hwrf.hwrftask.HWRFTask
def	__init__ (self, dstore, conf, section, taskname=None, workdir=None, outdir=None, storminfo=UNSPECIFIED, taskvars=UNSPECIFIED, kwargs)
	Creates an HWRFTask. More...
def	isfakestorm (self)
def	ismultistorm (self)
def	get_workdir (self)
	Returns the directory the class should work in, as set by the "workdir" metadata value. More...
def	set_workdir (self, val)
	Sets the directory the class should work in. More...
def	get_outdir (self)
	Gets the directory that should receive output data. More...
def	set_outdir (self, val)
	Sets the directory that should receive output data. More...
def	realtime (self)
	Is this job a real-time forecast job? More...
def	redirect (self)
	Should subprograms' outputs be redirected to separate files?
def	scrub (self)
	Should temporary files be deleted as soon as they are not needed?
def	tvset (self, opt, val)
	Sets a taskvar option's value. More...
def	tvdel (self, opt)
	Deletes an object-local value set by tvset. More...
def	tvget (self, opt)
	Gets a taskvar's value. More...
def	tvhave
	Is a taskvar set? More...
def	taskvars (self)
	The dict of object-local values used for string substitution. More...
def	confint
	Alias for self.conf.getint for section self.section. More...
def	confstr
	Alias for self.conf.getstr for section self.section. More...
def	conffloat
	Alias for self.conf.getfloat for section self.section. More...
def	confbool
	Alias for self.conf.getbool for section self.section. More...
def	confget
	Alias for self.conf.get for section self.section. More...
def	confitems
	Alias for self.conf.items for section self.section. More...
def	confstrinterp (self, string, section=None, kwargs)
	Alias for self.icstr for backward compatibility. More...
def	conftimestrinterp (self, string, ftime, atime=None, section=None, kwargs)
	Alias for self.timestr for backward comaptibility. More...
def	confraw
	Get a raw configuration value before string expansion. More...
def	icstr (self, string, section=None, kwargs)
	Expands a string in the given conf section. More...
def	timestr (self, string, ftime, atime=None, section=None, kwargs)
	Expands a string in the given conf section, including time vars. More...
def	getdir
	Alias for hwrf.config.HWRFConfig.get() for the "dir" section. More...
def	getexe
	Alias for hwrf.config.HWRFConfig.get() for the "exe" section. More...
def	getconf (self)
	Returns this HWRFTask's hwrf.config.HWRFConfig object. More...
def	getsection (self)
	Returns this HWRFTask's section name in the HWRFConfig. More...
def	log
	Obtain a logging domain. More...
def	inputiter (self)
	Iterates over all inputs required by this task. More...
Public Member Functions inherited from produtil.datastore.Task
def	__init__ (self, dstore, taskname, logger=None, kwargs)
	Task constructor. More...
def	jlogfile (self)
	returns the jlogfile logger. More...
def	postmsg (self, message, args, kwargs)
	same as produtil.log.jlogger.info() More...
def	setstate (self, val)
	Sets the state of this job. More...
def	getstate (self)
	Returns the job state. More...
def	strstate (self)
	A string representation of the job state. More...
def	gettaskname (self)
	Returns the task name part of the database ID, which is the same as the prodname. More...
def	products (self, args, kwargs)
	Iterate over the products this task produces. More...
def	log (self)
	Returns the logger object for this task. More...
def	clean (self)
	Cleans up any unneeded data used by this task. More...
def	unrun (self)
	Undoes the effect of run(). More...
def	run (self)
	Performs the work this Task should do and generates all products. More...
def	is_completed (self)
	Is this task complete? More...
def	completed (self)
	Read-only property: is this task completed? Same as is_completed() More...
def	runpart (self)
	Run some of this task's work, deliver some products. More...
Public Member Functions inherited from produtil.datastore.Datum
def	__init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs)
	Datum constructor. More...
def	__enter__ (self)
	Acquires this object's thread lock. More...
def	__exit__ (self, etype, evalue, traceback)
	Releases this object's thread lock. More...
def	validate (self)
	Validates this object's Datastore, prodname and category. More...
def	getid (self)
	Returns the database ID of this datum. More...
def	getdatastore (self)
	Returns the datastore of this datum. More...
def	transaction (self)
	Creates, but does not lock, a Transaction for this datum's datastore. More...
def	getprodtype (self)
	Returns the product type of this Datum. More...
def	getprodname (self)
	Returns the product name part of the database ID. More...
def	getcategory (self)
	Returns the product category part of the database ID. More...
def	getlocation (self)
	Returns the "location" field of this Datum's database entry. More...
def	setlocation (self, v)
	Sets the "location" field of this Datum's database entry. More...
def	__hash__ (self)
	Integer hash function. More...
def	__str__ (self)
	Human-readable description of this Datum. More...
def	__repr__ (self)
	Python code-like description of this Datum. More...
def	__cmp__ (self, other)
	Compares two Datums' prodnames and categories. More...
def	set_loc_avail (self, loc, avail)
	Sets the location and availability of this Datum in a single transaction. More...
def	update (self)
	Discards all cached metadata and refreshes it from the Datastore. More...
def	__getitem__ (self, k)
	Returns the value of the specified metadata key or raises KeyError. More...
def	meta
	Return the value of a metadata key. More...
def	get
	Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More...
def	__setitem__ (self, k, v)
	Sets the value of the specified metadata key. More...
def	__delitem__ (self, k)
	Deletes the specified metadata key, which must not be "available" or "location". More...
def	__contains__ (self, k)
	Determines if a metadata key is set. More...
def	iteritems (self)
	Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More...

Public Attributes
	runwrf
	The hwrf.fcsttask.WRFAtmos or subclass thereof that runs the WRF. More...
	wrf
	The hwrf.wrf.WRFSimulation describing the WRF simulation that was run. More...
	postdoms
	The array of WRF domains to post-process (hwrf.wrf.WRFDomain objects)
	wrfdoms
	The array of WRF domains (hwrf.wrf.WRFDomain objects)
	moad
	The Mother Of All Domains (MOAD) in the WRF simulation, an hwrf.wrf.WRFDomain object. More...
	storm1inner
	The innermost domain, an hwrf.wrf.WRFDomain.
	storm1outer
	The intermediate domain, an hwrf.wrf.WRFDomain.
	nonsatpost
	The non-satellite post-processor, an hwrf.post.PostManyWRF object. More...
	satpost
	The satellite post-processor, an hwrf.post.PostManyWRF object. More...
	wrfcopier
	The task that copies WRF input and output files directly to COM. More...
	nhcp
	The task that generates custom products for NHC. More...
	track
	The main track file from self.tracker. More...
	tracker
	Task that runs the GFDL vortex tracker on all three domains combined. More...
	gribber
	The hwrf.gribtask.GRIBTask that does the regribbing. More...
	trackerd02
	The hwrf.tracker.TrackerTask that runs the GFDL vortex tracker on the combined outermost and intermediate domain vortex, omitting the innermost domain. More...
	trackerd01
	The hwrf.tracker.TrackerTask that runs the GFDL vortex tracker on the outermost domain vortex only. More...
Public Attributes inherited from hwrf.hwrftask.HWRFTask
	storminfo
	The hwrf.storminfo.StormInfo describing the vitals information for the storm processed by this HWRFTask. More...

Additional Inherited Members
Properties inherited from hwrf.hwrftask.HWRFTask
	workdir
	The directory in which this task should be run. More...
	outdir
	The directory in which this task should deliver its final output. More...
	conf
	This HWRFTask's hwrf.config.HWRFConfig object. More...
	section
	The confsection in self.section for this HWRFTask (read-only) More...
Properties inherited from produtil.datastore.Task
	state
	Read-write property: the job state. More...
	taskname
	Read-only property: the name of this task. More...
Properties inherited from produtil.datastore.Datum
	prodname = property(getprodname,None,None,)
	Read-only property, an alias for getprodname(): the product name part of the database ID. More...
	category = property(getcategory,None,None,)
	Read-only property, an alias for getcategory(), the category name part of the database ID. More...
	prodtype
	Read-only property, an alias for getprodtype(), the product type. More...
	did
	Read-only property, an alias for getid(). More...
	dstore
	Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More...
	location
	Read-write property, an alias for getlocation() and setlocation(). More...

Constructor & Destructor Documentation

def hwrf.hwrfsystem.HWRFForecastPostProcessing.__init__	(		self,
			ds,
			conf,
			section,
			runwrf,
			wrf,
			postdoms,
			wrfdoms,
			moad,
			storm1inner,
			storm1outer,
			kwargs
	)

Constructor for HWRFForecastPostProcessing.

Parameters

ds	the produtil.datastore.Datastore to use
conf	the HWRFConfig to use for configuration options. This conf is passed down to the RegribMany during regribbing operations.
section	the config section to use.
runwrf	the object that runs the forecast, such as an hwrf.fcsttask.WRFAtmos or hwrf.mpipomtc.WRFCoupledPOM
wrf	the hwrf.wrf.WRFSimulation that represents the forecast model.
postdoms	a list of domains to post-process in order of GRID id.
wrfdoms	a list of WRF domains in order of GRID id
moad	the Mother Of All Domains (MOAD), an hwrf.wrf.WRFDomain
storm1inner	the first storm's intermediate domain, an hwrf.wrf.WRFDomain
storm1outer	the first storm's innermost domain, an hwrf.wrf.WRFDomain
kwargs	additional keyword arguments are passed to the superclass constructor

Definition at line 363 of file hwrfsystem.py.

Member Function Documentation

def hwrf.hwrfsystem.HWRFForecastPostProcessing.make_extra_trackers

(

self

)

Generates intermediate and outermost domain trackers.

Generates trackers that use intermediate and outermost domain data to track the storm and analyze its intensity. These are intended for use in analyzing the effects of resolution and upscale feedback on hurricane track, structure and intensity. Creates these member variables and returns them:

• trackerd01 — a tracker that just uses the outermost domain data
• trackerd02 — a tracker that uses the outermost and intermediate domain data

  Returns

  A tuple (trackerd01,trackerd02) of the new trackers

  Precondition

  The make_gribber_tracker() must have been called with extra_trackers=True, so it will add the extra GRIB products needed by the new trackers.

Definition at line 748 of file hwrfsystem.py.

def hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker	(		self,
			extra_trackers = False,
			satpost_flag = True,
			gofile_flag = False
	)

Generate the regribbing objects and the main tracker.

Generates the gribber, tracker, track and nhcp member variables and returns them in a tuple:

• nhcp — the hwrf.nhc_products.NHCProducts object that makes NHC-specific output products and validates the wrfdiag files.
• track — the final track file output by the tracker, returned by hwrf.tracker.send_atcfunix() called on the new tracker object
• tracker — the hwrf.tracker.TrackerTask that runs the GFDL vortex tracker
• gribber — the hwrf.gribtask.GRIBTask that runs the regribbing on the post-processors created by make_satpost() and make_nonsatpost()

Parameters

extra_trackers	logical flag: are the parent and intermediate domain trackers also desired? If so, extra regribbing is arranged, that will be required by make_extra_trackers()
satpost_flag	logical flag: is satellite post-processing desired

Precondition

The make_nonsatpost() must have been called. If the satpost_flag=True, then make_satpost() must also have been called.

Returns

a tuple (nhcp,track,tracker,gribber) of the objects that were created

Definition at line 531 of file hwrfsystem.py.

def hwrf.hwrfsystem.HWRFForecastPostProcessing.make_nonsatpost	(		self,
			default_step = 1.
	)

Creates the non-satellite post-processor.

Creates the nonsatpost member, which runs the non-satellite post processing. That is the object that runs everything except the synthetic satellite brightness temperatures. This is a hwrf.post.PostManyWRF object.

Returns

the created object

Parameters

default_step

non-satellite post-processor input step, if none is provided in the config file.

Definition at line 417 of file hwrfsystem.py.

def hwrf.hwrfsystem.HWRFForecastPostProcessing.make_satpost	(		self,
			default_step = 6
	)

Creates the satellite post-processor.

Creates the satpost member, an hwrf.post.PostManyWRF object that runs the satellite post-processing.

Returns

the created object

Parameters

default_step

satellite post-processor input step, if none is provided in the config file.

Definition at line 437 of file hwrfsystem.py.

def hwrf.hwrfsystem.HWRFForecastPostProcessing.make_wrfcopier	(		self,
			wrfcopier_name = 'copywrf',
			multistorm = False
	)

Generates the WRF input/output copier task.

Creates the wrfcopier member variable which copies inputs and outputs of the forecast job directly to COM. This is an hwrf.copywrf.WRFCopyTask object.

Returns

the created object

Parameters

wrfcopier_name

the section and task name to use for the hwrf.copywrf.WRFCopyTask that is created.

Definition at line 454 of file hwrfsystem.py.

Member Data Documentation

hwrf.hwrfsystem.HWRFForecastPostProcessing.gribber

The hwrf.gribtask.GRIBTask that does the regribbing.

Definition at line 731 of file hwrfsystem.py.

Referenced by hwrf.hwrfsystem.HWRFForecastPostProcessing.make_extra_trackers().

hwrf.hwrfsystem.HWRFForecastPostProcessing.moad

The Mother Of All Domains (MOAD) in the WRF simulation, an hwrf.wrf.WRFDomain object.

Definition at line 391 of file hwrfsystem.py.

Referenced by hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker().

hwrf.hwrfsystem.HWRFForecastPostProcessing.nhcp

The task that generates custom products for NHC.

This is an hwrf.nhc_products.NHCProducts object.

Definition at line 728 of file hwrfsystem.py.

hwrf.hwrfsystem.HWRFForecastPostProcessing.nonsatpost

The non-satellite post-processor, an hwrf.post.PostManyWRF object.

Definition at line 428 of file hwrfsystem.py.

Referenced by hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker().

hwrf.hwrfsystem.HWRFForecastPostProcessing.runwrf

The hwrf.fcsttask.WRFAtmos or subclass thereof that runs the WRF.

Definition at line 387 of file hwrfsystem.py.

Referenced by hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker(), and hwrf.hwrfsystem.HWRFForecastPostProcessing.make_wrfcopier().

hwrf.hwrfsystem.HWRFForecastPostProcessing.satpost

The satellite post-processor, an hwrf.post.PostManyWRF object.

Definition at line 447 of file hwrfsystem.py.

Referenced by hwrf.hwrfsystem.HWRFForecastPostProcessing.make_gribber_tracker().

hwrf.hwrfsystem.HWRFForecastPostProcessing.track

The main track file from self.tracker.

Definition at line 729 of file hwrfsystem.py.

hwrf.hwrfsystem.HWRFForecastPostProcessing.tracker

Task that runs the GFDL vortex tracker on all three domains combined.

This is an hwrf.tracker.TrackerTask.

Definition at line 730 of file hwrfsystem.py.

hwrf.hwrfsystem.HWRFForecastPostProcessing.trackerd01

The hwrf.tracker.TrackerTask that runs the GFDL vortex tracker on the outermost domain vortex only.

Definition at line 787 of file hwrfsystem.py.

hwrf.hwrfsystem.HWRFForecastPostProcessing.trackerd02

The hwrf.tracker.TrackerTask that runs the GFDL vortex tracker on the combined outermost and intermediate domain vortex, omitting the innermost domain.

Definition at line 786 of file hwrfsystem.py.

hwrf.hwrfsystem.HWRFForecastPostProcessing.wrf

The hwrf.wrf.WRFSimulation describing the WRF simulation that was run.

Definition at line 388 of file hwrfsystem.py.

Referenced by hwrf.post.PostManyWRF.__init__(), hwrf.fcsttask.WRFAtmos.__init__(), hwrf.fcsttask.AnalysisCycle.anlintime(), hwrf.fcsttask.AnalysisCycle.anlouttime(), hwrf.fcsttask.WRFAnl.anltime(), hwrf.fcsttask.WRFAnl.deliver_products(), hwrf.fcsttask.AnalysisCycle.deliver_products(), hwrf.fcsttask.RealNMM.fort65_at_time(), hwrf.multistorm.WRFAtmosMultiStorm.make_namelist(), hwrf.fcsttask.WRFTaskBase.make_namelist(), hwrf.fcsttask.RealNMM.make_namelist(), hwrf.fcsttask.WRFAnl.make_products(), hwrf.fcsttask.AnalysisCycle.make_products(), hwrf.hwrfsystem.HWRFForecastPostProcessing.make_wrfcopier(), hwrf.post.PostManyWRF.products(), hwrf.fcsttask.WRFAnl.products(), hwrf.fcsttask.WRFAnl4Trak.products(), hwrf.fcsttask.AnalysisCycle.products(), hwrf.fcsttask.WRFAnl4Trak.retime_wrfout(), hwrf.post.PostManyWRF.taskname_for(), hwrf.fcsttask.WRFAnl.wrfanl_at_time(), hwrf.fcsttask.RealNMM.wrfbdy_at_time(), hwrf.fcsttask.RealNMM.wrfinput_at_time(), and hwrf.fcsttask.AnalysisCycle.wrfinput_at_time().

hwrf.hwrfsystem.HWRFForecastPostProcessing.wrfcopier

The task that copies WRF input and output files directly to COM.

This is an hwrf.copywrf.WRFCopyTask object.

Definition at line 524 of file hwrfsystem.py.

---

The documentation for this class was generated from the following file:

• /mnt/pan2/projects/hwrfv3/Samuel.Trahan/trunk/ush/hwrf/hwrfsystem.py