|
HWRF
trunk@4391
|
This is a FcstTask since it prepares input to a forecast model. The FcstTask class provides some methods required by the final merge task. This task merges all the domain 01 files from the relocation.Merge(RelocationTask) of each storm and produces a single d01 merged file that is comprise of all the storms in a multstorm.
Definition at line 135 of file finalmergetask.py.
Public Member Functions | |
| def | __init__ (self, dstore, conf, section, taskname=None, kwargs) |
| def | update_state (self) |
| def | wrfinput_at_time (self, atime, domain) |
| def | run (self) |
| def | convert_wrfinput2bin (self) |
| def | finalmerge2netcdf (self) |
| def | final_merge (self, gfsrealinit_bin, gdasmerge_bins) |
| def | products (self, kwargs) |
| def | add_wrfinput (self, r, stormid) |
| def | add_merge (self, r, stormid) |
| def | deliver_products (self, missing=None, logger=None, keep=False, frominfo=None, kwargs) |
| def | copy_inputs (self) |
| def | run_ext |
| def | run_exe |
 Public Member Functions inherited from hwrf.fcsttask.FcstTask | |
| def | __init__ (self, dstore, conf, section, outdir=None, taskname=None, kwargs) |
| Creates a FcstTask object. More... | |
| def | has_input (self, typename) |
| is there any input data of this type to this task? More... | |
| def | add_input (self, typename, inobj) |
| add input of a specified type More... | |
| def | check_input (self, typenames, kwargs) |
| check if input data is available More... | |
| def | link_input (self, typenames, just_check=False, kwargs) |
| link or check for inputs More... | |
| Public Member Functions inherited from hwrf.hwrftask.HWRFTask | |
| def | __init__ (self, dstore, conf, section, taskname=None, workdir=None, outdir=None, storminfo=UNSPECIFIED, taskvars=UNSPECIFIED, kwargs) |
| Creates an HWRFTask. More... | |
| def | isfakestorm (self) |
| def | ismultistorm (self) |
| def | get_workdir (self) |
| Returns the directory the class should work in, as set by the "workdir" metadata value. More... | |
| def | set_workdir (self, val) |
| Sets the directory the class should work in. More... | |
| def | get_outdir (self) |
| Gets the directory that should receive output data. More... | |
| def | set_outdir (self, val) |
| Sets the directory that should receive output data. More... | |
| def | realtime (self) |
| Is this job a real-time forecast job? More... | |
| def | redirect (self) |
| Should subprograms' outputs be redirected to separate files? | |
| def | scrub (self) |
| Should temporary files be deleted as soon as they are not needed? | |
| def | tvset (self, opt, val) |
| Sets a taskvar option's value. More... | |
| def | tvdel (self, opt) |
| Deletes an object-local value set by tvset. More... | |
| def | tvget (self, opt) |
| Gets a taskvar's value. More... | |
| def | tvhave |
| Is a taskvar set? More... | |
| def | taskvars (self) |
| The dict of object-local values used for string substitution. More... | |
| def | confint |
| Alias for self.conf.getint for section self.section. More... | |
| def | confstr |
| Alias for self.conf.getstr for section self.section. More... | |
| def | conffloat |
| Alias for self.conf.getfloat for section self.section. More... | |
| def | confbool |
| Alias for self.conf.getbool for section self.section. More... | |
| def | confget |
| Alias for self.conf.get for section self.section. More... | |
| def | confitems |
| Alias for self.conf.items for section self.section. More... | |
| def | confstrinterp (self, string, section=None, kwargs) |
| Alias for self.icstr for backward compatibility. More... | |
| def | conftimestrinterp (self, string, ftime, atime=None, section=None, kwargs) |
| Alias for self.timestr for backward comaptibility. More... | |
| def | confraw |
| Get a raw configuration value before string expansion. More... | |
| def | icstr (self, string, section=None, kwargs) |
| Expands a string in the given conf section. More... | |
| def | timestr (self, string, ftime, atime=None, section=None, kwargs) |
| Expands a string in the given conf section, including time vars. More... | |
| def | getdir |
| Alias for hwrf.config.HWRFConfig.get() for the "dir" section. More... | |
| def | getexe |
| Alias for hwrf.config.HWRFConfig.get() for the "exe" section. More... | |
| def | getconf (self) |
| Returns this HWRFTask's hwrf.config.HWRFConfig object. More... | |
| def | getsection (self) |
| Returns this HWRFTask's section name in the HWRFConfig. More... | |
| def | log |
| Obtain a logging domain. More... | |
| def | inputiter (self) |
| Iterates over all inputs required by this task. More... | |
| Public Member Functions inherited from produtil.datastore.Task | |
| def | __init__ (self, dstore, taskname, logger=None, kwargs) |
| Task constructor. More... | |
| def | jlogfile (self) |
| returns the jlogfile logger. More... | |
| def | postmsg (self, message, args, kwargs) |
| same as produtil.log.jlogger.info() More... | |
| def | setstate (self, val) |
| Sets the state of this job. More... | |
| def | getstate (self) |
| Returns the job state. More... | |
| def | strstate (self) |
| A string representation of the job state. More... | |
| def | gettaskname (self) |
| Returns the task name part of the database ID, which is the same as the prodname. More... | |
| def | products (self, args, kwargs) |
| Iterate over the products this task produces. More... | |
| def | log (self) |
| Returns the logger object for this task. More... | |
| def | clean (self) |
| Cleans up any unneeded data used by this task. More... | |
| def | unrun (self) |
| Undoes the effect of run(). More... | |
| def | run (self) |
| Performs the work this Task should do and generates all products. More... | |
| def | is_completed (self) |
| Is this task complete? More... | |
| def | completed (self) |
| Read-only property: is this task completed? Same as is_completed() More... | |
| def | runpart (self) |
| Run some of this task's work, deliver some products. More... | |
| Public Member Functions inherited from produtil.datastore.Datum | |
| def | __init__ (self, dstore, prodname, category, meta=None, cache=30, location=None, kwargs) |
| Datum constructor. More... | |
| def | __enter__ (self) |
| Acquires this object's thread lock. More... | |
| def | __exit__ (self, etype, evalue, traceback) |
| Releases this object's thread lock. More... | |
| def | validate (self) |
| Validates this object's Datastore, prodname and category. More... | |
| def | getid (self) |
| Returns the database ID of this datum. More... | |
| def | getdatastore (self) |
| Returns the datastore of this datum. More... | |
| def | transaction (self) |
| Creates, but does not lock, a Transaction for this datum's datastore. More... | |
| def | getprodtype (self) |
| Returns the product type of this Datum. More... | |
| def | getprodname (self) |
| Returns the product name part of the database ID. More... | |
| def | getcategory (self) |
| Returns the product category part of the database ID. More... | |
| def | getlocation (self) |
| Returns the "location" field of this Datum's database entry. More... | |
| def | setlocation (self, v) |
| Sets the "location" field of this Datum's database entry. More... | |
| def | __hash__ (self) |
| Integer hash function. More... | |
| def | __str__ (self) |
| Human-readable description of this Datum. More... | |
| def | __repr__ (self) |
| Python code-like description of this Datum. More... | |
| def | __cmp__ (self, other) |
| Compares two Datums' prodnames and categories. More... | |
| def | set_loc_avail (self, loc, avail) |
| Sets the location and availability of this Datum in a single transaction. More... | |
| def | update (self) |
| Discards all cached metadata and refreshes it from the Datastore. More... | |
| def | __getitem__ (self, k) |
| Returns the value of the specified metadata key or raises KeyError. More... | |
| def | meta |
| Return the value of a metadata key. More... | |
| def | get |
| Alias for self.meta() Returns the value of the specified metadata key or returns default if it is unset. More... | |
| def | __setitem__ (self, k, v) |
| Sets the value of the specified metadata key. More... | |
| def | __delitem__ (self, k) |
| Deletes the specified metadata key, which must not be "available" or "location". More... | |
| def | __contains__ (self, k) |
| Determines if a metadata key is set. More... | |
| def | iteritems (self) |
| Iterates over all metadata (key,value) pairs for this Datum, including "available" and "location". More... | |
Public Attributes | |
| fake_stormid | |
| real_stormids | |
| priority_stormid | |
| dest_dir | |
| fmprodname | |
| state | |
| keeprun | |
| Public Attributes inherited from hwrf.fcsttask.FcstTask | |
| inputs | |
| a mapping of typename to a list of input objects | |
| Public Attributes inherited from hwrf.hwrftask.HWRFTask | |
| storminfo | |
| The hwrf.storminfo.StormInfo describing the vitals information for the storm processed by this HWRFTask. More... | |
Additional Inherited Members | |
| Properties inherited from hwrf.hwrftask.HWRFTask | |
| workdir | |
| The directory in which this task should be run. More... | |
| outdir | |
| The directory in which this task should deliver its final output. More... | |
| conf | |
| This HWRFTask's hwrf.config.HWRFConfig object. More... | |
| section | |
| The confsection in self.section for this HWRFTask (read-only) More... | |
| Properties inherited from produtil.datastore.Task | |
| state | |
| Read-write property: the job state. More... | |
| taskname | |
| Read-only property: the name of this task. More... | |
| Properties inherited from produtil.datastore.Datum | |
| prodname = property(getprodname,None,None,) | |
| Read-only property, an alias for getprodname(): the product name part of the database ID. More... | |
| category = property(getcategory,None,None,) | |
| Read-only property, an alias for getcategory(), the category name part of the database ID. More... | |
| prodtype | |
| Read-only property, an alias for getprodtype(), the product type. More... | |
| did | |
| Read-only property, an alias for getid(). More... | |
| dstore | |
| Read-only property, an alias for getdatastore(), the Datastore in which this Datum resides. More... | |
| location | |
| Read-write property, an alias for getlocation() and setlocation(). More... | |
| def hwrf.finalmergetask.FinalMergeTask.add_merge | ( | self, | |
| r, | |||
| stormid | |||
| ) |
Adds an input source (via self.add_input) that will provide the wrfinput output file from a prior gdas_merge, real_nmm or relocation. This is used for the storm's final output of the per-storm initialization.
Definition at line 401 of file finalmergetask.py.
| def hwrf.finalmergetask.FinalMergeTask.add_wrfinput | ( | self, | |
| r, | |||
| stormid | |||
| ) |
Adds an input source (via self.add_input) that will provide the wrfinput output file from real_nmm. The given object must have a wrfinput_at_time(atime,domain) function that returns a Product for a given analysis time and WRFDomain object. Returns self.
Definition at line 391 of file finalmergetask.py.
| def hwrf.finalmergetask.FinalMergeTask.convert_wrfinput2bin | ( | self | ) |
Runs the hwrf_diffwrf_3dvar program to convert all the required
wrfinput_d01 files from netcdf to binary.
returns a tuple (gfsrealinit_bin, gdas_merge_bins)
with the the binary filenames that will be used
as input for the final merge executable.
Definition at line 259 of file finalmergetask.py.
Referenced by hwrf.finalmergetask.FinalMergeTask.run().
| def hwrf.finalmergetask.FinalMergeTask.copy_inputs | ( | self | ) |
Copies, or makes, one or more input files.
Definition at line 456 of file finalmergetask.py.
Referenced by hwrf.finalmergetask.FinalMergeTask.run(), hwrf.relocate.Stage2.run(), and hwrf.relocate.Merge.run().
| def hwrf.finalmergetask.FinalMergeTask.deliver_products | ( | self, | |
missing = None, |
|||
logger = None, |
|||
keep = False, |
|||
frominfo = None, |
|||
| kwargs | |||
| ) |
Delivers products to intercom via Product.deliver. Any keyword arguments are passed on to Product.deliver. By default, keep=False, which means the local copy of the file may no longer exists. If frominfo is specified, it will be ignored.
Definition at line 409 of file finalmergetask.py.
Referenced by hwrf.finalmergetask.FinalMergeTask.add_merge(), hwrf.mpipomtc.POMInit.run(), hwrf.ww3.WW3Init.run(), hwrf.finalmergetask.FinalMergeTask.run(), hwrf.wps.Geogrid.run(), hwrf.gsi.GSIBase.run(), hwrf.wps.Ungrib.run(), hwrf.wps.Metgrid.run(), and hwrf.relocate.Merge.run().
| def hwrf.finalmergetask.FinalMergeTask.final_merge | ( | self, | |
| gfsrealinit_bin, | |||
| gdasmerge_bins | |||
| ) |
Runs the hwrf-utilities hwrf_final_merge program with
the following inputs and generates the final merge output file,
all in this tasks self.workdir .
gfsrealinit_bin - the name of the d01 priority storm
gfsrealinit binary filename
gdasmerge_bins - a list of gdas_merge d01 binary
filenames for each real storm
Definition at line 347 of file finalmergetask.py.
Referenced by hwrf.finalmergetask.FinalMergeTask.run().
| def hwrf.finalmergetask.FinalMergeTask.finalmerge2netcdf | ( | self | ) |
Runs the hwrf_diffwrf_3dvar program to convert the final merge binary file to netcdf.
Definition at line 320 of file finalmergetask.py.
Referenced by hwrf.finalmergetask.FinalMergeTask.run().
| def hwrf.finalmergetask.FinalMergeTask.products | ( | self, | |
| kwargs | |||
| ) |
Iterates over all products generated by this task.
Definition at line 384 of file finalmergetask.py.
Referenced by hwrf.wrf.ExternalWRFTask.__init__(), hwrf.tracker.TrackerTask.call_completed_callbacks(), hwrf.wps.WPSTask.deliver_products(), hwrf.relocate.RelocationTask.deliver_products(), hwrf.wps.WPSTask.undeliver_products(), hwrf.prep.PrepHybrid.unrun(), hwrf.post.PostManyWRF.unrun(), hwrf.wrf.ExternalWRFTask.unrun(), and hwrf.wrf.ExternalWRFTask.wrf_check().
| def hwrf.finalmergetask.FinalMergeTask.run | ( | self | ) |
Performs all work needed to run the Final Merge program and create the final merged product. Creates the work directory, CD's to it, Creates a domain mapping dictionary for all the storms. ,link_all_inputs, run_exe, and deliver_products.
Definition at line 195 of file finalmergetask.py.
Referenced by produtil.datastore.Task.runpart().
| def hwrf.finalmergetask.FinalMergeTask.run_exe | ( | self, | |
exename = 'final_merge', |
|||
runner = None, |
|||
sleeptime = None |
|||
| ) |
Runs the executable this task is responsible for running. Determines if the program ran correctly. The exename is the name of the argument in the [exe] section of the HWRFConfig. Options: runner=Runner - pass a produtil.prog.Runner object if desired. This overrides any decision of what to run: the exename will be ignored, and whatever is supplied in runner is simply passed to produtil.run.run. sleeptime - passed to produtil.run.run to determine how often to check the child process. By default, the sleeptime option in this task's config section is used, or if that is absent, 30 seconds.
Definition at line 662 of file finalmergetask.py.
| def hwrf.finalmergetask.FinalMergeTask.run_ext | ( | self, | |
| cmd, | |||
echo = None, |
|||
inputs = None, |
|||
incopies = None, |
|||
outputs = None, |
|||
opt_outputs = None |
|||
| ) |
Run an external command linking in fort.X files for input
and output. If redirect=True, redirect logs to a separate file.
It will use "self.getexe()" on the command to find the
external program to execute.
If a list is passed in as the echo variable, then /bin/echo
will be run with that list and piped to the external command.
The inputs dictionary should consist of a fortran file
number and the source file, such as:
inputs = {11:tcvitals, 12:wrfout_d01}
would produce symbolic links:
fort.11 -> tcvitals
fort.12 -> wrfout_d01
input files can also be copied using incopies:
incopies = {11:tcvitals, 12:wrfout_d01}
would create files instead of links.
The outputs and opt_outputs (optional outputs) should be of the
dictionary as the inputs. As in:
outputs = {56:new_data_4x, 85:storm_radius}
this would mean the "fort.56" file would be renamed to "new_data_4x"
and the "fort.85" renamed to "storm_radius".
If opt_outputs is given then the fortran file is tested to see if it
exists and only if it does is it renamed to the output filename.
A log file will be created consisting of the stdout and stderr of the
command run. It will be named consisting of the taskname and command.
For example, if this is relocation stage 1 and the command is
hwrf_pert_ct then the log file is "rel_stage_1_hwrf_pert_ct.log"
Definition at line 542 of file finalmergetask.py.
Referenced by hwrf.relocate.Stage3.anl_4x(), hwrf.relocate.Stage3.anl_bogus_10m(), hwrf.relocate.Stage3.anl_cs_10m(), hwrf.relocate.Merge.blend_gsi(), hwrf.finalmergetask.FinalMergeTask.copy_inputs(), hwrf.relocate.Stage2.create_nest(), hwrf.relocate.Stage2.create_track(), hwrf.relocate.Stage3.create_track(), hwrf.finalmergetask.FinalMergeTask.final_merge(), hwrf.relocate.Stage1.guess_track(), hwrf.relocate.Merge.inter_2to1(), hwrf.relocate.Merge.inter_2to1ges(), hwrf.relocate.Stage3.inter_2to2(), hwrf.relocate.Merge.inter_2to2(), hwrf.relocate.Stage3.inter_2to2_again(), hwrf.relocate.Merge.inter_2to3(), hwrf.relocate.Merge.inter_2to6(), hwrf.relocate.Merge.inter_3to2(), hwrf.relocate.Stage3.inter_4to6(), hwrf.relocate.Stage1.merge_nest(), hwrf.relocate.Stage2.merge_nests(), hwrf.relocate.Stage1.pert_ct(), hwrf.relocate.Stage3.pert_ct_gfs(), hwrf.relocate.Stage3.pert_ct_weak(), hwrf.relocate.Stage1.split_wrf(), and hwrf.relocate.Stage2.wrf_split().
1.8.10