HWRF  trunk@4391
Public Member Functions | Public Attributes | List of all members
hwrf.launcher.HWRFLauncher Class Reference

A replacement for the hwrf.config.HWRFConfig used throughout the HWRF system. More...

Detailed Description

A replacement for the hwrf.config.HWRFConfig used throughout the HWRF system.

You should never need to instantiate one of these — the launch() and load() functions do that for you. This class is the underlying implementation of most of the functionality described in launch() and load()

Definition at line 555 of file launcher.py.

Inheritance diagram for hwrf.launcher.HWRFLauncher:
hwrf.config.HWRFConfig

Public Member Functions

def __init__
 Creates a new HWRFLauncher. More...
 
def storm_for_stormnum (self)
 Not implemented. More...
 
def decide_domain_center
 Decide the outermost domain's center. More...
 
def choose_vitbase
 Decides the location of the vitals file. More...
 
def read_fake_tcvitals
 
def read_tcvitals_and_messages
 Reads in the tcvitals file and message files. More...
 
def set_storm (self, syndat, oldsyndat)
 Sets the storm that is to be run. More...
 
def set_storm_multistorm (self, multistorm_real_sids, syndat4multistorm, oldsyndat4multistorm)
 
def tcautoseed
 Sets the random seed for ensemble perturbations. More...
 
def gen_vitals
 Generate tcvitals files. More...
 
def sanity_check_ensemble
 Runs a sanity check on the ensemble configurations. More...
 
def sanity_check_archive
 Runs a sanity check on the archiving settings. More...
 
def sanity_check_config_files
 Runs sanity checks related to config files. More...
 
def sanity_check_coupling
 Runs sanity checks related to coupling. More...
 
def timeless_sanity_check
 Runs all sanity checks that are not dependent on the cycle. More...
 
def sanity_check_forecast_length
 Ensures the forecast length is valid. More...
 
def sanity_check_directory
 Runs a sanity check on the provided directory paths. More...
 
def sanity_check_sanity_check
 Checks to see if the sanity checks can be run. More...
 
def sanity_get_fix_version
 Sanity checks the fix file version. More...
 
def sanity_check_fix_files
 Sanity checks the fix files. More...
 
def sanity_check_executables
 Sanity checks some of the executables. More...
 
def sanity_check (self)
 Runs nearly all sanity checks. More...
 
def sanity_check_da (self, logger)
 Sanity checks the data assimilation. More...
 
def sanity_check_expt (self, logger)
 Sanity checks the hwrf_expt module. More...
 
def guess_default_values (self)
 Tries to guess default values for many configuration settings. More...
 
def make_holdvars
 Creates the com/storm*.holdvars.txt file. More...
 
- Public Member Functions inherited from hwrf.config.HWRFConfig
def __init__
 HWRFConfig constructor. More...
 
def readstr (self, source)
 read config data and add it to this object More...
 
def read (self, source)
 reads and parses a config file More...
 
def readfp (self, source)
 read config data from an open file More...
 
def readstr (self, string)
 reads config data from an in-memory string More...
 
def set_options (self, section, kwargs)
 set values of several options in a section More...
 
def read_precleaned_vitfile (self, vitfile)
 reads tcvitals More...
 
def realtime (self)
 is this a real-time simulation? More...
 
def set (self, section, key, value)
 set a config option More...
 
def __enter__ (self)
 grab the thread lock More...
 
def __exit__ (self, a, b, c)
 release the thread lock More...
 
def register_hwrf_task (self, name)
 add an hwrf.hwrftask.HWRFTask to the database More...
 
def log
 returns a logging.Logger object More...
 
def getdatastore (self)
 returns the Datastore More...
 
def getcycle (self)
 get the analysis time More...
 
def setcycle (self, cycle)
 set the analysis time More...
 
def set_time_vars (self)
 internal function that sets time-related variables More...
 
def add_section (self, sec)
 add a new config section More...
 
def has_section (self, sec)
 does this section exist? More...
 
def has_option (self, sec, opt)
 is this option set? More...
 
def getdir
 query the "dir" section More...
 
def getloc
 search the config, exe and dir sections in that order More...
 
def getexe
 query the "exe" section More...
 
def __getitem__ (self, arg)
 convenience function; replaces self.items and self.get More...
 
def makedirs (self, args)
 calls produtil.fileop.makedirs() on directories in the [dir] section More...
 
def keys (self, sec)
 get options in a section More...
 
def items
 get the list of (option,value) tuples for a section More...
 
def write (self, fileobject)
 write the contents of this HWRFConfig to a file More...
 
def getraw
 return the raw value of an option More...
 
def strinterp (self, sec, string, kwargs)
 perform string expansion More...
 
def timestrinterp (self, sec, string, ftime, atime=None, kwargs)
 performs string expansion, including time variables More...
 
def getint
 get an integer value More...
 
def getfloat
 get a float value More...
 
def getstr
 get a string value More...
 
def get
 get the value of an option from a section More...
 
def options (self, sec)
 what options are in this section? More...
 
def getboolean
 alias for getbool: get a bool value More...
 
def getbool
 get a bool value More...
 

Public Attributes

 cycle
 

Additional Inherited Members

- Properties inherited from hwrf.config.HWRFConfig
 datastore
 read-only property: the Datastore object for this HWRF simulation More...
 
 cycle
 the analysis cycle, a datetime.datetime object More...
 

Constructor & Destructor Documentation

def hwrf.launcher.HWRFLauncher.__init__ (   self,
  conf = None 
)

Creates a new HWRFLauncher.

Parameters
confThe configuration file.

Definition at line 561 of file launcher.py.

Member Function Documentation

def hwrf.launcher.HWRFLauncher.choose_vitbase (   self,
  storm_num = None 
)

Decides the location of the vitals file.

Decides the location of the vitfile that should be read in by read_precleaned_vitfile. Optionally, you can specify the storm number (1-10) of the storm whose vitals should be read in. Otherwise, a reasonable guess will be made.

Parameters
storm_numthe index of the storm from 1-10
Returns
the vitals path

Definition at line 598 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.gen_vitals().

def hwrf.launcher.HWRFLauncher.decide_domain_center (   self,
  logger = None 
)

Decide the outermost domain's center.

If the domain center is not already set in the [config] section domlat and domlon variables, decides the domain center using the hwrf.storminfo.StormInfo.hwrf_domain_center routine.

Parameters
loggerthe logging.Logger for log messages.

Definition at line 578 of file launcher.py.

def hwrf.launcher.HWRFLauncher.gen_vitals (   self,
  STID,
  cycling_interval,
  revital,
  storm_num = None 
)

Generate tcvitals files.

Given an hwrf.revital.Revital object, preferably from read_precleaned_vitfile or read_tcvitals_and_messages, searches for the specified storm's vitals. Creates the files that are expected to exist in the WORKhwrf directory. The filenames are based off of the vitbase variable, but with various suffixes appended. This function should only be called once per workflow, per storm.

Parameters
STIDthe three character stormid (12L)
cycling_intervalseconds between HWRF cycles (6*3600)
revitalThe hwrf.revital.Revital with tcvitals data
storm_numThe storm index 1-10

Definition at line 813 of file launcher.py.

def hwrf.launcher.HWRFLauncher.guess_default_values (   self)

Tries to guess default values for many configuration settings.

Tries to set default values for some mandatory conf options. The default values come from either other options or from environment variables. If no suitable default can be found, execution will continue, but later jobs may fail.

Config options are as follows. If $VAR appears, that refers to ENV["VAR"]:

  • config.cycle — the cycle to run as a ten digit date (2014091418) Taken from cycle if present, otherwise $YMDH
  • config.storm_num — the storm number as a priority 1 to 5. Taken from $storm_num or uses the default of 1.
  • config.stormlabel — "storm" with the storm number appended (ie.: storm5 if storm_num=5).
  • dir.HOMEhwrf — set to HOMEhwrf or $HOMEhwrf
  • dir.WORKhwrf — set to WORKhwrf or $WORKhwrf
  • dir.syndat — tcvitals directory. Default: $COMINARCH
  • config.input_catalog — input catalog (conf section) name. Default: $INPUT_CATALOG or "hwrfdata"
  • config.PARAFLAG — NO if you are NCEP Central Operations (NCO), and YES otherwise. This is used to turn on or off DBNet alerts and other NCO-specific options.

In addition, the following directories are added to the [dir] section:

  • USHhwrf — the location of ush scripts and the parent directory of the hwrf, pom and produtil packages
  • FIXhwrf — the location of the HWRF fix directory
  • JOBhwrf — the location of the HWRF jobs directory. This is not needed by normal users.
  • EXhwrf — the location of the HWRF scripts directory
  • PARMhwrf — the location of the HWRF parm/ directory
  • EXEChwrf — the location of the HWRF exec/ directory
  • utilexec — the location of the HWRF nwport/exec or /nwprod/exec directory

If set, these variables will be copied to the [config] section:

  • EXPT — optional: the experiment identifier, which must be alphanumeric, and can contain underscores. Default: HWRF
  • SUBEXPT — optional: the subexperiment identifier, which must be alphanumeric, and can contain underscores. Default: set to value of EXPT

Definition at line 1549 of file launcher.py.

def hwrf.launcher.HWRFLauncher.make_holdvars (   self,
  part1 = '{PARMhwrf}/hwrf_holdvars.txt',
  part2 = None 
)

Creates the com/storm*.holdvars.txt file.

Creates the storm*.holdvars.txt file needed by the old ksh-based scripts. This is done for backward compatibility only. The two arguments (part1 and part2) are two files to pass through self.strinterp and then into the holdvars file. Part 1 is mandatory, but part2 is optional. It also fills in a few custom derived variables:

  • cap_run_gsi — capitalized version of [config] section run_gsi
  • cap_run_relocation — capitalized version of [config] section run_relocation
  • holdvars_model — "COUPLED" if [config] section run_ocean is true, and "ATMOS" if it is false.
    Parameters
    part1The first input file to read
    part2The second input file to read or None to disable

Definition at line 1658 of file launcher.py.

def hwrf.launcher.HWRFLauncher.read_fake_tcvitals (   self,
  fakestorm_vitals = None 
) 
Intended use is for the multistorm fake storm. Same as the
read_tcvitals_and_messages method except the vitals are
from fakestorm_vitals in hwrf_multistorm.conf. basd on the arguments.

Definition at line 621 of file launcher.py.

def hwrf.launcher.HWRFLauncher.read_tcvitals_and_messages (   self,
  vitdir = None,
  vitpattern = None,
  include_messages = True,
  other_cycle = None 
)

Reads in the tcvitals file and message files.

Reads in the tcvitals files for the current cycle and optionally another cycle, which may be in the same file. Also reads in message files if requested. Cleans the result up and returns it as an hwrf.revital.Revital object.

Parameters
vitdiroptional: the directory in which to find the tcvitals. Default: [dir] section syndat variable.
vitpatternoptional: passed into strftime to generate the name of the vitals file within vitdir. Default: [conf] section vitpattern variable, or syndat_tcvitals.Y if missing.
include_messagesoptional flag: if True, attempts to find the hurricane message files, and includes them in the list of files to read in. Default: True.
other_cycleoptional: another cycle whose vitals file should also be parsed. This can be anything accepted by to_datetime_rel(...,self.cycle). This is intended to allow year-crossing cycling, such as a January 1, 00:00 UTC cycle that is a warm start off of a prior December 31, 18:00 UTC cycle. If the other_cycle's vitals file is the same as the one from self.cycle, then the file is only read once.
Returns
an hwrf.revital.Revital with the vitals data

Definition at line 644 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.read_fake_tcvitals().

def hwrf.launcher.HWRFLauncher.sanity_check (   self)

Runs nearly all sanity checks.

Runs simple sanity checks on the HWRF installation directory and configuration to make sure everything looks okay. May throw a wide variety of exceptions if sanity checks fail.

Definition at line 1433 of file launcher.py.

def hwrf.launcher.HWRFLauncher.sanity_check_archive (   self,
  logger = None 
)

Runs a sanity check on the archiving settings.

Parameters
loggera logging.Logger for log messages

Definition at line 1087 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_config_files (   self,
  logger = None 
)

Runs sanity checks related to config files.

Sanity checks the provided *.conf files. For example, some config files are incompatible with others, and some must be loaded in a specific order.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1130 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_coupling (   self,
  logger = None 
)

Runs sanity checks related to coupling.

Should be runnable with or without a specified cycle.

Parameters
loggerA logging.Logger for log messages

Definition at line 1147 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_da (   self,
  logger 
)

Sanity checks the data assimilation.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1507 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_directory (   self,
  thedir,
  dirvar,
  writable = True,
  logger = None 
)

Runs a sanity check on the provided directory paths.

Checks to make sure the specified directory exists and can be read and executed. If writable=True, also checks to see if it can be written. The dirvar is an explanation of what the directory relates to, for example HOMEhwrf.

Parameters
thedira directory to check
dirvarthe variable that will be set to this directory (such as PARMhwrf, USHhwrf, etc.)
writableDo we need to write to this directory?
loggerthe logging.Logger for log messages

Definition at line 1240 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.sanity_check(), and hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_ensemble (   self,
  enset,
  logger = None 
)

Runs a sanity check on the ensemble configurations.

Checks that:

  1. If the GEFS-based forecast ensemble is in use, a valid ensemble ID is chosen.
  2. If a valid ensemble ID is chosen, the GEFS-based forecast ensemble is in use.
  3. The user does not enable both the GEFS-based forecast ensemble and the GFS-based DA ensemble.
  4. If the GFS-based DA ensemble is in use, at least thirty members are chosen, and no more than eighty.
    Parameters
    enseta set of ensemble ids
    loggera logging.Logger for log messages

Definition at line 1000 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_executables (   self,
  logger = None 
)

Sanity checks some of the executables.

Checks to see if a few of the executables are available. This is not an exhaustive check: most executables are not checked. This check is just to see if the user forgot to install executables entirely.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1354 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_expt (   self,
  logger 
)

Sanity checks the hwrf_expt module.

Loads the hwrf_expt module, runs its init_module routine, and then runts its sanity_check, passing the specified logger.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1526 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_fix_files (   self,
  logger = None 
)

Sanity checks the fix files.

Checks to see if the fix files are available and match the expected fix file version.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1304 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_forecast_length (   self,
  logger = None 
)

Ensures the forecast length is valid.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1229 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.timeless_sanity_check().

def hwrf.launcher.HWRFLauncher.sanity_check_sanity_check (   self,
  logger = None 
)

Checks to see if the sanity checks can be run.

In essence, this is a sanity check of the sanity check routines.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1267 of file launcher.py.

def hwrf.launcher.HWRFLauncher.sanity_get_fix_version (   self,
  logger = None 
)

Sanity checks the fix file version.

Gets the expected fix file version from [sanity] fix_version. Raises HWRFConfigInsane if there is an error while getting it.

Parameters
loggerthe logging.Logger for log messages

Definition at line 1278 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.sanity_check_fix_files(), and hwrf.launcher.HWRFLauncher.sanity_check_sanity_check().

def hwrf.launcher.HWRFLauncher.set_storm (   self,
  syndat,
  oldsyndat 
)

Sets the storm that is to be run.

Sets the syndat and oldsyndat member variables, and several related options in the [config] section, to the storm in the provided tcvitals or message file data.

  • config.STID — The three character storm id (ie.: 12L) of the storm to run.
  • config.stnum — the numeric part of config.STID
  • config.basin1 — the basin part of STID (ie.: the L in 12L)
  • config.basin1lc — the lower-case version of config.basin1
Parameters
syndatthe hwrf.storminfo.StormInfo for this cycle's vitals
oldsyndatthe hwrf.storminfo.StormInfo for the prior cycle

Definition at line 722 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.gen_vitals().

def hwrf.launcher.HWRFLauncher.set_storm_multistorm (   self,
  multistorm_real_sids,
  syndat4multistorm,
  oldsyndat4multistorm 
) 
This is meant to be an internal implementation function and
should not be called directly. This is meant to only be used internally
by the fakestorm of a multistorm run.

Adds the syndat_multstorm and oldsyndat_multistorm member
variables for the fake storm.  They contain the StormInfo objects
for all the storm in a multistorm run from the provided tcvitals
or message file data.

It is ultimately used for access to each storm's lat/lon
information in a multistorm run. This is needed for the
swcorner calculation for all the "stormNouter" storms.

Definition at line 747 of file launcher.py.

def hwrf.launcher.HWRFLauncher.storm_for_stormnum (   self)

Not implemented.

This is intended to return the one letter basin, numeric storm ID and year for the specified storm number (1-10).

Bug:
The hwrf.launcher.HWRFLauncher.storm_for_stormnum() is not implemented and should probably be removed.

Definition at line 569 of file launcher.py.

def hwrf.launcher.HWRFLauncher.tcautoseed (   self,
  loud = True 
)

Sets the random seed for ensemble perturbations.

Automatically decides a random seed for the tcvitals perturbation, based on the storm number, basin and cycle. The number and basin used are before the invest renumbering (self.syndat.old()).

Parameters
loudIf loud=True (the default), then a message is sent to the jlogfile via postmsg with the seed, and information about the calculation that went into it.

Definition at line 786 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.gen_vitals().

def hwrf.launcher.HWRFLauncher.timeless_sanity_check (   self,
  enset = None,
  logger = None 
)

Runs all sanity checks that are not dependent on the cycle.

Runs any sanity checks that are possible without knowing the cycle that is to be run. This is intended to be used by the workflow automation system (rocoto, ecflow, etc.) to make sure everything is functional before starting any jobs.

Parameters
enseta set of ensemble ids
loggerthe logging.Logger for log messages

Definition at line 1190 of file launcher.py.

Referenced by hwrf.launcher.HWRFLauncher.sanity_check().

The documentation for this class was generated from the following file:
Generated by   doxygen 1.8.10