improver.calibration.stochastic_noise module

Plugin for adding stochastic noise to a cube using Short-Space Fourier Transform (SSFT).

class StochasticNoise(ssft_init_params=None, ssft_generate_params=None, db_threshold=0.03, db_threshold_units='mm/hr', num_workers=2, scale_non_positive_noise=False, allow_seeded_parallel_processing=False, arbitrary_offset=5.0)

Bases: BasePlugin

Class to apply spatially-structured stochastic noise to non-positive regions of a field, building on the Short-Space Fourier Transform (SSFT) approach from Nerini et al. (2017).

This plugin is intended for use with positive zero-bounded diagnostics only, and is a particularly useful tool for Ensemble Copula Coupling-Quantile (ECC-Q) realization generation. While EEC-Q is used to improve the accuracy of forecasts by calibrating ensemble members to better represent the true distribution of the forecast variable, the rank-based reordering (sorting) of ensemble members at each grid point can lead to unrealistic individual members (e.g. single-pixel precipitation artifacts) when multiple raw ensemble members have identical values (‘ties’) of zero (very common in precipitation forecasts) and the post-processed calibrated probabilities indicate a non-zero value should occur. By adding spatially-structured noise to break ties in these non-positive regions, more realistic spatial structures can be generated in the final ECC-Q realizations, while still respecting the calibrated probabilities.

__init__(ssft_init_params=None, ssft_generate_params=None, db_threshold=0.03, db_threshold_units='mm/hr', num_workers=2, scale_non_positive_noise=False, allow_seeded_parallel_processing=False, arbitrary_offset=5.0)

Initialise the plugin.

If ssft_init_params or ssft_generate_params are not provided, default values from the Pysteps documentation will be used.

Parameters:

• ssft_init_params (Optional[dict]) – Keyword arguments for initializing SSFT filter using pysteps.noise.fftgenerators.initialize_nonparam_2d_ssft_filter. Default is an empty dict, which will use the pysteps defaults.

• ssft_generate_params (Optional[dict]) – Keyword arguments for generating stochastic noise using pysteps.noise.fftgenerators.generate_noise_2d_ssft_filter. Default is an empty dict, which will use the pysteps defaults.

• db_threshold (float) – Threshold value below which data will be set to a constant in dB scale to avoid issues with log(0). Value provided in units of db_threshold_units. Default is 0.03 mm/hr.

• db_threshold_units (str) – Units of the db_threshold value. Default is “mm/hr”.

• num_workers (Optional[int]) – Number of worker threads for parallel FFT computation. If not specified, uses the smaller of the plugin’s default (number of available CPUs) or the number of realizations in the input cube.

• scale_non_positive_noise (bool) – If True, noise in non-positive regions (where template.data <= 0) will be scaled such that the maximum noise value in those regions is zero and all other noise values are negative. This prevents the addition of positive noise to non-positive regions, which could artificially increase values where the input cube indicates no signal should occur. Default is False.

• allow_seeded_parallel_processing (bool) – If True, allows multiple workers to be used even when a seed is provided in ssft_generate_params. This may improve computation speed, but can introduce run-to-run variation because pySTEPS uses global RNG seeding. If False, seeded runs are forced to a single worker for reproducibility. Default is False.

• arbitrary_offset (float) – An arbitrary offset value to add to the dB values of sub-threshold pixels. This is used to ensure that all sub-threshold pixels have a distinct value in dB space, which allows them to be handled appropriately in the _from_dB method. The default value of 5 was chosen to provide a clear separation from the threshold value in dB space, but can be adjusted if needed.

Raises:

ValueError – If db_threshold is not a positive value.

Example dictionaries for initializing and generating SSFT filter:

ssft_init_params = {"win_size": (100, 100), "overlap": 0.3, "war_thr": 0.1}
ssft_generate_params = {"overlap": 0.3, "seed": 0}

See Pysteps documentation for further keyword arguments.

_abc_impl = <_abc._abc_data object>

_from_dB(data)

Convert cube data from dB scale with thresholding.

Function based on dB_transform function (with arg inverse=True) from pySTEPS/pysteps.

Parameters:

data (ndarray) – data in dB scale.

Return type:

ndarray

Returns:

np.ndarray with data converted from dB scale to original scale. Note: After conversion to original scale, values below the threshold are set to zero.

_to_dB(cube)

Convert cube data to dB scale and apply thresholding using db_threshold specified in the plugin initialization.

Function based on dB_transform function (with arg inverse=False) from pySTEPS/pysteps.

Parameters:

cube (Cube) – Cube containing data to be converted to dB scale.

Return type:

Cube

Returns:

Cube with data converted from linear scale to dB scale.

do_fft(data)

Generate stochastic noise using SSFT for a 2-D array slice (one realization).

Parameters:

data (ndarray) – 2D array for which stochastic noise is to be added.

Returns:

2D array of generated stochastic noise.

Return type:

np.ndarray

process(input_cube)

Add locally-conditioned stochastic noise to a cube object using Short-Space Fourier Transform (SSFT).

Parameters:

input_cube (Cube) – Cube to which stochastic noise will be added.

Return type:

Cube

Returns:

Cube with added stochastic noise.

Warning

UserWarning:

If a seed is provided in ssft_generate_params and allow_seeded_parallel_processing is True, a warning is raised to indicate that using multiple workers with a fixed seed may introduce run-to-run variation because pySTEPS uses global RNG seeding.