improver.utilities.compare module

Compare netcdf files using python-netCDF4 library.

This comparison is done using netCDF4 rather than iris so that it is not coupled to changes in netCDF representation across iris versions. It is also able to compare non-CF conventions compliant netCDF files that iris has difficulty loading.

Many functions in this module take an argument called ‘reporter’ which is a function to be called to report comparison differences. This provides flexibility regarding what action should be taken when differences are found. For example, the action to take could be to print the message, log the message, or raise an appropriate exception.

compare_attributes(name, actual_ds, desired_ds, ignored_attributes, reporter)

Compare attributes in a netCDF dataset/group.

Parameters:

• name (str) – group name

• actual_ds (Dataset) – dataset produced by test run

• desired_ds (Dataset) – dataset considered good

• ignored_attributes (Optional[List[str]]) – list of attributes to exclude from comparison.

• reporter (Callable[[str], None]) – callback function for reporting differences

Return type:

None

compare_data(name, actual_var, desired_var, rtol, atol, reporter)

Compare attributes in a netCDF variable.

Parameters:

• name (str) – variable name

• actual_var (Variable) – variable produced by test run

• desired_var (Variable) – variable considered good

• rtol (float) – relative tolerance

• atol (float) – absolute tolerance

• reporter (Callable[[str], None]) – callback function for reporting differences

Return type:

None

compare_datasets(name, actual_ds, desired_ds, rtol, atol, exclude_vars, ignored_attributes, reporter)

Compare netCDF datasets. This function can call itself recursively to handle nested groups in netCDF4 files which are represented using the same Dataset class by python-netCDF4.

Parameters:

• name (str) – group name

• actual_ds (Dataset) – dataset produced by test run

• desired_ds (Dataset) – dataset considered good

• rtol (float) – relative tolerance

• atol (float) – absolute tolerance

• exclude_vars (List[str]) – variable names to exclude from comparison

• ignored_attributes (Optional[List[str]]) – list of attributes to exclude from comparison.

• reporter (Callable[[str], None]) – callback function for reporting differences

Return type:

None

compare_dims(name, actual_ds, desired_ds, exclude_vars, reporter)

Compare dimensions in a netCDF dataset/group.

Parameters:

• name (str) – group name

• actual_ds (Dataset) – dataset produced by test run

• desired_ds (Dataset) – dataset considered good

• exclude_vars (List[str]) – variable names to exclude from comparison

• reporter (Callable[[str], None]) – callback function for reporting differences

Return type:

None

compare_netcdfs(actual_path, desired_path, rtol=0.0001, atol=0.0001, exclude_vars=None, ignored_attributes=None, reporter=None)

Compare two netCDF files.

Parameters:

• actual_path (PathLike) – data file produced by test run

• desired_path (PathLike) – data file considered good eg. KGO

• rtol (float) – relative tolerance

• atol (float) – absolute tolerance

• exclude_vars (Optional[List[str]]) – variable names to exclude from comparison

• ignored_attributes (Optional[List[str]]) – list of attributes to exclude from comparison.

• reporter (Optional[Callable[[str], None]]) – callback function for reporting differences

Return type:

None

compare_vars(name, actual_ds, desired_ds, rtol, atol, exclude_vars, ignored_attributes, reporter)

Compare variables in a netCDF dataset/group.

Parameters:

• name (str) – group name

• actual_ds (Dataset) – dataset produced by test run

• desired_ds (Dataset) – dataset considered good

• rtol (float) – relative tolerance

• atol (float) – absolute tolerance

• exclude_vars (List[str]) – variable names to exclude from comparison

• ignored_attributes (Optional[List[str]]) – list of attributes to exclude from comparison.

• reporter (Callable[[str], None]) – callback function for reporting differences

Return type:

None