fresco_cld_err

Read and regrid cloud information from the official TROPOMI CLOUD product and from the FRESCO cloud product in the TROPOMI NO2 data product file.

Output from this code includes a NetCDF file with:

  1. Data regridded to a 1 deg x 1 deg global grid and saved as monthly means for scenes with the reference cloud product (specified as input argument) cloud fraction > 0.7 and cloud tops in the upper troposphere (450-180 hPa).

  2. Meridional frequency of upper tropospheric (450-180 hPa) cloud fractions > 0.7 for each product.

usage: fresco_cld_err.py [-h] [--s5p_data_dir S5P_DATA_DIR]
                         [--out_dir OUT_DIR] [--season SEASON]
                         [--start_date START_DATE] [--end_date END_DATE]
                         [--plot_dir PLOT_DIR] [--out_res OUT_RES]
                         [--dlr_cld_top DLR_CLD_TOP]
                         [--file_version FILE_VERSION]
                         [--ref_cld_prod REF_CLD_PROD]

optional arguments:
  -h, --help            show this help message and exit
  --s5p_data_dir S5P_DATA_DIR
  --out_dir OUT_DIR
  --season SEASON       Can be jja, son, djf, mam. Overrides
                        start_date/end_date
  --start_date START_DATE
                        Start date of processing window (yyyy-mm-dd)
  --end_date END_DATE   End date of processing window (yyyy-mm-dd)
  --plot_dir PLOT_DIR
  --out_res OUT_RES
  --dlr_cld_top DLR_CLD_TOP
  --file_version FILE_VERSION
  --ref_cld_prod REF_CLD_PROD
exception uptrop.fresco_cld_err.FileMismatchException

Raised when there are an unequal number of cloud and N02 files

exception uptrop.fresco_cld_err.LatLonException

Raised when there is a LatLon mismatch

exception uptrop.fresco_cld_err.NanInDataException

Raised when filtering has failed

exception uptrop.fresco_cld_err.ShapeMismatchException

Raised when a FRESCO and DLR file are not the same shape

class uptrop.fresco_cld_err.CloudComparisonData(td_file_path, tf_file_path, ref_cld_prd)

Class for holding the data for a fresco-dlr file pair. Applies data filters on creation.

__init__(td_file_path, tf_file_path, ref_cld_prd)

Returns an instance of CloudComparisonData for comparing fresco and dlr files

Parameters

  • td_file_path (str) – The path to the DLR file

  • tf_file_path (str) – The path to the Fresco file

  • ref_cld_prd (str) – The reference cloud product for identifying optically thick clouds in the upper troposphere

Returns

A filtered and sanity-checked cloud comparison

Return type

CloudComparisonData

check_parity()

Checks the shape of fresco and dlr data match

Raises

ShapeMismatchException – Raised when the shapes do not match

filter_tdfile()

Filters DLR data

Applies the following filters to the dlr data Snow: Drops values flagged as snow Poor quality data: quality flag is < 0.5 Drop the following values if DLR-OCRA is the reference cloud product:

  • cloud fraction < 0.7

  • cloud top pressure is between PMAX and PMIN

Raises

NanInDataException – Raised when any NaNs remain in data

filter_tffile()

Filters fresco data

Applies the following filters to the fresco data Snow: Drops values for snow, coastline, snow/ice misclassified as clouds Poor quality data: quality flag is < 0.45 Drop the following values if FRESCO is the reference cloud product:

  • cloud fraction < 0.7

  • cloud top pressure is between PMAX and PMIN

get_nobs()

Gets the number of valid observations in this cloud data

Returns a tuple of number of valid cloud observations of (dlr product, fresco product) A valid observation has:

  • Quality of <0.5 (dlr) or <0.45(fresco)

  • Cloud fraction >= 0.7

  • top-of-atmosphere pressure between PMIN and PMAX

  • No snow (snow mask = 0)

Returns

A tuple of number of valid observations of (dlr, fresco)

Return type

tuple (int, int)

read_tdfile(tdfile)

Read DLR cloud data

Parameters

tdfile (str) – Path to the DLR file

read_tffile(tffile)

Read in FRESCO cloud data

Parameters

tffile (str) – Path to the tffile

class uptrop.fresco_cld_err.CloudVariableStore(data_shape, start_date, end_date, res, cloud_prod)

Class containing the running results of various datas for this project; also saving to netCDF and plotting.

__init__(data_shape, start_date, end_date, res, cloud_prod)

Creates an empty CloudVariableStore of shape data_shape

Parameters

data_shape (List of length 2) – Tuple of (presumably) two elements for shape of cloud data

bin_cloud_fraction(tropomi_data)

# Gather data on frequency of cloud fraction >= 0.7: # This is done before filtering for scenes with knmi cloud frac # >= 0.7 to also include all relevant DLR scenes:

Parameters

tropomi_data (CloudComparisonData) – The tropomi data containing the cloud fractions

calc_cloud_statistics()

Finalises cloud statistics

For each pixel in the final model, calculate the cloud fraction, cloud top pressure, cloud albedo and (for DLR) cloud base pressure. Cloud pressure is given in Pa.

plot_clouds_products(plot_dir)

For cloud fraction, cloud top pressure, plots the products for DLR, FRESCO and DLR-FRESCO

Parameters

plot_dir (str) – The directory that will contain the plots

update_nobs(tropomi_data)

Given a tropomi_data object, updates the number of observations.

Parameters

tropomi_data (CloudComparisonData) – The tropomi data objects

update_pixel(tropomi_data, trop_i, trop_j)

Updates the appropriate pixel of running cloud variables with the tropomi data at trop_i, trop_j.

Parameters

  • tropomi_data (CloudComparisonData) – The tropomi dataset

  • trop_i (int) – First index of pixel

  • trop_j (int) – Second index of pixel

write_to_netcdf(out_dir)

Given an out_directory, writes totalled data to netcdf.

The file will be named ‘fresco-dlr-cloud-products-[MMName]-[StrYY]-[out_res]-ref-[ref_cld_prod]-v3.nc’

Parameters

out_dir (str) – The directory that will contain the file.

uptrop.fresco_cld_err.get_files_for_month(sen_5_p_dir, month_index, ndays=31)

Gets fresco an For a given month index (jan-may 2020 being 1-5, jun-dec 2019 being 6-12), returns every DLR and Fresco file_path. Also sets the globals StrMM, StrYY, MMName (used in the plotting method)(at least until I fix it)

Parameters

  • sen_5_p_dir (str) – The directory containing the DLR and Fresco files

  • month_index (int) – The month index

  • ndays (int) – The number of days in the month

Raises

FileMismatchException – Raised if there is an unequal number of fresco and dlr files

Returns

The list of dlr files and the list of fresco files

Return type

tuple(dlr files, fresco files)

uptrop.fresco_cld_err.process_file(tdfile, tffile, running_total_container)

Processes a paired dlr and fresco product, adds the pixels to the running total in the running_total_container and updates the cloud_fraction

Parameters

  • tdfile (str) – Path to the dlr file

  • tffile (str) – Path to the fresco file

  • running_total_container (CloudVariableStore) – Instance of CloudVariableStore to update with the dlr and fresco file