compare_tropomi_pandora

Code to compare TROPOMI and Pandora column NO2 at high altitude sites to assess skill of TROPOMI at reproducing Pandora observations of free tropospheric NO2.

Code is set up to process Pandora total or tropospheric column NO2 at the Mauna Loa, Izana, or Altzomoni sites.

usage: compare_tropomi_pandora.py [-h] [--trop_dir TROP_DIR]
                                  [--pan_dir PAN_DIR] [--out_dir OUT_DIR]
                                  [--no2_col NO2_COL]
                                  [--cloud_product CLOUD_PRODUCT]
                                  [--pandora_site PANDORA_SITE]
                                  [--str_diff_deg STR_DIFF_DEG]
                                  [--str_diff_min STR_DIFF_MIN]
                                  [--apply_bias_correction APPLY_BIAS_CORRECTION]
                                  [--start_date START_DATE]
                                  [--end_date END_DATE]

optional arguments:
  -h, --help            show this help message and exit
  --trop_dir TROP_DIR
  --pan_dir PAN_DIR
  --out_dir OUT_DIR
  --no2_col NO2_COL     Either Tot or Trop; default is Tot
  --cloud_product CLOUD_PRODUCT
                        options are fresco, dlr-ocra; default is fresco
  --pandora_site PANDORA_SITE
                        options are izana,mauna_loa,altzomoni; default is
                        izana
  --str_diff_deg STR_DIFF_DEG
                        options are: 03,02,01,005; default is 02
  --str_diff_min STR_DIFF_MIN
                        options are: 60,30,15; default is 30
  --apply_bias_correction APPLY_BIAS_CORRECTION
  --start_date START_DATE
                        Start date of processing window (yyyy-mm-dd)
  --end_date END_DATE   End date of processing window (yyyy-mm-dd)
exception uptrop.compare_tropomi_pandora.BadCloudShapeException
exception uptrop.compare_tropomi_pandora.BadNo2ColException
exception uptrop.compare_tropomi_pandora.InvalidCloudProductException
exception uptrop.compare_tropomi_pandora.NoDataException
exception uptrop.compare_tropomi_pandora.NoPandoraException
exception uptrop.compare_tropomi_pandora.UnequalFileException
class uptrop.compare_tropomi_pandora.CloudData(filepath, product_type, tropomi_data=None)

A class containing cloud data extracted from either tropomi data or ocra data.

__init__(filepath, product_type, tropomi_data=None)

Returns an instance of the cloud data needed from filtering. This can come from either a freco cloud product (part of Tropomi) or a dlr-ocra file

Parameters

  • filepath (str) – Path to the file

  • product_type (str) – Can be ‘dlr-ocra’ or ‘fresco’

  • tropomi_data (TropomiData) – An instance of CloudComparisonData. Required if type is ‘fresco’

read_fresco_data(filepath, tropomi_data)

Reads fresco data. Uses tropomi_data to filter for misclassified snow.

read_ocra_data(filepath)

Reads ocra data

class uptrop.compare_tropomi_pandora.DataCollector(start_date, end_date)

Collates tropomi and pandora data for a region around a Pandora site

__init__(start_date, end_date)

Creates a collator between two dates.

Parameters

  • start_date (DateTime) – The start date (inclusive)

  • end_date (DateTime) – The end date (inclusive)

add_pandora_data_to_day(date, hour, diff_hh, pandora_data)

Adds pandora-measured NO2 and error on date at hour to collector Pandora flag threshold selected is from https://www.atmos-meas-tech.net/13/205/2020/amt-13-205-2020.pdf NO2 and error are converted from DU to molec/cm2

Parameters

  • date (DateTime) – The date to search in the pandora data for

  • hour (float) – The hour of the Tropomi overpass

  • diff_hh (float) – The range around hour to sample

  • pandora_data (PandoraData) – The PandoraData object containing the data for date and hour

add_trop_data_to_day(date, trop_data)

Adds the tropomi gc_data, gc_data error, cloud pressure and cloud fraction to a date in this object Call set_trop_ind_for_day before this function

Parameters

  • date (DateTime) – The date to add the data to.

  • trop_data – The tropomi data on a day

apply_weight_to_means()

Applies weighting to every aggregated variable. Call at end of processing.

plot_data()

Time series of daily means

set_trop_ind_for_day(date, diff_deg, trop_data, pandora_data)

Sets tomiind (the index for processing) for a date and area around a pandora site

Parameters

  • date (DateTime) – The date of data to find

  • diff_deg – The size of the grid square over the Pandora site to extract Tropomi data from

  • trop_data (TropomiData) – The CloudComparisonData object containing tropomi data

  • pandora_data (PandoraData) – The PandoraData object containining Pandora data for date

Raises

NoDataException – Raised if there is no tropomi data for date

write_to_netcdf(file)

Saves aggregated data to netcdf

class uptrop.compare_tropomi_pandora.PandoraData(file_path, col_type)

Extracts and preprocesses pandora data from a pandora datafile. See docs for read_pandora for file details

__init__(file_path, col_type)

Returns an instance of PandoraData from file_path. Will apply a correction factor of 0.9 to gc_data and no2_err to bring the product up to ‘pseudo 1.8’. Also applies corrections for Manua Loa if needed

Parameters

  • file_path (str) – Path to the pandora file

  • col_type (str) – Can be ‘Tot’ or ‘Trop’

class uptrop.compare_tropomi_pandora.TropomiData(filepath, apply_bias_correction, no2_col)

A class for reading, preprocessing and cloud-masking Tropomi data files

__init__(filepath, apply_bias_correction, no2_col)

Returns a new instance of CloudComparisonData containing the data from file_path. You can also choose whether to apply bias correction and whethere you want the total or troposphere only column of this data

Parameters

  • filepath (str) – The path to the Tropomi netcdf file

  • apply_bias_correction (bool) – Whether to apply bias correction

  • no2_col (str (can be 'Tot' or 'Trop')) – Whether to use all atmospheric data or just the troposphere

Returns

Returns a new CloudComparisonData instance.

Return type

TropomiData

apply_cloud_filter(cloud_product)

Applies a cloud filter and finishes preprocessing.

Parameters

cloud_product (CloudData) – An instance of CloudData for filtering with

Raises

BadCloudShapeException – Raised if the cloud_product is not the same shape as the Tropomi slice

preprocess()

Prepares the Tropomi data for use. Applies bias correction if needed here. Bias correction to stratosphere and troposphere is obtained in this work from comparison of TROPOMI to Pandora over Mauna Loa (stratospheric column) and Izana and Altzomoni (tropospheric column). The correction is confirmed by also comparing TROPOMI and MAX-DOAS tropospheric columns at Izana.

uptrop.compare_tropomi_pandora.get_days_since_data_start(date, data_start=None)

Returns the number of days since the start date. If no start date is given, assumed 01/06/2019

uptrop.compare_tropomi_pandora.get_ocra_files_on_day(tropomi_dir, date)

Gets a sorted list of tropomi files in tropomi_dir on date

Parameters

  • tropomi_dir (str) – The directory containing tropomi files

  • date (DateTime) – The date to search for

Returns

A list of filepaths to ocra files in the tropomi dir

Return type

list of str

uptrop.compare_tropomi_pandora.get_pandora_file(pan_dir, pandora_site, site_num, c_site, no2_col, fv)

Gets the pandora file for the given set of parameters

uptrop.compare_tropomi_pandora.get_tropomi_files_on_day(tropomi_dir, date)

Gets a sorted list of tropomi files in tropomi_dir on date

Parameters

  • tropomi_dir (str) – The directory containing tropomi files

  • date (DateTime) – The date to search for

Returns

A list of filepaths to tropomi files

Return type

list of str