Welcome to Uptrop’s documentation!
Here you will find all the relevant details of the functionality of the code used to generate results described and discussed in Marais et al. (2020).
The python code was developed for the ERC Starting Grant project UpTrop (https://maraisresearchgroup.co.uk/uptrop.html) led by Eloise Marais. Substantive code edits and improvements and documentation preparation are by software engineer John Roberts.
Contents:
Introduction
A library for applying the cloud-slicing technique to TROPOMI total columns to obtain upper tropospheric (450-180 hPa) mixing ratios of NO2. This includes a feasibility test of the technique using synthetic observations from GEOS-Chem, validation and quantification of bias corrections by comparing TROPOMI and Pandora total columns at high-altitide sites, application of cloud-slicing using either the FRESCO-S or ROCINN-CAL cloud products, and comparison of effective cloud fractions and cloud top pressures from the two cloud products.
In the modules, the FRESCO-S product is referred to as FRESCO and the ROCINN-CAL product as DLR.
A description of the data used, the sources of the data, the details of the GEOS-Chem model simulations, and other relevant information for developing the codes can be found in Marais et al. (2020).
The easiest way to use the library is to call the scripts from a terminal; see the rest of this page for examples. You can also import the components into your own scripts - see the individual module descriptions linked in the sidebar.
Requirements
Python >= 3.6
Numpy
NetCDF4
Dateutil
Matplotlib
Basemap
If you are using Conda, an environment.yml file is provided.
Examples of use
Cloud-slice synthetic partial columns from GEOS-Chem over North America at 4x5 in Spring:
python ut_no2_gc_test.py --gc_dir "path/to/geoschem/data" --out_path "/path/to/output/file" --resolution 4x5 --season mam --region NA
Co-sample daily Pandora and TROPOMI total NO2 columns at Izana in the first week of May:
python compare_tropomi_pandora.py --trop_dir "/path/to/tropomi/data" --pan_dir "/path/to/pandora/data" --out_dir "/path/to/output/dir" --start_date 01-05-2019 --end_date 07-05-2019 --apply_bias_correction True --no2_col Trop --pandora_site izana
Cloud-slice Jun-Aug TROPOMI partial NO2 columns using the FRESCO-S cloud product in Summer:
python tropomi_ut_no2.py --trop_dir "/path/to/tropomi/data/folder/" --out_dir "/path/to/output/folder/" --cloud_product fresco --season jja
Obtain January mean FRESCO-S and ROCINN-CAL cloud top pressures and fractions at 1x1:
python fresco_cld_err.py --trop_dir "/path/to/tropomi/data/folder/" --out_dir "/path/to/output/folder/" --start_date 01-01-2019 --end_date 31-01-2019 --out_res 1x1 --dlr_cld_top height
For a more advanced example bash script that runs each script over a given date range, see tests/integration_test.sh in your Uptrop folder.
Conventions
All scripts take a path to a directory containing the relevant data (
--trop_dir,--pan_dirand--gc_dir)All scripts need either a
--seasonor a--start_dateand--end_dateprovided. If all are given,--seasonwill override start_date/end_dateIf
--seasonis given, then this script runs for that season 2019-2020:djf; dececmber-januarary-februray 2019-2020mam; march-april-may 2020jja; june-july-august 2020son; september-october-november 2020--start_dateand--end_dateare given as dd-mm-yyyy; for example, 20th Januarary 2019 is 20-01-2019
Scripts
ut_no2_gc_test
Use synthetic partial columns from GEOS-Chem to obtain cloud-sliced NO2 in the upper troposphere and compare this to UT NO2 obtained if simply average the NO2 mixing ratios from the model over the same pressure range (the “truth”). Both are obtained by Gaussian weighting toward the pressure center.
GEOS-Chem partial columns are obtained over Europe, North America, and China at the GEOS-FP meteorology native resolution (0.25x0.3125) (latxlon) for June-August 2016-2017.
Input options to process the data include the region, the horizontal resolution, and model simulation years.
usage: ut_no2_gc_test.py [-h] [--gc_dir GC_DIR] [--out_dir OUT_DIR]
[--resolution RESOLUTION] [--region REGION]
[--strat_filter_threshold STRAT_FILTER_THRESHOLD]
[--start_date START_DATE]
[--end_date END_DATE] [-p PLOT]
[--do_temp_correct DO_TEMP_CORRECT]
[--apply_cld_frac_filter APPLY_CLD_FRAC_FILTER]
[--do_cld_hght_test DO_CLD_HGHT_TEST]
optional arguments:
-h, --help show this help message and exit
--gc_dir GC_DIR
--out_dir OUT_DIR
--resolution RESOLUTION
Can be 8x10, 4x5, 2x25 or 1x1
--region REGION Can be EU, NA, or CH
--strat_filter_threshold STRAT_FILTER_THRESHOLD
--start_date START_DATE
--end_date END_DATE
-p PLOT, --plot PLOT
--do_temp_correct DO_TEMP_CORRECT
--apply_cld_frac_filter APPLY_CLD_FRAC_FILTER
--do_cld_hght_test DO_CLD_HGHT_TEST
cloud_slice_ut_no2
Cloud-slicing steps applied to a cluster of data using as input the partial NO2 columns in molecules/m2 and cloud top heights in hPa.
If successful, the output is NO2 mixing ratios in pptv. Other output is the estimated error on the NO2 mixing ratio and the mean cloud top pressure (hPa) for the cluster.
If the cloud-slicing step is unsuccessful, all values are NaN and a reason the cloud-slicing failed is provided.
Use as part of a Python script:
from uptrop.cloud_slice_ut_no2 import cldslice, CLOUD_SLICE_ERROR_ENUM
# Dummy functions
partial_columns = get_partial_cloud_columns()
cloud_top_heights = get_cloud_top_heights()
ratio, ratio_error, exception_number, mean_cloud_error = cldslice(partial_columns, cloud_top_heights)
if exception_number != 0:
print(CLOUD_SLICE_ERROR_ENUM[exception_number])
print("Cloud ratio: {}".format(ratio))
print("Cloud ratio error: {}".format(ratio_error))
print("Mean cloud error: {}".format(mean_cloud_error))
compare_tropomi_pandora
Code to compare TROPOMI and Pandora column NO2 at high altitude sites to assess skill of TROPOMI at reproducing Pandora observations of free tropospheric NO2.
Code is set up to process Pandora total or tropospheric column NO2 at the Mauna Loa, Izana, or Altzomoni sites.
usage: compare_tropomi_pandora.py [-h] [--trop_dir TROP_DIR]
[--pan_dir PAN_DIR] [--out_dir OUT_DIR]
[--no2_col NO2_COL]
[--cloud_product CLOUD_PRODUCT]
[--pandora_site PANDORA_SITE]
[--str_diff_deg STR_DIFF_DEG]
[--str_diff_min STR_DIFF_MIN]
[--apply_bias_correction APPLY_BIAS_CORRECTION]
[--start_date START_DATE]
[--end_date END_DATE]
optional arguments:
-h, --help show this help message and exit
--trop_dir TROP_DIR
--pan_dir PAN_DIR
--out_dir OUT_DIR
--no2_col NO2_COL Either Tot or Trop; default is Tot
--cloud_product CLOUD_PRODUCT
options are fresco, dlr-ocra; default is fresco
--pandora_site PANDORA_SITE
options are izana,mauna_loa,altzomoni; default is
izana
--str_diff_deg STR_DIFF_DEG
options are: 03,02,01,005; default is 02
--str_diff_min STR_DIFF_MIN
options are: 60,30,15; default is 30
--apply_bias_correction APPLY_BIAS_CORRECTION
--start_date START_DATE
Start date of processing window (yyyy-mm-dd)
--end_date END_DATE End date of processing window (yyyy-mm-dd)
fresco_cld_err
Read and regrid cloud information from the official TROPOMI CLOUD product and from the FRESCO cloud product in the TROPOMI NO2 data product file.
Output from this code includes a NetCDF file with:
Data regridded to a 1 deg x 1 deg global grid and saved as monthly means for scenes with the reference cloud product (specified as input argument) cloud fraction > 0.7 and cloud tops in the upper troposphere (450-180 hPa).
Meridional frequency of upper tropospheric (450-180 hPa) cloud fractions > 0.7 for each product.
usage: fresco_cld_err.py [-h] [--s5p_data_dir S5P_DATA_DIR]
[--out_dir OUT_DIR] [--season SEASON]
[--start_date START_DATE] [--end_date END_DATE]
[--plot_dir PLOT_DIR] [--out_res OUT_RES]
[--dlr_cld_top DLR_CLD_TOP]
[--file_version FILE_VERSION]
[--ref_cld_prod REF_CLD_PROD]
optional arguments:
-h, --help show this help message and exit
--s5p_data_dir S5P_DATA_DIR
--out_dir OUT_DIR
--season SEASON Can be jja, son, djf, mam. Overrides
start_date/end_date
--start_date START_DATE
Start date of processing window (yyyy-mm-dd)
--end_date END_DATE End date of processing window (yyyy-mm-dd)
--plot_dir PLOT_DIR
--out_res OUT_RES
--dlr_cld_top DLR_CLD_TOP
--file_version FILE_VERSION
--ref_cld_prod REF_CLD_PROD
tropomi_ut_no2
Process and apply the cloud-slicing approach to partial columns of NO2 from S5P/TROPOMI for June 2019 to May 2020.
The default is to obtain seasonal means at 1x1 for partial columns above clouds with cloud fraction >=0.7 and within the cloud top pressure range of 450-180 hPa.
Options are available to use cloud information from either the FRESCO-S or ROCINN-CAL cloud product, to obtain seasonal means at 2x2.5 or 4x5, and to use a cloud fraction threshold of 0.8, 0.9, or 1.0.
usage:
[-h] [--trop_dir TROP_DIR] [--out_dir OUT_DIR] [--season SEASON]
[--start_date START_DATE] [--end_date END_DATE] [--grid_res GRID_RES]
[--cloud_product CLOUD_PRODUCT] [--cloud_threshold CLOUD_THRESHOLD]
[--pmin PMIN] [--pmax PMAX]
optional arguments:
-h, --help show this help message and exit
--trop_dir TROP_DIR Directory containing tropomi data
--out_dir OUT_DIR Directory to contain finished netcdf4
--season SEASON Can be jja, son, djf, mam
--start_date START_DATE
Start date of processing window (yyyy-mm-dd)
--end_date END_DATE End date of processing window (yyyy-mm-dd)
--grid_res GRID_RES Can be 1x1, 2x25, 4x5
--cloud_product CLOUD_PRODUCT
can be fresco or dlr-ocra
--cloud_threshold CLOUD_THRESHOLD
recommended value is 07. Can also test 08, 09, 10
--pmin PMIN Lower bound on cloud height. Defaults to 180.
--pmax PMAX Upper bound on cloud height. Defaults to 450.