2.9. Download some datasets#
Here are some scripts to download datasets that can be used to prepare initial and boundary conditions for CROCO, as well as forcings (e.g. atmospheric forcing, rivers).
These scripts are compatible with a croco grid passing through the dateline.
Then, to read these data during pre-processing, depending on their own formatting,
the toolbox uses the configuration file readers.jsonc.
See the “Readers for input data” section for more information.
Any other dataset can alternatively be added.
Note
Python API and dap libraries used in the following script are included in
conda environnement croco_pyenv.
Make sure to be in this environment to execute the scripts.
2.9.1. Download Mercator dataset#
download_mercator.py allows to download dataset from
CMEMS website using the new Copernicus Marine API.
Further informations on the API is given here: https://help.marine.copernicus.eu/en/articles/7949409-copernicus-marine-toolbox-introduction
A CMEMS account is needed in order to retreive the data.
This script takes advantage of “subset” function of copernicusmarine API which allows to download a subset of the original files.
Help on usage of the download_mercator.py script and details
on the download_mercator.ini configuration file
can be found by doing:
python download_mercator.py -h
This command also provides names of available variables.
All useful parameters and user settings have to be edited in the configuration
file (e.g. download_mercator.ini):
# use ./download_mercator.py -h for more information about parameters
[Times]
Ystart = 2013
Mstart = 1
Dstart = 1
Yend = 2013
Mend = 1
Dend = 31
Yorig = 2000
Morig = 1
Dorig = 1
[Download_Options]
use_grd_extent = True
grd_extent_padding = 1.0
custom_extent = None
[Croco_Files]
croco_files_dir = ../results/CROCO_FILES
croco_grd_prefix = croco_grd
[IBC_Input_Files]
ibc_dir = ../data/DATA_IBC/GLORYS/
ibc_prefix = GLORYS
[Mercator_Download]
dataset = cmems_mod_glo_phy_my_0.083deg_P1D-m
variables = ["thetao"]
depths = [0,50]
Each field is detailed in the following table:
|
Requested start year (integer of 4 digits) |
|
Requested start month (integer) |
|
Requested start day (integer) |
|
Requested end year (integer of 4 digits) |
|
Requested end month (integer) |
|
Requested end day (integer) |
|
True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly |
|
Buffer (in degrees) added around grid extent |
|
Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] |
|
Croco files directory |
|
Prefix for croco grid file used to request the area extent if use_grd_extent = True |
|
Path to the directory where to store the downloaded data |
|
Prefix given to downloaded data |
|
Requested CMEMS dataset ID List of all available products at Copernicus website here a list of commonly used datasets:
|
|
List of variable names to be extracted Example : [“thetao”,”so”,”uo”,”vo”,”zos”] See available variable short names in help of the script |
|
Depths selection Set to all to download all depths Or provide minimum and maximum depth (in meters) [ min depth, max depth ] |
Then you can run the script
(note: make sure to be in the croco_pyenv environment):
python download_mercator.py download_mercator.ini
2.9.2. Download HYCOM dataset#
download_hycom.py allows to download data from HYCOM products around your
area of interest.
HYCOM products can be explored on the HYCOM data server.
Note
The script uses HYCOM OpenDAP server.
Help on usage of the download_hycom.py script and details
on the download_hycom.ini configuration file
can be found by doing:
python download_hycom.py -h
All useful parameters and user settings have to be edited in the configuration
file (e.g. download_hycom.ini):
# use ./download_hycom.py -h for more information about parameters
[Times]
Ystart = 2019
Mstart = 1
Dstart = 1
Yend = 2019
Mend = 1
Dend = 2
Yorig = 2000
Morig = 1
Dorig = 1
[Download_Options]
use_grd_extent = False
grd_extent_padding = 1.0
custom_extent = [50.5,-1.5,49.3,1.0]
[Croco_Files]
croco_files_dir = ../results/CROCO_FILES
croco_grd_prefix = croco_grd
[IBC_Input_Files]
ibc_dir = ../data/DATA_IBC/HYCOM/
ibc_prefix = HYCOM
[Hycom_Download]
dataset = dap2://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0
Each field is detailed in the following table:
|
Requested start year (integer of 4 digits) |
|
Requested start month (integer of 1 or 2 digits) |
|
Requested start day (integer of 1 or 2 digits) |
|
Requested end year (integer of 4 digits) |
|
Requested end month (integer of 1 or 2 digits) |
|
Requested end day (integer of 1 or 2 digits) |
|
True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly |
|
Buffer (in degrees) added around grid extent |
|
Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] |
|
Croco files directory |
|
Prefix for croco grid file used to request the area extent if use_grd_extent = True |
|
Path to the directory where to store the downloaded data |
|
Prefix given to downloaded data |
|
OpenDap server url for the requested product |
Then you can run the script
(note: make sure to be in the croco_pyenv environment):
python download_hycom.py download_hycom.ini
2.9.3. Download atmospheric forcing from ERA5#
download_era5.py allows to download data for atmospheric forcing
from ECMWF ERA5 reanalysis datasets from the Climate
Data Store (CDS) of Copernicus https://cds.climate.copernicus.eu
This script uses the CDS API to connect and download specific ERA5 variables, for a chosen area and monthly time periods, required by CROCO to perform simulations with atmospheric forcing.
The script uses ERA5 parameter names (not parameter IDs) and performs some reformating
to match the format expected by CROCO under the CPP key ERA_ECMWF when using ONLINE
interpolation of atmospheric forcing.
Note that native ERA5 file are kept with _native.nc suffix.
Note
Follow instructions on CDS API
to create the file $HOME/.cdsapirc This file should contain your login and
key information like this :
url: https://cds.climate.copernicus.eu/api
key: ***your key value***
Help on usage of the download_era5.py script and details on the
download_era5.ini configuration file can be found by doing:
python download_era5.py -h
This command also provides all short names of available variables.
An example of configuration file is given (download_era5.ini):
# use ./download_era5.py -h for more information about parameters
[Times]
Ystart = 2013
Mstart = 1
Dstart = 1
Yend = 2013
Mend = 1
Dend = 31
Yorig = 2000
Morig = 1
Dorig = 1
[Download_Options]
use_grd_extent = True
grd_extent_padding = 1.0
custom_extent = None
[Croco_Files]
croco_files_dir = ../results/CROCO_FILES
croco_grd_prefix = croco_grd
[ERA5_Download]
era5_dir = ../data/DATA_METEO/ERA5/
n_overlap = 0
time_frames = 00/06/12/18
variables = ['tp','t2m']
conv_cff = [1000./3600., 1.]
units = ['kg m-2 s-1','K']
wave_extract = False
wave_var = ['swh', 'mwd', 'pp1d' ,'cdww']
wave_conv_cff = [1., 1., 1. , 1.]
wave_units = ['m','Degrees true','s', 'dimensionless']
Each field is detailed in the following table:
|
Requested start year (integer of 4 digits) |
|
Requested start month (integer) |
|
Requested end year (integer of 4 digits) |
|
Requested end month (integer) |
|
Origin year used in processed file |
|
True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly |
|
Buffer (in degrees) added around grid extent |
|
Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] |
|
Croco files directory |
|
Prefix for croco grid file used to request the area extent if use_grd_extent = True |
|
Path where to store the downloaded data |
|
Overlapping days (at the beginning/end of each month)` |
|
|
|
List of variable short names to be extracted Example : [‘lsm’,’sst’,’tp’,’strd’,’ssr’,’t2m’,’q’,’u10’,’v10’] See available variable short names in help of the script |
|
List of conversion factors between available and requested units in reformatted data Example: [1.,1.,1000./3600.,1./3600.,1./3600.,1.,1. ,1.,1.] See available variable units in help of the script Example for tp: m in 1 hour -> kg m-2 s-1 equal to 1000./3600. Example for heat fluxes: J m-2 in 1 hour -> W m-2 : equal to 1./3600. |
|
List of requested units in reformatted files
Example: [‘(0-1)’,’K’,’kg m-2 s-1’,’W m-2’,’W m-2’,’K’,’kg kg-1’,’m s-1’,’m s-1’]
Carefully check consistency with conversion factor |
|
True/False flag to extract wave variables or not |
|
List of wave variable short names to be extracted Example: [‘swh’, ‘mwd’, ‘pp1d’ ,’cdww’] |
|
|
|
|
2.9.4. Download river discharge forcing from GloFAS#
download_glofas_river.py allows to download data for river discharges
from Global Flood Awareness System (GloFAS) see
CMEMS-GLOFAS
It uses CDS API for data request.
Note
Follow instructions on CDS API
to create the file $HOME/.cdsapirc This file should contain your login and
key information like this :
url: https://ewds.climate.copernicus.eu/api
key: ***your key value***
Help on usage of the download_glofas_river.py script and details on the
download_glofas_river.ini configuration file can be found by doing:
python download_glofas_river.py -h
An example of configuration file is given (download_glofas_river.ini):
# use ./download_glofas_river.py -h for more information about parameters
[Times]
Ystart = 2013
Mstart = 1
Dstart = 1
Yend = 2013
Mend = 1
Dend = 31
Yorig = 2000
Morig = 1
Dorig = 1
[Download_Options]
use_grd_extent = True
grd_extent_padding = 1.0
custom_extent = None
[Croco_Files]
croco_files_dir = ../results/CROCO_FILES
croco_grd_prefix = croco_grd
[Glofas_Download]
dataset = cems-glofas-historical
rivers_dir = ../data/DATA_RIVERS/
rivers_prefix = cems_glofas
Each field is detailed in the following table:
|
Requested start year (integer of 4 digits) |
|
Requested start month (integer) |
|
Requested end year (integer of 4 digits) |
|
Requested end month (integer) |
|
Origin year used in processed file |
|
True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly |
|
Buffer (in degrees) added around grid extent |
|
Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] |
|
Croco files directory |
|
Prefix for croco grid file used to request the area extent if use_grd_extent = True |
|
Name of the requested dataset |
|
Path of the directory where to store the dowloaded data |
|
Prefix for the downloaded data |
Note
Download GLOFAS data create monthly files and then concatenate them into one along time axis.