Download some datasets ---------------------- Here are some scripts to download datasets that can be used to prepare initial and boundary conditions for CROCO, as well as forcings (e.g. atmospheric forcing, rivers). These scripts are compatible with a croco grid passing through the dateline. Then, to read these data during pre-processing, depending on their own formatting, the toolbox uses the configuration file ``readers.jsonc``. See the "Readers for input data" section for more information. Any other dataset can alternatively be added. .. note:: Python API and dap libraries used in the following script are included in conda environnement ``croco_pyenv``. Make sure to be in this environment to execute the scripts. .. _download_mercator_lk: Download Mercator dataset ^^^^^^^^^^^^^^^^^^^^^^^^^ ``download_mercator.py`` allows to download dataset from CMEMS website using the new Copernicus Marine API. Further informations on the API is given here: https://help.marine.copernicus.eu/en/articles/7949409-copernicus-marine-toolbox-introduction A CMEMS account is needed in order to retreive the data. This script takes advantage of "subset" function of copernicusmarine API which allows to download a subset of the original files. Help on usage of the ``download_mercator.py`` script and details on the ``download_mercator.ini`` configuration file can be found by doing: :: python download_mercator.py -h This command also provides names of available variables. All useful parameters and user settings have to be edited in the configuration file (e.g. ``download_mercator.ini``): .. literalinclude:: ../../prepro/Examples/benguela_multifiles/download_mercator_phy.ini :language: ini Each field is detailed in the following table: .. list-table:: * - ``Ystart`` - Requested start year (integer of 4 digits) * - ``Mstart`` - Requested start month (integer) * - ``Dstart`` - Requested start day (integer) * - ``Yend`` - Requested end year (integer of 4 digits) * - ``Mend`` - Requested end month (integer) * - ``Dend`` - Requested end day (integer) * - ``use_grd_extent`` - True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly * - ``grd_extent_padding`` - Buffer (in degrees) added around grid extent * - ``custom_extent`` - Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] * - ``croco_files_dir`` - Croco files directory * - ``croco_grd_prefix`` - Prefix for croco grid file used to request the area extent if use_grd_extent = True * - ``ibc_dir`` - Path to the directory where to store the downloaded data * - ``ibc_prefix`` - Prefix given to downloaded data * - ``dataset`` - Requested CMEMS dataset ID List of all available products at `Copernicus website `_ here a list of commonly used datasets: * Daily Reanalysis : cmems_mod_glo_phy_my_0.083deg_P1D-m * Monthly Reanalysis : cmems_mod_glo_phy_my_0.083deg_P1M-m * Daily Analysis and Forecast : cmems_mod_glo_phy_anfc_0.083deg_P1D-m * - ``variables`` - List of variable names to be extracted Example : ["thetao","so","uo","vo","zos"] See available variable short names in help of the script * - ``depths`` - Depths selection Set to all to download all depths Or provide minimum and maximum depth (in meters) [ min depth, max depth ] Then you can run the script (note: make sure to be in the ``croco_pyenv`` environment): :: python download_mercator.py download_mercator.ini .. _download_hycom_lk: .. tip:: If you need to download data from multiple datasets, you can define several configuration files, one for each dataset. In each configuration file, make sure to define an ``ibc_prefix`` using a common base prefix and a unique dataset-specific name, separated by an underscore (_). This naming convention allows you to use ``ibc_multi_files`` together with ``ibc_prefix`` (for make_ini or make_bry) and the ``ibc_file_*`` parameters (see :ref:`multi_files parameters for make_ini ` ). An example is provided in prepro/Examples/benguela_multifiles, which includes: * ``download_mercator_phy.ini`` * ``download_mercator_bio.ini`` * ``ibc.ini`` In this example, biogeochemical variables are downloaded using the GLORYS_BIO ibc_prefix, whereas classical physical variables are downloaded using the GLORYS_PHY ibc_prefix. This setup makes it possible to select, for each variable, the appropriate input file from a combination of ``ibc_prefix`` = GLORYS and ``ibc_file_*`` = PHY or BIO. Download HYCOM dataset ^^^^^^^^^^^^^^^^^^^^^^ ``download_hycom.py`` allows to download data from HYCOM products around your area of interest. HYCOM products can be explored on the `HYCOM data server `_. .. note:: The script uses HYCOM OpenDAP server. Help on usage of the ``download_hycom.py`` script and details on the ``download_hycom.ini`` configuration file can be found by doing: :: python download_hycom.py -h All useful parameters and user settings have to be edited in the configuration file (e.g. ``download_hycom.ini``): .. literalinclude:: ../../prepro/Examples/download_hycom.ini :language: ini Each field is detailed in the following table: .. list-table:: * - ``Ystart`` - Requested start year (integer of 4 digits) * - ``Mstart`` - Requested start month (integer of 1 or 2 digits) * - ``Dstart`` - Requested start day (integer of 1 or 2 digits) * - ``Yend`` - Requested end year (integer of 4 digits) * - ``Mend`` - Requested end month (integer of 1 or 2 digits) * - ``Dend`` - Requested end day (integer of 1 or 2 digits) * - ``use_grd_extent`` - True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly * - ``grd_extent_padding`` - Buffer (in degrees) added around grid extent * - ``custom_extent`` - Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] * - ``croco_files_dir`` - Croco files directory * - ``croco_grd_prefix`` - Prefix for croco grid file used to request the area extent if use_grd_extent = True * - ``ibc_dir`` - Path to the directory where to store the downloaded data * - ``ibc_prefix`` - Prefix given to downloaded data * - ``dataset`` - OpenDap server url for the requested product Then you can run the script (note: make sure to be in the ``croco_pyenv`` environment): :: python download_hycom.py download_hycom.ini .. _download_era5_lk: Download atmospheric forcing from ERA5 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``download_era5.py`` allows to download data for atmospheric forcing from ECMWF ERA5 reanalysis datasets from the Climate Data Store (CDS) of Copernicus https://cds.climate.copernicus.eu This script uses the `CDS API cds.climate.copernicus.eu `_ for Climate Data Store, to connect and download specific ERA5 variables, for a chosen area and monthly time periods, required by CROCO to perform simulations with atmospheric forcing. The script uses ERA5 parameter names (not parameter IDs) and performs some reformating to match the format expected by CROCO under the CPP key ``ERA_ECMWF`` when using ``ONLINE`` interpolation of atmospheric forcing. Note that native ERA5 file are kept with ``_native.nc`` suffix. .. note:: Follow instructions on `CDS API cds.climate.copernicus.eu `_ to create the file ``$HOME/.cdsapirc`` This file should contain your login and key information like this : :: url: https://cds.climate.copernicus.eu/api key: ***your key value*** Help on usage of the ``download_era5.py`` script and details on the ``download_era5.ini`` configuration file can be found by doing: :: python download_era5.py -h This command also provides all short names of available variables. An example of configuration file is given (``download_era5.ini``): .. literalinclude:: ../../prepro/Examples/benguela_multifiles/download_era5.ini :language: ini Each field is detailed in the following table: .. list-table:: * - ``Ystart`` - Requested start year (integer of 4 digits) * - ``Mstart`` - Requested start month (integer) * - ``Yend`` - Requested end year (integer of 4 digits) * - ``Mend`` - Requested end month (integer) * - ``Yorig`` - Origin year used in processed file * - ``use_grd_extent`` - True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly * - ``grd_extent_padding`` - Buffer (in degrees) added around grid extent * - ``custom_extent`` - Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] * - ``croco_files_dir`` - Croco files directory * - ``croco_grd_prefix`` - Prefix for croco grid file used to request the area extent if use_grd_extent = True * - ``era5_dir`` - Path where to store the downloaded data * - ``n_overlap`` - Overlapping days (at the beginning/end of each month)` * - ``time_frames`` - Requested time (daily hours '00/01/.../23') usual: '00/06/12/18' available: '00/01/02/03/04/05/06/07/08/09/10/11/12/13/14/15/16/17/18/19/20/21/22/23' * - ``variables`` - List of variable short names to be extracted Example : ['lsm','sst','tp','strd','ssr','t2m','q','u10','v10'] See available variable short names in help of the script * - ``conv_cff`` - List of conversion factors between available and requested units in reformatted data Example: [1.,1.,1000./3600.,1./3600.,1./3600.,1.,1. ,1.,1.] See available variable units in help of the script Example for tp: m in 1 hour -> kg m-2 s-1 equal to 1000./3600. Example for heat fluxes: J m-2 in 1 hour -> W m-2 : equal to 1./3600. * - ``units`` - List of requested units in reformatted files Example: ['(0-1)','K','kg m-2 s-1','W m-2','W m-2','K','kg kg-1','m s-1','m s-1'] Carefully check consistency with conversion factor ``conv_cff`` values * - ``wave_extract`` - True/False flag to extract wave variables or not * - ``wave_var`` - List of wave variable short names to be extracted Example: ['swh', 'mwd', 'pp1d' ,'cdww'] * - ``wave_conv_cff`` - List of conversion factors between available and requested units in reformatted data Example: [1., 1., 1. , 1.] * - ``wave_units`` - List of requested units in reformatted files Example: ['m','Degrees true','s', 'dimensionless'] Carefully check consistency with conversion factor ``wave_conv_cff`` values .. _download_glofas_lk: Download river discharge forcing from GloFAS ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``download_glofas_river.py`` allows to download data for river discharges from Global Flood Awareness System (GloFAS) see `CMEMS-GLOFAS `_ It uses `CDS API ewds.climate.copernicus.eu `_ for data request, from CEMS Early Warning Data Store. .. note:: Follow instructions on `CDS API ewds.climate.copernicus.eu `_ to create the file ``$HOME/.cdsapirc`` This file should contain your login and key information like this : :: url: https://ewds.climate.copernicus.eu/api key: ***your key value*** Help on usage of the ``download_glofas_river.py`` script and details on the ``download_glofas_river.ini`` configuration file can be found by doing: :: python download_glofas_river.py -h An example of configuration file is given (``download_glofas_river.ini``): .. literalinclude:: ../../prepro/Examples/benguela_multifiles/download_glofas_river.ini :language: ini Each field is detailed in the following table: .. list-table:: * - ``Ystart`` - Requested start year (integer of 4 digits) * - ``Mstart`` - Requested start month (integer) * - ``Yend`` - Requested end year (integer of 4 digits) * - ``Mend`` - Requested end month (integer) * - ``Yorig`` - Origin year used in processed file * - ``use_grd_extent`` - True: compute extent from CROCO grid file (croco_grd). False: use custom_extent directly * - ``grd_extent_padding`` - Buffer (in degrees) added around grid extent * - ``custom_extent`` - Custom extent, no padding applied. Order is [latmax, lonmin, latmin, lonmax] * - ``croco_files_dir`` - Croco files directory * - ``croco_grd_prefix`` - Prefix for croco grid file used to request the area extent if use_grd_extent = True * - ``dataset`` - Name of the requested dataset * - ``rivers_dir`` - Path of the directory where to store the dowloaded data * - ``rivers_prefix`` - Prefix for the downloaded data .. note:: Download GLOFAS data create monthly files and then concatenate them into one along time axis.