21. XIOS¶
- As a start point for this tutorial, we will use the BASIN test case (see section 5.1)
cd ~/CONFIGS/BASIN
Is everything ok ? Compiling ? Running ? Are the 2 files basin_rst.nc and basin_his.nc created ?
What is the walltime (or real time)?
Now add the XIOS functionnality in the croco executable:
If the XIOS is installed on your target machine (it is the case on Datarmor), there are only 2 new simple steps to follow :
Edit
cppdef.h
:Need to define 2 news cpp keys fot this test case:
/* ! Basin Example ! ===== ======= */ # define XIOS # undef OPENMP .....
Edit the compilation script
jobcomp
:Need to add the XIOS library path
# # set XIOS directory if needed # XIOS_ROOT_DIR=$HOME/xios-2.5 #
For this tutorial, we need to comment three lines (217, 218 and 219) in
jobcomp
:# $CPP1 -P -traditional -imacros cppdefs.h ${ROOT_DIR}/XIOS/field_def.xml_full $RUNDIR/field_def.xml # $CPP1 -P -traditional -imacros cppdefs.h ${ROOT_DIR}/XIOS/domain_def.xml $RUNDIR/domain_def.xml # $CPP1 -P -traditional -imacros cppdefs.h ${ROOT_DIR}/XIOS/iodef.xml $RUNDIR/iodef.xml
For this tutorial, we need to modify the routine
send_xios_diags.F
:cp ~/croco/croco/XIOS/send_xios_diags.F .
Edit send_xios_diags.F and comment lines 2077, 2078 and 2133:
! call xios_send_field("uwnd",uwnd) ! call xios_send_field("vwnd",vwnd) ! call xios_send_field("bvf",bvf)
Compile the model again:
./jobcomp
Before running the model with XIOS module, we need three xml files (field_def.xml, domain_def.xml and iodef.xml) :
cp /home/datawork-croco/datarmor-only/CONFIGS/TUTO20/BASIN_WITH_XIOS/field_def.xml . cp /home/datawork-croco/datarmor-only/CONFIGS/TUTO20/BASIN_WITH_XIOS/domain_def.xml . cp /home/datawork-croco/datarmor-only/CONFIGS/TUTO20/BASIN_WITH_XIOS/iodef.xml.OneFile iodef.xmlHave a look at iodef.xml file :
selected fields to output,
output frequency
output_freq
,what kind of output (instantaneous, average)
operation
,…
At the end of the iodef.xml file, look at the line
<variable id="using_server" type="bool">false</variable>The boolean
false
means that croco will run with XIOS in “attached mode”. Each computing processor will write in the output file. In this “attached mode”, XIOS behaves like a netcdf4 layer.In this
iodef.xml
file, the configuration for outputs is the same as in croco.in file.
Run the model in “attached mode”:
qsub job_croco_mpi.pbsCompare the new file
Basin_Example_10d_inst_0001-01-01-0001-04-30.nc
with the previous onebasin_his.nc
:ncview basin_his.nc & ; ncview Basin_Example_10d_inst_0001-01-01-0001-04-30.ncNote
If your output file start with a
?
, it is due to a tab before the configuration title in croco.in:Basin Example
. Just replace the tab by a blank space.—> For large configuration, XIOS is very efficient in netcdf parallel writting.
Edit iodef.xml file and add new 2D and 3D fields to be written in the output file by uncommenting lines :
<field field_ref="w" name="w" /> <field field_ref="salt" name="salt" /> <field field_ref="sustr" name="sustr" /> <field field_ref="svstr" name="svstr" /> <field field_ref="rho" name="rho" />Run the model:
qsub job_croco_mpi.pbsHave a look at the new file
Basin_Example_10d_inst_0001-01-01-0001-04-30.nc
Add an extra file for average output in editing iodef.xml (or you can get an example there):
cp /home/datawork-croco/datarmor-only/CONFIGS/TUTO20/BASIN_WITH_XIOS/iodef.xml.Twofiles iodef.xmlHave a look at the iodef.xml file to understand how to simply add a new output file
run the model:
qsub job_croco_mpi.pbsHave a look at the new netcdf file
Basin_Example_5d_aver_0001-01-01-0001-04-30.nc
What is the walltime (or real time)?
Run the model in “detached mode”:
edit iodef.xml and modify boolean at ``true`` in line:
<variable id="using_server" type="bool">true</variable>The boolean
true
means that croco will run with XIOS in “detached mode”. Each computing processor will send fields to one or several XIOS servers which will be in charge of writing the outputs files.Edit job_croco_mpi.pbs to add one XIOS server
##PBS -l select=1:ncpus=28:mpiprocs=4:mem=8g #PBS -l select=1:ncpus=28:mpiprocs=5:mem=8g #time $MPI_LAUNCH croco croco.in >& croco.out time $MPI_LAUNCH -n 4 croco croco.in : -n 1 xios_server.exe >& croco.outThere will be 4 computing processors sending fields to 1 xios server writting in output files.
Run the model:
qsub job_croco_mpi.pbsTheorically, computing processors will run faster (keep in mind that reading and writting files is slow, computing is fast!).
What is the walltime (or real time)?
Is it worth to use detached mode in this case?
Adding an online dignostic using ONLY xios:
In the output file, we need to have a new variable computed from already defined variables. For instance, we want to have zeta*zeta …
Edit
field_def.xml
and add the new variable zeta2:<field id="zeta" long_name="free-surface" unit="meter" /> <field id="zeta2" long_name="squared free-surface" unit="meter2" > (zeta*zeta) </field>Then edit
iodef.xml
and add the new variable to be written in the output file:<field_group id="inst_fields" operation="instant"> <field field_ref="zeta" name="zeta" /> <field field_ref="zeta2" name="zeta2" />No need to compile, just run the model:
qsub job_croco_mpi.pbs
If you have time, add xios in the previous BENGUELA_LR
cd $confs/Run_BENGUELA_LR cp /home/datawork-croco/datarmor-only/CONFIGS/TUTO20/BENGUELA_LR_XIOS/* .Compile once:
./jopbcompRun :
qsub job_croco_mpi.pbsExplore files, edit and modify iodef.xml, and run again …