1/20
DIVA developments and next training: sugges ons for
harmoniza on and improvements of results
C. Troupin, A. Barth & J.-M. Beckers (GHER–ULiège)
3rd Product Mee ng, Plouzané (France), 15 October, 2019 sdn-userdesk@seadatanet.org – www.seadatanet.org
2/20
2/20
Thanks for using DIVAnd
1
Well tested
2Always improvable
3/20
3/20
Why interpola on is complex?
100 000’s of
data points
3/20
Why interpola on is complex?
100 000’s of
data points
Regions without observa ons
3/20
Why interpola on is complex?
100 000’s of
data points
Regions without observa ons
4/20
Developing cu ng-edge interpola on products
4/20
Developing cu ng-edge interpola on products
Back in 2009 at EGU…
Comparison with WOA
Generally: same features Resolution of coastline Artificial mixing
Table:Comparison of computational cost
(Rixen et al., 2001).
OA VIM Matrix inversion 5 1015 5 1013 Analysis (no inv.) 9 1010 2.5 1010
Error (no inv.) 3 1016 2 1015
Nd=300000 , Nφ=500 × 600,
N`=80000
5/20
6/20
Harmonisa on
Ensure consistency…
7/20
3 axes of harmoniza on
Data
Interpola on
Procedure
8/20
Data sources
SeaDataNet
World Ocean
Database
Coriolis Ocean
database for
ReAnalysis
1Which source(s)?
2Which version?
3How to eliminate
duplicates?
8/20
Data sources
SeaDataNet
World Ocean
Database
Coriolis Ocean
database for
ReAnalysis
1Which source(s)?
2Which version?
3How to eliminate
duplicates?
9/20
9/20
10/20
Interpola on tool
Julia version:
≥ 1.0
(now at 1.2)
11/20
Procedure
Domain
Spa al resolu on:
depending on region
Total me coverage:
depending on region
Decade defini ons:
consistency
(merging)
11/20
Procedure
Basic parameters
Bathymetry:
GEBCO or EMODnet Bathymetry, resolu on depending on
domain
Correla on length:
op mized
(if coverage allows) + good
Noise-to-signal ra o:
op mized
(if coverage allows)
Data weights:
op onal
(check sensi vity on a few levels)
11/20
Procedure
Other parameters
surfextend =
true
(ver cal extension at surface)
12/20
2
13/20
Con nuous improvement using user feedback
Event Number of commits
DIVAnd code Notebooks 1st workshop (6 April 2018) 773 265 2nd training course (26 June 2019) 73 7
14/20
Examples of what we’ve changed
V2.0.2 (Aug 21, 2018): matrix alloca on in ODV reading
V2.1.0 (Sep 3, 2018):
▶ Support Julia 1.0
▶ Op miza on in interpola on rou nes
▶ Improved type-stability
V2.1.1 (Oct 19, 2018): allow one to force the direct solver
14/20
Examples of what we’ve changed
V2.0.2 (Aug 21, 2018): matrix alloca on in ODV reading
V2.1.0 (Sep 3, 2018):
▶ Support Julia 1.0
▶ Op miza on in interpola on rou nes
▶ Improved type-stability
V2.1.1 (Oct 19, 2018): allow one to force the direct solver
14/20
Examples of what we’ve changed
V2.0.2 (Aug 21, 2018): matrix alloca on in ODV reading
V2.1.0 (Sep 3, 2018):
▶ Support Julia 1.0
▶ Op miza on in interpola on rou nes
▶ Improved type-stability
V2.1.1 (Oct 19, 2018): allow one to force the direct solver
14/20
Examples of what we’ve changed
V2.0.2 (Aug 21, 2018): matrix alloca on in ODV reading
V2.1.0 (Sep 3, 2018):
▶ Support Julia 1.0
▶ Op miza on in interpola on rou nes
▶ Improved type-stability
V2.1.1 (Oct 19, 2018): allow one to force the direct solver
15/20
Examples of what we’ve changed
V2.3.0 (Jan 24, 2019):
▶ me varying background
▶ edit mask with flood-fill
▶ op miza on
V2.3.1 (Feb 11, 2019): fix interpola on issue for me series near masked grid points (coast line)
V2.4.0 (Jun 25, 2019):
▶ Fixed issue on DIVAnd.fit
▶ diva3d correla on length fi ng using an empty tuple for len
15/20
Examples of what we’ve changed
V2.3.0 (Jan 24, 2019):
▶ me varying background
▶ edit mask with flood-fill
▶ op miza on
V2.3.1 (Feb 11, 2019): fix interpola on issue for me series near masked grid points (coast line)
V2.4.0 (Jun 25, 2019):
▶ Fixed issue on DIVAnd.fit
▶ diva3d correla on length fi ng using an empty tuple for len
15/20
Examples of what we’ve changed
V2.3.0 (Jan 24, 2019):
▶ me varying background
▶ edit mask with flood-fill
▶ op miza on
V2.3.1 (Feb 11, 2019): fix interpola on issue for me series near masked grid points (coast line)
V2.4.0 (Jun 25, 2019):
▶ Fixed issue on DIVAnd.fit
▶ diva3d correla on length fi ng using an empty tuple for len
16/20
Speeding up things
Don’t forget:
Packages are pre-compiled when a kernel is started
Func ons gets compiled during the 1st execu on
16/20
Speeding up things
Data reading
1
Read the original netCDF ODV file. . . .30”
o b s v a l , o b s l o n , o b s l a t , obsdepth , ob st ime , o b s i d = NCODV .l o a d( F l o a t 6 4 , O D V f i l e 1 , ” Water body s a l i n i t y ”) ;
2
Re-write the data . . . 15”
DIVAnd .s a v eobs ( o b s f i l e , ” Water body s a l i n i t y ”, o b s v a l , ( o b s l o n , o b s l a t , obsdepth , o b s t i m e ) , o b s i d )
3
Use the newly wri en files for the climatologies . . . 10”
DIVAnd .s a v eobs ( o b s f i l e , ” Water body s a l i n i t y ”, o b s v a l , ( o b s l o n , o b s l a t , obsdepth , o b s t i m e ) , o b s i d )
17/20
Compu ng weights ”offline”
u s i n g DIVAnd u s i n g J L D d a t a d i r = ” / d a t a / S e a D a t a C l o u d / N o r t h S e a / ” varname = ” S a l i n i t y ” o b s f i l e = j o i n p a t h ( d a t a d i r , ” N o r t h S e a _ o b s . nc ”) netcdfODV = j o i n p a t h ( d a t a d i r , ” data_from_SDC_NS_DATA_DISCRETE_TS_V1b . nc ”) i s f i l e ( netcdfODV ) @info (” R e a d i n g d a t a from t h e o b s e r v a t i o n f i l e ”)
@time o b s v a l , o b s l o n , o b s l a t , obsdepth , obstim e , o b s i d = DIVAnd .l o a dobs ( F l o a t 6 4 , o b s f i l e , varname ) @info (” T o t a l number o f d a t a p o i n t s : $ ( l e n g t h ( o b s v a l ) ) ”) ;
@time r d i a g = 1 . 0 . / DIVAnd . w e i g h t _ R t i m e s O n e ( ( o b s l o n , o b s l a t ) , ( 0 . 0 3 , 0 . 0 3 ) ) ; @show maximum ( r d i a g ) , mean ( r d i a g )
18/20
19/20
Julia is becoming more famous!
20/20
Thanks
for your a en on
1
Shall we upload the products to OceanBrowser?
2
