Differences between revisions 44 and 45
Revision 44 as of 2014-06-25 14:37:37
Size: 8652
Editor: ?AlastairCMcKinstry
Comment:
Revision 45 as of 2015-04-06 10:18:14
Size: 8717
Editor: ?AlexanderBürger
Comment: updated link to DIANA
Deletions are marked like this. Additions are marked like this.
Line 49: Line 49:
 * [[http://met.no/diana | DIANA ]] Open-source meteorological workstation from met.no  * [[http://diana.met.no/ | DIANA ]] Open-source meteorological workstation from MET Norway; source code on [[https://github.com/metno/diana | GitHub]]

Meteorology in Debian Science

The Sentinel page gathers meteorology packages.

Meteorology tools in Debian / currently being packaged

Data Manipulation

  • cdo (upstream) Climate Data Operators

  • nco command-line operators to analyze netCDF files

  • arkimet Arkimet is a set of tools to archive, dispatch and distribute data files.

  • DB-All.e (upstream) Database for working with punctual meteorological data. It can also be used only as a BUFR encoder and decoder library.

  • OASIS (upstream) climate model Coupler

  • ?GeoNetwork (529453) (upstream) Data publication / search network.

  • cdat : Climate Data Analysis Tools

  • cmor : Climate Model Output Rewriter - rewrite output in a format suitable for the IPCC experiments.

  • Ferret : Data visualization and analysis from NOAA. Note: There is a different GNU Ferret in Debian already.

Libraries

  • grib_api (upstream) GRIB API from ECMWF

  • emoslib (upstream) Interpolation library from ECMWF

  • HDF-EOS : extension of HDF for Earth Observation Systems: The library supports the construction of new data structures: Grid, Point and Swath

  • g2clib : GRIB2 library from NCEP. Present in fedora, needed by NCL.

  • (upstream) library for handling physical units. used by NCL, etc.

  • libterralib used by Magics++, etc.

  • OpenDAP (upstream) : Network Data Access Protocol

  • gctp (upstream) General Cartographic Transformation Package. Used by Met packages.

  • terralib (upstream) GIS classes and functions library

  • Spherepack. Now released under BSD license.

  • libLAS (ITP) : ASPRS LiDAR data translation toolset

  • magics++ (upstream) meteorological plotting software from ECMWF

Analysis and Visualization

Software that could be included in Debian

  • ESMF NCAR Earth System Modelling Framework, a coupler for Climate model. Similar to OASIS, used mainly by American models.

* MCT Model Coupling Toolkit. Used by ESMF and now OASIS3-MCT

Issues:

Earth System Grid

Basically, package all the code used in ESG nodes. Its all FLOSS code, and needs proper management and tools. To start with, this will need CDAT, Thredds and their dependencies.

CMOR tables

Currently there is a package cmip5-cmor-tables. There are a bunch more tables; see upstream ?here. Replace the cmip5-cmor-tables package with a cmor-tables package? I've created an Alioth packaging project for this.

SZIP Compression

SZIP is a compression library for HDF and other scientific data. HDF which we ship, can be built against it, but SZIP is patented (US only?) and is free for non-commercial use (ie. not DFSG-free). It is unlikely to change and we could not ship it (can it be shipped in non-free ?). However it would be desirable for HDF on Debian to be able to use it if present: hence a patch could be written to dynamically open it if present. The patent is owned by NASA who have agreed not to enforce it; it expires in a year or so. A GNU GPL'd version of SZIP is being written so that SZIP can be included in WMO standards.

Python and NetCDF

See this discussion. There are multiple packages in Debian/ the python archives providing NetCDF interfaces; some pass netcdf-type objects around. This is sub-optimal as they don't interoperate. This should be fixed.

Python packages / APIs include:

* pyCDF (circa 2007): * Scientific.IO.NetCDF : from python-netcdf or pupynere * scipy.io.netcdf : from scipy, or python-netcdf4

python-netcdf is built against recent netcdf4, and so reads/writes netcdf4 files, but does not provide NC4 capabilities such as compression, etc. Similarly python-scientific, python-netcdf do not provide python3 versions. Is python-scientific obsolete? Transition towards scipy ?

Python3 Transition

As part of the general transition in Debian/Ubuntu to Python3, it is my release goal for Jessie to get all the met. packages working with Python3.

See DebianScienceMetPython3 to see progress.

pkg-config pc files for all libraries

As a release goal, provide pkg-config files for all libraries provided by ?DebMet, and hence dependencies. This is most useful when thinking of MPI. See DebianSciencePkgConfig for details.

Spherepack

spherepack is packaged, but NCL uses a variant with double-precision parameters, preceded by 'd' prefix. Implement this within spherepack, and apply as an external dependency.

Common GRIB tables

grib-api and ncl have Grib tables that could be factored out into a separate package.

GSHHS : Global Self-consistent Hierarchical High-resolution Shorelines

Several packages include their own copies of GSHHS maps and code. These should be merged, and one set of maps provided. DebianScienceGshhsMaps describes the work in progress.