This is the old NDG website. The new NDG website can be found
Data discovery and delivery are
inherent components of many aspects of science. They can be considered
part of a processing chain that starts with raw data from a variety of
sources, and ends with the graphical production of information that is
directly used in scientific research. The NERC DataGrid provides tools
and information products that can be used to facilitate this process.
I use the NDG now?
The NDG is still under active
development, but you can certainly use some NDG components now.
The most useful components now are:
- The NDG Production Discovery gateway (provides access to
thousands of datasets).
- The NDG development discovery gateway (this is a
technology preview, and may not be stable).
- The Climate Science Modelling
- The NDG Vocabulary
See the NDG trac site for the most up to
date information about the NDG.
is the NDG?
See the NDG flyer.
The NDG consists, or
will consist of, the following
(This page last updated, April 10,
Science Modelling Language information Model. (CSML is both a UML
description and an XML schema which is an “Application Schema” of the
“Geographic Markup Language”). CSML provides format independent
descriptions of the parameters and organization of datasets according
to the “Sampling Features” and “Observations and Measurements”
frameworks of the Open Geospatial Consortium (OGC).
The CSML Toolbox:
provides code which creates and manipulates documents which conform to
the CSML schema.
The CSML Data
Services: A set of web services which can expose CSML documents and the
data which they point to via the Open Geospatial Consortium protocols:
WFS, WMS and WCS, and which provide limited processing options on the
datasets exposed by those services (via the WPS). The CSML Data
Services are optionally secured by the NDG Security Infrastructure
CSML documents and
underlying data are held at the NDG partners: British Oceanographic
Data Centre, the British Atmospheric Data Centre, the Plymouth Marine
Laboratory, and the National Oceanographic Centre, Southampton.
The NDG Data
Graphical User Interface: provides the web interface which allows a
user to exercise the CSML web services to manipulate data.
The NDG Data
Graphical Interface and the CSML Data Services are deployed at each of
the NDG partners.
Objects for Linking Environmental Sciences, MOLES, schema (an XML
schema). MOLES provides a common and standardized way of describing key
aspects of datasets: the Activities which generate data, the
Observation Stations at which the data are collected (or produced in
the case of simulations), the Data Production Tools, and the Data
The MOLES XQuery
definitions and associated software: Because MOLES documents are
intended to provide a common way of accessing definitions, software is
needed to import and export information stored in other common formats.
The MOLES XQuery system supports import and export (in three languages:
XQuery, Java and Python) from the NASA Global Change Master Directory
Directory Interchange Format (DIF), Dublin Core, the Marine Data
Information Partnership, and ISO19139 metadata formats.
have been produced for datasets held at all the NDG partners (this like
all the other population tasks within the NDG has, and continues to be,
a very time consuming task).
The NDG Vocabulary
Service: MOLES, CSML, and other schema describing data need to make use
of “controlled vocabularies” (constrained and standardized ways of
describing components within the documents – it is these standardized
vocabularies which will support the “Semantic Web”, ie software which
can go beyond simply reporting document contents, but make “inferences”
based on their contents). The NDG Vocabulary service itself consists of
a web service deployed at the British Oceanographic Data Centre, the
work required to maintain the vocabularies exposed by the lists, and
some prototype inference engines which map terms from one list to
The MOLES browser:
software which exposes MOLES documents to users, and allows them to
navigate between the various components of the data descriptions and
make leaps from one dataset to another based on common components
(activities, data production tools, observation stations, parameters
Deployment of the
MOLES browser at the NDG partners, coupled to the NDG security
infrastructure, and underlying MOLES databases.
The NDG Discovery
Infrastructure: databases of documents designed to support data
location are produced at all the NDG partners, and stored at each site
in databases which support the Open Archives Initiative Protocol for
Metadata Harvesting (OAI-PMH). Clients (not limited to NDG clients)
which harvest these documents can expose the records in their own
websites and provide links to NDG data. These discovery records are
produced using the MOLES XQuery software and include links to both
browse services and
The NDG Discovery
Gateway: provides a user interface to discovery records harvested from
the NDG Discovery Infrastructure, and supports both SOAP and REST web
service interfaces to the underlying discovery database. (These
interfaces can and are used by other projects, e.g. the Marine Data
information Partnership allow their discovery records to be harvested
by the NDG, and have developed their own interface to those records
based on the SOAP interface).
The NDG Security
Infrastructure, Server Toolkit: provides the services and software
libraries needed to
authentication infrastructure (based on PKI technology and X509 proxy
certificates held in multiple MyProxy databases – the “Session
preexisting user database to the NDG authentication infrastructure,
to resources (the “gatekeepers”)
authorization information and map between authorization paradigms at
different NDG partner (the “attribute authorities”)
The NDG Security
Infrastructure, Client Toolkit: a software library to allow programmers
to modify existing, and develop new, Java and Python applications which
are secured by NDG technologies. The initial NDG applications using
this client toolkit are the MOLES browser and the NDG Data Graphical
User interfaces described above.
The NDG security
infrastructure is deployed at each of the NDG partners (including
bespoke software at each site to integrate the local access control to
the NDG infrastructure via the server toolkit)