ArcGIS Pro

Accessing Multidimensional Scientific Data using Python

 

With the 10.3 release, a new Python library, netCDF4, began shipping as part of the ArcGIS platform.  netCDF4 allows you to easily inspect, read, aggregate and write netCDF files.  NetCDF (Network Common Data Form) is one of the most important formats for storing and sharing scientific data.

The ArcGIS platform has had geoprocessing tools which read and write netCDF data since the 9.x release.  However, there may be times when you may want to access or create netCDF data directly using Python.  There are four ways of interacting with netCDF files in ArcGIS; geoprocessing tools, the NetCDFFileProperties ArcPy class, the new netCDF Python module, and the multidimensional mosaic dataset.  Which method you use depends on what you are trying to accomplish.  A summary of different ways of interacting with netCDF files appears in the table below.  This blog post will focus on the new netCDF4 Python library.

The netCDF4 library makes it easy for Python developers to read and write netCDF files.  For example, this code snippet opens a netCDF file, determines its type, and prints the first data value:


>>>import netCDF4           # the module name is case sensitive
>>>d = netCDF4.Dataset(r'c:datatemperature.nc', 'r')
>>> d.data_model
>>>print(d.variables['tmin'][0][0][0])   # tmin[year][lat][lon]

'NETCDF3_CLASSIC'
-23.2861

NetCDF4 stores the data read from a netCDF file in a numPy array.  This means you have access to the powerful slicing syntax of numPy arrays.  Slicing allows you to extract part of the data by specifying indices.  The variable tmin in the example above has three dimensions; year, latitude and longitude.  You can specify an index (or a range of indices) to slice the three dimensional data cube into a smaller cube.  This code snippet extracts the first five years of data for the variable tmin and prints summary statistics:


>>>A = d.variables['tmin'][0:5][:][:]
>>>print("Minimum = %.2f, Maximum = %.2f, Mean = %.2f" % (A.min(), A.max(), A.mean()))

Minimum = -60.77, Maximum = 27.69, Mean = 0.41

Here are some potential uses of the netCDF module:

See http://unidata.github.io/netcdf4-python/ for documentation of the netCDF4 module.

** Note: Previously this blog post erroneously reported that the netCDF4 python library began shipping as part of the ArcGIS platform at version 10.2.  It did not ship with the platform until version 10.3. **

Methods of interacting with netCDF files:

Method

Use
this method to …

Benefits
and limitations

Example

Geoprocessing tools (all of the tools located in the Multidimension toolbox)

~ create a map, table  or chart from netCDF data
~ chain netCDF data to other tools in a GIS workflow
~ work with a single slice of the data
~ supports reading and writing netCDF files
easy access through the familiar geoprocessing tool UI
works with one slice of the data at a time

Explore spatial patterns in
precipitation patterns

netCDF4 Python module

~  access netCDF data directly using Python

~ have more control  over the structure and contents of a netCDF file

~  advanced slicing of data

~  easy access to a number of numPy functions

Build a custom geoprocessing tool to combine
several netCDF files into a single file

netCDFFileProperties ArcPy Class

~ explore the structure of a netCDF file

~ easy way to access the properties of
variables and dimensions

no access to the data

Create an inventory of a large
collection of netCDF files

Multidimensional mosaic dataset

~ temporally and/or spatially aggregate a large collection of
netCDF files

~ perform ‘on-the-fly’ analysis

~ serve data as a service

~ only works with regularly gridded data

~ can manage very large collections of files

Aggregate model output from different
regions into one seamless dataset

About the author

Kevin Butler is a Product Engineer on Esri’s Analysis and Geoprocessing Team working as a liaison to the science community. He holds a Ph.D. in Geography from Kent State University. Over the past decade he has worked on strategic projects, partnering with customers and other members of the science community to assist in the development of large ecological information products such as the ecological land units, ecological marine units and ecological coastal units. His research interests include a thematic focus on spatial statistical analytical workflows, a methodological focus on spatial clustering techniques and a geographic focus on Puerto Rico and midwestern cities.

Connect:
0 Comments
Inline Feedbacks
View all comments

Next Article

What's new in ArcGIS Hub first quarter

Read this article