Welcome to ESRI Blogs

Should I load my raster data into a geodatabase?

I noticed there were a lot of raster related questions at the user conference this year. People mainly wanted to know the advantages of loading raster data into a geodatabase, and generally how to manage their raster data. So I talked to Simon Woo on the raster team about it and he put together this list of pros and cons for loading raster data into a geodatabase (ArcSDE or File) as opposed to keeping the raster data in a file format outside of the geodatabase.

Note: This post only applies to the ArcSDE and file geodatabase. A personal geodatabase does not offer these same advantages.

Pros:
Fast data access
The geodatabase stores your data in blocks, which allows ArcGIS to access only the blocks of data that you need to use. Once loaded, this format allows ArcGIS to access your data very quickly.

Multi-user access
The geodatabase allows multi-user access to data whereas most file formats are not meant for multi-user operations.

Compression
Raster data can be compressed as it is being loaded into a geodatabase. Various types of compression schemes can be used including LZW, JPEG, and JPEG 2000. Note: If your file format is already compressed with a high compression factor, then the geodatabase compression may not be as highly compressed.

Better handling of NoData values
The geodatabase stores your Nodata pixels as a bit-mask rather than a value in the raster dataset. This means that you don’t need to use a pixel value as a place-holder for your NoData values like you do in most file formats.

Security (ArcSDE only)
Since your DBMS will require a user name and password, you can store your data in an ArcSDE database and know that only authorized users can access your data.

Cons:
Lack of third-party software support

Not all third party programs will allow an ArcGIS geodatabase to be used in conjunction with their software. This means that you are limited to the software packages that can access your data. This should be one of the considerations before you store your raster data in a geodatabase.

Lower compression than some file formats
The geodatabase allows your data to be stored with compression; however there is compression software available that can more highly compress your data – such as wavelet compression technology. For these datasets the file format may take up less storage space than the compression used by the geodatabase.

Loading Time
Since the data needs to be converted to a geodatabase block format, it takes time to load your data into the geodatabase. The larger your data, the more time it will take to load. Thankfully you only need to load your data into the geodatabase once.

For more information check out the help topic Why store rasters in a geodatabase, which goes into similar details as well as gives a comparison of storing raster datasets and raster catalogs.

I'm adding a few more useful raster help topics to this post based on the comments and feedback we're getting. The first two are case studies on building large raster datasets and storing raster datasets. Another good topic is loading large raster datasets into ArcSDE.

 

Published Thursday, August 14, 2008 10:09 AM by JonMurphy

Comments

# Geoprocessing

I heard at the user conference that geoprocessing operations can be tediously slow on geodatabase rasters.  It would be helpful if you included details on geoprocessing in this comparison.

Thursday, August 14, 2008 9:19 PM by majgis

# re: Should I load my raster data into a geodatabase?

my company has 1TB/year raster data. It's very hard to decide whether to store it in a ArcSDE database or file format. I want to know is there any problem to manager these huge amount of data in a sde database, if the raster data size exceed 10TB.

Saturday, August 16, 2008 9:54 AM by iKeepSmile

# re: majgis's question

"I heard at the user conference that geoprocessing operations can be tediously slow on geodatabase rasters.  It would be helpful if you included details on geoprocessing in this comparison."

----------------------------------------------

In general, geoprocessing in a geodatabase should be relatively similar processing times as in a file based system.  There are a few exceptions to this.

   1) Mosaicking tends to be much faster in a geodatabase (as seen in a previous blog post).

   2) Applying compression will increase the time it take to write a compressed raster to a geodatabase.

   3) Any Spatial Analyst or 3D Analyst tool will take longer, since it is outputing everything to GRID and then converting to a geodatabase format.

Simon W.

ESRI - Team Raster

Tuesday, August 19, 2008 10:19 AM by simo3919

# re: iKeepSmile's question

I want to know is there any problem to manage these huge amounts of data in an sde database, if the raster data size exceed 10TB.

--------------------------------------------

This is a very common question, and it is very complicated to answer since it is on a case to case interpretation on whether your project is suited for the geodatabase.  We will come up with a more comprehesive decison tree and post it in an upcoming blog.

In terms of the amount of raster data, in house we have compiled ArcSDE databases with at least one terabyte of raster data. Some of our business partners have compiled ArcSDE databases of over 17,000 images totaling over 22 terabytes. So it is definitely possible to manage these large amounts of data.

Simon Woo

ESRI - Team Raster

Tuesday, August 19, 2008 10:23 AM by simo3919

# Compare Image Server

We (and some of our users) will be very happy if you can compare to Image Server and not to just images on disk.

Thanks

Mody Buchbinder

Systematics, Israel

Sunday, August 24, 2008 4:34 AM by mody

# re: Should I load my raster data into a geodatabase?

Mody,

We've been getting feedback on the blog and elsewhere about image server. I'll be getting a post together and putting it up on the blog shortly which should help with your image server questions.

Jonathan Murphy

Tuesday, August 26, 2008 2:15 PM by JonMurphy
Anonymous comments are disabled