Support Center News Blog : raster

To Mosaic or not to Mosaic?

coll5484 — Fri, 20 Nov 2009 14:09:00 GMT

Hi, my name is Jeff, and I am a member of the Geodata Raster team at the Eastern ESRI Support Services located in Charlotte, NC. I posed the question above because whether to mosaic or not is one of the most common workflow questions when dealing with rasters. There are many raster formats and depending on the format and the ultimate data product, the answer may change. Some immediate questions to ask are:

Am I looking to create a backdrop for a map or provide imagery capable of being analyzed?
Are the rasters highly compressed, for example .sid or .ecw?
Is file storage a concern for the mosaic? How much room do I have for the mosaic?

There are many more specific questions to consider, but these general questions can prevent future headaches when determining if creating a mosaic is the best course of action.

The mosaic process can create one file that contains several other rasters, but the creation of it can be time consuming and problematic. Plus, there are plenty of other situations where an unmanaged raster catalog may provide the functionality you desire without changing the format or creating a new file.

If you are looking for the ease of adding one file to a map document rather than multiple files or are looking to maintain the format of your compressed raster data, then an unmanaged raster catalog may be the answer. The unmanaged raster catalog resides in a geodatabase and will maintain the extents of all of the rasters added to it, but will not import the rasters. They will maintain the format and then can be added all at once to a map document by simply adding the raster catalog. The unmanaged raster catalog will maintain the location of the original file. This workflow can be particularly handy if you have a large number of SID files that have a file size of 1 gig but an uncompressed size of 20 gigs. A mosaic of several SID files would be considerably large, but the unmanaged raster catalog can utilize the files as they are and load them together at once.

The creation of an unmanaged raster catalog is much quicker than creating a mosaic and can be a great way to display a large number of rasters.

While unmanaged raster catalogs cannot be used for analysis in many of the geoprocessing tools, they can be used for digitizing and backdrops for many different maps.

Below are a few Web Help documents that pertain to this discussion:

Creating a Raster Catalog
Mosaicking Raster Datasets
Exploring Geodatabase Raster Catalogs

- Jeff S., Support Analyst, Geodata Raster team, ESRI Support Services

Working with Rasters: Georeferencing

coll5484 — Fri, 30 Oct 2009 13:00:00 GMT

Good day, folks! My name is Khalid, and I am a member of the Geodata Raster Unit at ESRI Support Services. This post is the first of a series that examines basic concepts revolving around raster data. Gaining familiarity with the terminology associated with a new subject matter is arguably the most challenging hurdle in mastering any new content: a notion that is not lost on GIS professionals inaugurating themselves into the world of imagery, scanned documents, and other manifestations of raster data. The hodgepodge of formats and properties intrinsic to rasters may cause confusion with understanding the processes needed for preparing the data so that it may be properly utilized in a GIS application. With this in mind, we will explore some of the more common topics associated with raster data by taking a look at georeferencing: what it is and when it is needed.

Before we get into georeferencing, it is important to develop an understanding of raster data, so please take the time to review the linked article. In short, if you have a scanned paper map, it is a raster. If you have an aerial photograph, it is a raster. If you took a picture from your digital camera, it is a raster. A raster can come in many formats; this list summarizes the formats supported by ArcGIS Desktop. The proper geographical placement of the raster, as with any other spatial data, is critically important for use in a GIS application. The idea of georeferencing a raster is done to ensure that the data essentially falls in the right place on the map. When it comes to rasters, satisfying this requirement involves evaluating a series of questions noted in this article on gathering raster data information:

Where did the raster come from?
Was there any special processing used to create the raster?
What has been done to it?

Understanding where the raster came from and what has been done to it plays a large part in determining whether the raster needs to be georeferenced. For example, a paper map that you might have scanned a few moments ago will certainly need to be georeferenced, whereas a DEM downloaded from the USGS Web site will not. If you are unsure about the details of the raster and whether it may or may not need georeferencing, examine its extent values and determine whether they appear to reflect real world coordinates in a projected or geographic coordinate system. If the data's extent appears to use real-world coordinates, review this article on identifying an unknown coordinate system. Hint: Projected coordinates typically have a minimum of 4 units for each extent, whereas geographic coordinates have 2 units for the top/bottom and up to 3 units for left/right. Also, be sure to check out the "Georeferencing a raster dataset" Web help article; it provides detailed information on this topic and has a neat video demonstrating the process.

- Khalid D., Support Analyst, Geodata Raster Unit, ESRI Support Services

Tips and tricks for sending data to ESRI Support Services

coll5484 — Fri, 19 Jun 2009 19:42:00 GMT

How to make subsets of Vector and Raster data in order to receive a faster resolution

Howdy Y’all! My name is Allison R., and I’m a Technical Lead for the Charlotte Support Center. I just wanted to say 'Hi' to everyone and post some helpful tips and tricks. When an ESRI Support analyst requests your data in order to test your process in-house, they are doing two things: checking to see if they can reproduce the reported behavior, and getting a process together that can be used to log a bug or enhancement request, if necessary.

Now some of you might be thinking, “Send my Data!! It is huge, that would take all day!” Well, I have a few tips and tricks I want to share with you that will help an ESRI Support analyst receive your data faster, review your steps more quickly, and get back to you in a flash.

Only send the amount of data necessary in order to reproduce the behavior you are reporting

What does that mean? Pare down your data to the smallest amount of data needed in order to reproduce the behavior you are reporting. Now, if you’re wondering how to do that let me give you some tips:

If the feature class has hundreds, thousands, or hundreds of thousands of features, try exporting a small subset of the dataset. This can be done in ArcMap by selecting a few features, right-clicking on the name of the feature class in the table of contents > Data > Export Data, and then exporting the data into a new database.
If the feature class has a lot of fields in the attribute table, then remove anything that is not necessary for testing. Fields in a feature class’s attribute table can cause the size of a feature class to increase dramatically. Here is another great reason to do this: if the tool or process that is being reported requires that a Support analyst pick specific fields, removing the unnecessary fields will ensure the correct fields are picked by the Support analyst. I have seen a lot of data with similar field names, and they can get really confusing! You can remove the extra fields quickly and easily by using the ‘Delete Field’ tool in ArcToolbox. Delete Field is located in Data Management Tools > Fields, and can be used to remove large quantities of fields at once.

Only send the data necessary in order to reproduce the behavior you are reporting

Sometimes our Support analysts get entire geodatabases or receive a lot of additional unnecessary data that does not relate to the behavior reported. This can be confusing for analysts (imagine looking at a geodatabase with dozens of tables, dozens of feature datasets, and hundreds of feature classes, when you only need to input two feature classes into a tool). Also, this can cause the size of the data to be HUGE! If ESRI Support analysts are provided with only the necessary data, it is smaller to transport and easier to understand. Here are some ideas to help make sure you send the smallest data possible:

Put only the data that you need to send to ESRI Support Services in a new geodatabase. Geodatabases can house a lot of feature classes, tables, feature datasets, complex datasets and more. When sent an entire geodatabase that contains additional elements not needed to reproduce the behavior being experienced, the Support analyst may have trouble finding the features, tables and other elements involved in the behavior. Remember: your analyst doesn’t know your data like you do, so the more you send the more review the analyst has to do before working on reproducing the reported behavior. If you need to send an analyst complex feature classes such as topology, you don’t have to rebuild this, just make a copy of the geodatabase and delete everything that is not needed.
By limiting the amount of data you send, paring down how many features are sent, and removing unnecessary fields, you are drastically reducing the size of the data you’re providing. One more thing you can do is use the Compact feature on the geodatabase to make sure it is as small as possible. When data is first added to a geodatabase, the records within each file are in order and are accessed efficiently by the file system. However, as records are deleted and added over time, the records within each file lose their order, and unused space develops as records are removed and new ones are added elsewhere in the file. This causes the file system to perform more record-seeking operations within each file, slowing the rate at which records are accessed. Compacting tidies up storage by reordering records and eliminating unused space. After compacting, the data in each file can be accessed more efficiently. Compacting also reduces the size of each file; it’s possible to be able to reduce the size of a geodatabase by one-half or more. To do this in ArcCatalog, right-click on the geodatabase > Click Compact Database.

ESRI E-mail servers can accept 6MB

Try to limit the amount of data sent to 6MB if possible, ESRI does have an FTP site where data can be uploaded, but uploading and downloading data can be time consuming. Some tricks to getting the data down to 6MB include:

Always ZIP your data. ESRI Support analysts can accept .zip, .RAR and .7z files and unzip them successfully. The most common type of ZIP file is .zip, and all Support analysts have this available so this would be the preferred method, but hey, we’re open to alternative methods.
If just zipping it is too large, try exporting your geodatabase into an XML (.xml) file. You can send your database schema and data in an .xml, which is smaller than your original database. The .xml is created in ArcCatalog by right-clicking on the geodatabase > Export > XML Workspace Document, and choose to Export the Data in the Binary format. Then place the resulting XML in a folder and zip it and send it to the Support analyst.
Similar to exporting a stand-alone geodatabase into an .xml document, SDE feature datasets can also be exported in an .xml. Instead of sending a whole data backup, please export just the feature dataset by right-clicking on the feature dataset through ArcCatalog > Export > XML Workspace Document, and choose to Export the Data in the Binary format.
Raster data can be Giant! It is not always necessary to send the entire raster dataset to ESRI Support Services for review; instead, send a small subset that can be emailed quickly and easily. To do this, zoom in to a pertinent location on the raster image. On the Drawing toolbar, select the Draw Rectangle tool and draw a rectangle over the area of the raster that needs to be clipped out. Right-click on the name of the Raster in the ArcMap table of contents > Data > Export Data, and under ‘Extent’ make sure that ‘Selected Graphics’ is selected. Set the other options up to mimic the original dataset.

Send detailed steps to reproduce the issue and an MXD

When working to package and send data to an ESRI Support analyst, it can be easy to forget to send the detailed steps to reproduce the behavior reported. Analysts understand the tools that are available in ArcToolBox and the functionality built into ArcMap and ArcCatalog, but they don’t necessarily understand your data or workflow. Please make sure to send detailed directions with screen shots, so they can be sure they are following your workflow exactly.

All of the steps above will help ESRI Support analysts help you in resolving your issue!

Thanks!

-Allison R., Technical Lead, ESRI Support Services, Charlotte Operations Department