Building a partial map cache at large scales

We’re occasionally asked how to build a partial map cache at large scale levels, while having a full cache at smaller scales. This is useful if you have highly detailed data in certain areas of your map. For example, you might have medium resolution data covering the world and high resolution data covering just your country. How do you get the high resolution data in your cache without having to cache the full extent of the world at large scales?

Conceptual illustration of a partial cache at large scales

Every map cache has a tiling scheme that defines the available scale levels. Think of your tiling scheme as an empty container of tiles. You don’t have to completely fill all scale levels in the tiling scheme; you can just partially fill the large scale levels. This post explains how to do that.

How does a tiling scheme get created?

The tiling scheme is represented by a file called conf.xml that is stored in your service’s cache folder in the server cache directory. (For example: c:arcgisserverarcgiscachemyCaliforniaServiceLayersconf.xml).

The way you get a tiling scheme is to run the Generate Map Server Cache tool. This tool creates a tiling scheme file from all the information you provided as tool input, such as the list of scale levels. After creating the tiling scheme, the tool also creates all tiles at all the scales you defined. If your goal is to create a partial cache, you need to stop the tool from creating all the tiles.

How can I create the tiling scheme without creating all the tiles

To get the tiling scheme without getting all the tiles, you can run Generate Map Server Cache and cancel it after about one minute. This won’t erase your tiling scheme; it will just stop the creation of tiles. If you’re nervous about whether one minute is long enough for conf.xml to get created, you can open Windows Explorer and verify that the completed conf.xml is in your cache folder before you cancel the tool.

So if I cancelled Generate Map Server Cache, how do I get tiles into my cache?

Even though you cancelled Generate Map Server Cache, you can still use Update Map Server Cache to add tiles to your cache. This tool isn’t just for updating existing caches; it is also very effective at creating new caches with a fine-grained level of control. You can tell the tool to cache only a subset of scale levels in your tiling scheme over a limited geographic extent.

You’ll need to run Update Map Server Cache several times in order to create your partial cache.

First, run the tool to Recreate All Tiles at small scales only, covering the full extent of the map document. This is the default extent that the tool uses.

The second time that you run the tool you can also choose Recreate All Tiles. But this time, select just the larger scales and give the tool a smaller rectangular extent. If you want to get real specific about which areas are cached at the large scales, you can script Update Map Server Cache to cover a series of rectangular extents. One example of this is Jeremy Bartley’s script that creates tiles based on the extents of features in a feature class.

Workflow summary

Here’s a summary of the steps for building a partial cache:

  1. Run Generate Map Server Cache and cancel it after the tiling scheme file has been created (waiting about one minute should be enough).
  2. Run Update Map Server Cache on the full extent of your map for the small scales only.
  3. Run Update Map Server Cache again on a limited extent of your map for the large scales only.
  4. Optional: Configure your server to return a Data Not Available tile for empty cache areas.

A note about the road ahead…

We’re happy to announce that this workflow has improved in ArcGIS Server 9.3. The Generate Map Server Cache tool will be deprecated in order to separate the workflow of creating the tiling scheme and creating cache tiles. (Your existing scripts will continue to work.) You’ll also be able to create a subset of tiles based on the boundary of a feature class. Finally, you’ll have the option to cache areas on demand as they are visited by users. We plan many blog posts on these topics in the months ahead.

-Sterling Quinn

Posted in Services | Tagged , , | Leave a comment

PIGWAD and Mars (or creating your own Mars Explorer)

NASA’s PIGWAD (Planetary Interactive GIS-on-the-Web Analyzable Database) site offers a wealth of planetary information, including lots of information for Mars. PIGWAD’s mission statement from their Web site:

(1) Produce a web-based, user-friendly interface aimed at the planetary research community that will support Geographic Information Systems (GIS) graphical, statistical, and spatial tools for analyses of planetary data, including the distribution of planetary GIS tutorials, tools, programs, and information; (2) Create planetary GIS databases consisting of peer-reviewed digital geologic maps, feature maps, topography, and remote-sensing data under the scientific oversight of the NASA Geologic Mapping Subcommittee (GEMS); and (3) Support and encourage the use of GIS in planetary research including geospatial open standards.

What this means is that there’s lots of information for you to use with ArcGIS Explorer, both downloadable data in the form of shapefiles and rasters, as well as Web-based content services, and for a variety of planets and their moons.

Here’s one of PIGWAD’s ArcIMS-driven sites that allows you to view a variety of different information for Mars.

Though the content above is viewed in an ArcIMS Web application, we can also copy and paste the URL that we see in the address bar at the top of the app and create a connection directly from ArcGIS Explorer to the underlying ArcIMS services. To do that we had to do a little trial and error to determine the correct URL (PIGWAD doesn’t publish the direct connect information), but it was easy to determine. In Open Content we chose ArcIMS as our connection type to create, and typed the connection information shown below:

Once connected we can view the entire list of available services, and a good one to choose is Mars_general_image.


Once chosen we’re presented with a series of dialogs that allow us to choose specific sublayers, how they are displayed, and other parameters. You can accept the default for most of those, but there are two that we’ll take a closer look at.

The first is the dialog for choosing the Service Imagery Format. If you know the service is imagery-based, or global in context, select that option. If you know it’s vector-based, choose the vector option. For vector services that provide only partial coverage this is the best option since it will make transparent anything outside the data. For a more thorough discussion of the options, see the Imagery Format Help topic.


And here’s where we choose the specific sublayer(s) that we want. There’s a lot of them, and the easiest way to choose just one is to turn them all off with the topmost checkbox, then scroll down and choose just the sublayer we want. Here we’ve chosen the color shaded relief layer.

We can repeat these steps as many times as we like to access the specific sublayers we want to use in our Explorer map. See the Selecting IMS Layers Help topic for more details.

Since we’re technically still working with good old Planet Earth as our foundation, we also may want to remove things that aren’t appropriate for Mars. So we can turn off the positional text (Tools > Options > Map Display), atmospheric halo and fog (File > Map Properties > Environment) and also remove tasks and layers that don’t apply.

Here’s our completed map showing the use of the swipe tool. You can see we’ve turned ArcGIS Explorer into a “Mars Explorer.”


Posted in Uncategorized | Tagged , | Leave a comment

Hooray for NASA's Phoenix!

Though only given 50-50 odds, NASA’s Mars Phoenix Lander completed a successful touchdown just a short while ago, completing a remarkable 296-day, 422-million-mile journey. In celebration of the milestone, the first-ever landing near Mars’ north pole, we decided to have a closer look at the red planet using ArcGIS Explorer.

How’d we do this? We made use of some of the freely available content published on the Geography Network. Here’s how.

First we started ArcGIS Explorer and clicked File > Open to show the Open Content dialog. We choose Servers, and then clicked the button at the top to choose ArcIMS (all services on the Geography Network are currently published via ArcIMS). We typed in the connection information shown here:

For Geography Network services we don’t need a user name or password, so we just left those fields blank. Once you establish a connection to a server it will be saved in your list of connections, and you won’t have to enter this information again. Once connected you’ll see it open in the connection list.

Here we’ve scrolled down the list of available Geography Network services (there’s a lot to choose from) until we reached NASA Mars, then double-clicked it.


We also accepted the defaults for the dialogs that appeared after our selection, except for the IMS Layer visibility. ArcIMS services can contain many sublayers, and you can choose which ones you want to add. In this case we wanted to add the two sublayers in the Mars service as individual layers so we could control the label visibility independently of the Mars imagery. So we repeated this twice, once to add the Mars imagery, and a second time to add the place labels.

When we were finished, we had two new layers in our map that allowed us to create the mars scene at the top of this post.

Posted in Uncategorized | Tagged , , , | Leave a comment

Aspect-slope map

By Aileen Buckley, Mapping Center Lead

Aspect-Slope - Thumbnail

An aspect-slope map simultaneously shows the aspect (direction) and degree (steepness) of slope for a terrain (or other continuous surface). Aspect categories are symbolized using hues (e.g., red, orange, yellow, etc.) and degree of slope classes are mapped with saturation (or brilliance of color) so that the steeper slopes are brighter. This will result in a map that has the colors shown to the right.

NOTE — This blog entry has been updated for ArcGIS 10.

Continue reading

Posted in Mapping, Migrate | Tagged , | 7 Comments

Road Ahead – New workflow for mosaicking raster datasets

The following post was written by Simon Woo, a product engineer on the geodatabase team specializing in Raster support in the geodatabase.

Previous versions of ArcGIS have provided various mosaicking workflows.  We’ve tried to make the mosaicking experience better in each version, but have had varying feedback on performance, especially when outputting to a file format. 

Prior to ArcGIS 9.3, the workflow to create a mosaic was to create a raster dataset and then use the Workspace to Raster Dataset tool to populate the new dataset.

In ArcGIS 9.3, we have provided a new workflow to make your mosaicking experience faster. 

The new workflow is to:

  • Create an unmanaged raster catalog with the Create Raster Catalog tool.
  • Load all the raster datasets into the unmanaged raster catalog with the Raster to Geodatabase tool.
  • Use the Raster Catalog to Raster Dataset tool to mosaic the datasets together. 

From the tests that we have run, this new workflow is considerably faster.  Below are a couple examples of comparison tests that have been run.   

Mosaic Test 1

Here’s the test data we used:
Type of data: 3-band
Input format: TIFF
Number of files: 20
Average file size: 38.8 MB
Output format: TIFF


Using the old workflow of creating a raster dataset and then using the Workspace to Raster Dataset tool, this data took 1 hour and 35 minutes to mosaic.

Here are the steps with the new workflow:

First create an unmanaged raster catalog.

This creates an unmanaged raster catalog. Now load the data into the new unmanaged raster catalog:

Now use the Raster Catalog to Raster Dataset tool to mosaic the rasters in the unmanaged raster catalog into a single raster dataset.



We conducted a similar test using GRID data and it yielded the following results:

Mosaic Test 2
Type of data: single band data
Input format: GRID
Number of files: 60
Average file size: 5.7 MB
Output format: GRID

Old Workflow – 45 minutes              
Create Raster Dataset time: 1 second
      Workspace to Raster Dataset time: 45 minutes

New workflow – 4 minutes
Create Unmanaged Raster Catalog: 1 second
Raster to Geodatabase tool: 1 minute
Raster Catalog to Raster Dataset tool: 3 minutes

The new workflow is shown in model form in the following graphic:

Posted in Geodata | Tagged , , | 2 Comments

Creating and Saving Bookmarks for Navigation

Bookmarks are handy for navigating to a particular location that you want to visit often. Here’s a tip on how to create and manage your own collection of bookmarks using results.

First, remember that each result has a view property that you can set that enables you to click the result to zoom to a location/view of your choice. Just navigate to where you want to zoom, right click the result and open its properties.  Choose Location > View.

Use the camera button to capture the view snapshot for the result, and save your changes. Now whenever you click on the result you will zoom to your specified view.

You can also set these properties for folders, and that’s how we’ll create a few bookmarks. Let’s create a collection of bookmarks that will take us to New York, London, and Redlands. Begin with an new folder by right clicking the result panel and choosing Add Folder.

The default name is “New Folder,” which we’ll want to change. Right click to open the context menu and choose Rename, or hover over the name and click to activate text entry, and change the name to Redlands.

Repeat these steps two more times, renaming the folders New York and London respectively. You should see something like this:

Now we’ll set the view for each of them, beginning with Redlands. Use the Find Place task or navigate to the view of your choice over Redlands. You can also tilt your view. When you’re happy with the view, open the properties for the Redlands folder.

Once the properties are opened, choose Location > View. You’ll see that we’ve not yet captured a view property, and the entry fields are blank.

You’ll also see the camera button in the upper right. To capture the current view, just click the button, then click OK.

Now whenever you click Redlands, you’ll zoom to the saved view. Repeat the steps for the New York and London bookmarks, and you’ll now have a collection of bookmarks that you can click to zoom to the desired location.

We can improve our bookmarks by creating another folder, renaming it, and dragging and dropping the bookmarks we created above into it. You should see something like this:

We can save these bookmarks with our map, or export the My Bookmarks folder to open again later, or share with others. Now you’re just a click away from any destination.

For more information on result properties, check the Results Window item properties Help topic.

Posted in Uncategorized | Tagged | Leave a comment

"Real time" 19th century data – and another sneak peek at 480

We’re blessed with the availability of “real time” content these days, and recently blogged about the upcoming support for GeoRSS feeds in Explorer 480 as part of the landscape of timely information.

Another recent post on the GIS Education Community blog discusses what “real time” may have been like when the earth was still terra incognita using the Pinkerton 1812 map from the Explorer Resource Center.

Below is a screenshot showing the Pinkerton 1812 layer swiped to reveal a sneak peek at the new graticule support we’ll be delivering with Explorer 480. This new user option allows your choice of graticule units and the ability to display reference grids such as the US National Grid and MGRS

Posted in Uncategorized | Tagged , | Leave a comment

New image service to be available for serving rasters in ArcGIS Server 9.3

A recent ArcUser
mentions the new image service that will be available in ArcGIS
Server 9.3:

“A new and fast service for use in browsers or desktop image processing programs,
the Image Service is new in ArcGIS 9.3. It is a lighter-weight service
optimized for delivering raw or finished imagery that can be RGB, panchromatic,
multispectral, or elevation data stored in a geodatabase or file system. Data
sources for Image Services are raster datasets, in formats such as TIFF, MrSID,
IMAGINE (IMG), or JPEG2000, or can be raster mosaics. With the Image Server
extension, Image Services can also use compiled image service definition
(ISCDef) data. Image Services can be managed either with the ArcGIS Server
Manager or ArcGIS Desktop ArcCatalog application.”

The article also mentions that optionally you’ll be able to expose image
services through the OGC Web
Coverage Service (WCS) specification

“With WCS, clients can obtain subsets of data; request that data be resampled,
reprojected, or returned in a specific raster data format; and output files in
GeoTIFF, National Imagery Transmission Format (NITF), Hierarchical Data Format
(HDF), JPEG, JPEG2000, and PNG formats.”

Posted in Services | Tagged , , | 4 Comments

Road Ahead – PostgreSQL database support at ArcGIS 9.3

The following post was written by Kasia Tuszynska, a product engineer on geodatabase team specializing in PostgreSQL and the geodatabase.

The release of ArcGIS 9.3 will see an addition to the list of databases supported by ArcSDE. PostgreSQL 8.3.0 is joining the ranks of Oracle, Sql Server Informix and DB2 and is the first open source database to be supported by ArcSDE. I previously worked with SQL Server and Oracle and did not really know what to expect from Postgres, but I found it a very clever and well thought out database, I loved the online community, and I learned more from reading forums than any book out there.

We had an interesting time implementing ArcSDE for Postgres because we wanted to support the PostGIS spatial type and our own spatial type implementation st_geometry. This was a conscious decision so our customers who use both our software and open source software would not have to choose between one and the other but could use both, and potentially even keep it all in one database. We also implemented our own spatial type, st_geometry, it was designed to work with the different geodatabase functionality, we implemented it to make sure that any customer who wants to try out what geodatabases have to offer or who would choose to transition from one database to another would not loose any geodatabase functionality.

ArcSDE for Postgres will be released on Windows, Red Hat Linux and SUSE Linux. Since Postgres is open source and released with a BSD license we could include it in our installation, so if a customer wanted to install Postgres and SDE in one swoop they could do that in windows by executing one installation wizard, or two shell scripts for an install on Red Hat Linux. With SUSE we were halted by the lack of precompiled binaries for Postgres so installation will have to be done the old fashioned way – from source, but the rest of the setup can be done with scripts.

Implementing ArcSDE on Postgres had us learn the ins and out of the MVCC concurrency model, altering the way we send updates to postgres. We spent quite a long time tweaking the sql related to editing, trying to shave seconds off of the performance time. A lot of effort has also been put into the documentation, for folks already familiar with ArcSDE administration moving to postgres will probably be a snap but there a couple of small things to remember for postgres, so when in doubt please check the help. For instance, to avoid having your feature classes looking like tables in ArcCatalog and to make them accessible to everyone the “usage” privilege has to be granted on a schema to the public role. Also, installing Postgres on Windows has to be done through an admin account but the account under which Postgres (postmaster) is to run can not belong to the admin user group … Postgres idiosyncrasies like this can drive one crazy especially if you are not used to it.

This is our initial release and I am hoping that all of our testing caught every annoying little bug, but in case we missed something please do not hesitate to drop us a bug in Nimbus.

Posted in Geodata | Tagged , , | 3 Comments

Dot maps for US General Soil Map (STATSGO) data

By Linda Barrett, Department of Geology and Environmental Science, University of Akron

Dot Maps for STATSGO data - Thumbnail

After seeing my poster that described using dot maps to show soils at the AAG conference in Boston a few weeks ago, Charlie Frye suggested that I write this entry to describe my maps and the technique used to create them.  This seemed especially appropriate given the recent entries in this blog about dot maps. Continue reading

Posted in Mapping | Tagged , | Leave a comment