Updating your map caches automatically: The key to caching dynamic data

For the fastest performance of your ArcGIS Server map services, we recommend that you use map caching wherever it’s appropriate. A map cache is a picture of your map at one point in time. Map caches are obviously appropriate for maps that don’t change frequently, such as street maps and imagery. But thanks to the update tools included with ArcGIS Server, you may still be able to use caching even if your data changes more often.

The Update Map Server Cache geoprocessing tool helps you update an existing map cache. You can write a script that uses this tool, then schedule the update to run on a regular basis. Even if you’ve never written a script or scheduled a task before, it’s worth the effort to learn how because of the performance improvements you can get from using a cached map. This post walks you through the process.

Deciding if you should cache your changing data

To understand whether your frequently-changing map can still be cached, it helps to ask these questions:

• How up-to-date does my map need to be?

  If the data you see on the map needs to be live, with no time delay acceptable, then caching is not appropriate. However, if a short delay is acceptable and the cache updates can be performed within that time window, you can still use caching.

• How big is my cache?

  A larger cache takes more time to update. But if you can isolate the areas of change, you may be able to shorten the update time.

  For example, Jeremy Bartley wrote a script that updates areas based on feature extents, instead of updating the entire cache. If you’re editing data in national parks, but you need a map of the entire country, you use this script to update your cache in the national parks only. This can save an enormous amount of time and server resources.

  When you use this script, remember that if data and symbology are frequently changing throughout your map, the update will take longer than if the changes were confined to a few specific areas.

To summarize, if the update can keep up with the changes in an acceptable amount of time, the map is appropriate for caching.

Scenarios

The following scenarios might give you an idea of what types of maps are appropriate or inappropriate for caching.

• National and regional weather maps

  Probably appropriate for caching. A map viewed at a national or regional scale does not require many tiles and can be updated before the weather drastically changes.

• Cadastral maps of a city that change on a daily basis

  Probably appropriate for caching because a city is not a large area and the cache can run overnight. However if no delay is acceptable, the map might not be appropriate for caching.

• Maps that update vehicle or ship positions every 10 minutes

  May be appropriate if the cache can be updated in less than 10 minutes. This depends on the size of the map, the cache scales, and available server resources.

• Maps for 911 dispatchers that display incidents and responders in real time

  Probably not appropriate for caching because no time delay is acceptable.

Preparing to write the script

When you initially create the map cache, note how long it takes. This will give you an idea of how long it would take to update the entire cache.

After the cache generation tool runs, you may want to copy the text from the geoprocessing message box and paste it into Notepad. This isn’t required; it’s just for convenience when writing your update script. You can copy and paste some of the parameters from the message box into your script.

Writing a script to update your cache

To automate cache updates, you can create a Python script that runs the Update Map Server Cache tool. You can write the Python inside of Notepad, or you can use a development environment such as IDLE or PythonWin.

The script does not have to be complex. It just needs to define the parameters for the tool and run it. The following Python script runs the Update Map Server Cache tool to re-create an entire cache:

# This script updates all tiles in an ArcGIS Server 9.2 map cache

# Access the geoprocessing tools
import arcgisscripting
gp = arcgisscripting.create()

# Set up all of the variables for the update tool
server_name = "myserver1"
object_name = "Precipitation"
data_frame = "Layers"
layers = ""
constraining_extent = ""
scales = "32000000;16000000;8000000;4000000;2000000"
update_mode = "Recreate All Tiles"
thread_count = "2"
antialiasing = "NONE"

# Run the Update Map Server Cache tool
try:
    print 'Starting Cache Update'
    gp.UpdateMapServerCache(server_name, object_name, data_frame, layers, constraining_extent, scales, update_mode, thread_count, antialiasing)
    print 'Finished Cache Update'

# Get the error messages if the tool fails
except:
    gp.AddMessage(gp.GetMessages(2))
    print gp.GetMessages(2)

If you’re just getting started with scripting you can just copy the above into Notepad, substitute your own variables, and save it with a .py extension. If you need help understanding the variables, see the Geoprocessing tool reference for Update Map Server Cache.

Scheduling the update to run automatically

Once you’ve saved your Python script, you can run it on any machine that has ArcGIS Server and Python installed by double-clicking the .py file in Windows Explorer. (Python is included with ArcGIS Desktop so you may already have it installed. Otherwise visit www.python.org to get Python.) Running the script manually in this way can be useful for testing, but in most cases you'll want to schedule the script to run on a regular basis.

Your operating system contains utilities that help you schedule tasks such as running a script. You'll need to provide the location of the script file, how often you want it to run, and the name and password that the task will run as.

The Scheduled Task Wizard in Windows walks you through the process of setting up the task. Open the wizard by clicking Start > All Programs > Accessories > System Tools > Scheduled Tasks > Add Scheduled Task.

If you want a few more options than the wizard provides, you can schedule the task from the command prompt. Just open Start > All Programs > Accessories > Command Prompt and enter your command: like the following, which schedules a Python script to run every 30 minutes:

schtasks /create /sc minute /mo 30 /tn "Update My Cache" /tr C:\92CachingScripts\Update.py

If you need help creating a command like this, see the Microsoft documentation for the schtasks command.

Dedicating server resources to the update

At this point you may wonder if your server can handle the cache updates and fill user requests at the same time. Updating a cache definitely uses server resources because the server is required to continually draw map tiles during the update. However, the update doesn't have to tie up your entire server.

Each service has a maximum number of instances of the service that are allowed to run, which you can adjust in the Service Properties. You can also use the caching tools to specify how many of those instances will be dedicated to caching (this is the “thread_count” variable in the script above). If you're short on server resources, you can lower the number of instances dedicated to caching so that others can continue to use your service while the update occurs.

In summary, if you want others to continue using your service while you are updating, make sure that the number of instances dedicated to caching is less than the maximum allowable number of instances.

It’s also important to note that running Update Map Server Cache forces two restarts of the service – once before the update starts and once after the update finishes. Your services may be momentarily unavailable during the restarts.

Scheduling your update to run on nights or weekends can also lessen the chance of the update interfering with regular server traffic. Many organizations have nightly backup or update processes, into which you could incorporate your cache update script.

-Sterling Quinn