Publishing a geoprocessing Select tool to ArcGIS Explorer

In this post, guest author Jeff Barrette of ESRI Olympia explains how to create a
geoprocessing service for selecting features in ArcGIS Explorer.

ArcGIS Explorer users may want to zoom to specific locations based upon data
that is available to them either locally or from a server. In ArcGIS Desktop,
users can simply select features of interest based upon feature attribute
values and then zoom to the extent of the resulting selected features. ArcGIS
Explorer does not have the equivalent of a Select By Attribute function
available but it can easily be added as a custom task. This post discusses the
steps necessary to build a geoprocessing select task that can be added into
ArcGIS Explorer to give users zoom to selected features functionality.

The general steps include first using ArcGIS Desktop to build a map document
(MXD) that contains the feature layers that need to be queried along with the
resulting selection symbology. The next step involves building a geoprocessing
model or script that performs the selection capabilities. Next, the
geoprocessing tool and the results map document need to be published and served
using ArcGIS Server. Finally, the new capabilities are added into ArcGIS
Explorer as a new, custom task.

For this example, the data referenced in the steps below is the States feature
class that is available from the ESRI Data and Maps data collection. The data
is located on the ArcGIS 9 Data & Maps StreetMap USA DVD under the
usacensus folder. This post is written for use with ArcGIS Desktop 9.2 (SP5),
ArcGIS Server 9.2 (SP5), and ArcGIS Explorer (build 440).

Setup

  1. Create a new folder on the file system called “SelectTask”.
    It is best to organize all of the components necessary to publish a
    geoprocessing task (file based data, SDE connection files, MXDs, toolboxes,
    etc) in a single location to minimize file access and permission issues. The
    graphic below also includes a folder called Publish. It is used to organize all
    published tasks as individual project folders.
    Publish folder
  2. Copy the states feature class to this new location.
  3. Set the appropriate permissions.
    Ideally, this folder is located on the same machine that ArcGIS Server is
    installed on. If the data is on the server, then permissions on the data should
    be all set. If on a different network drive, this folder should be shared and
    the SOC account should be granted access to it.
  4. Create a connection to the ArcGIS Server as an admin user.
    If this is the first time you are publishing something to ArcGIS Server, you
    are first required to establish a connection to the server.
  5. In ArcCatalog, expand GIS Servers.
    This is a node in the Catalog Tree.
  6. Double-click Add ArcGIS Server.
  7. Select Manage GIS Services and click Next.
  8. Provide the necessary connection information and click Finish.

Note: It is also recommended that you read through the Troubleshooting
Tips
section at the end of this post before proceeding with the steps
below to ensure user accounts and permissions are set appropriately.

Designing a Map Document (part 1)

  1. Open a new ArcMap session and add the States layer. It is not necessary
    to modify the symbology because with the steps in this document, we will not be
    consuming and displaying the map service, but rather using the layers in it to
    query against its features. In the steps to follow, you will symbolize the
    results layer to control how the results appear in ArcGIS Explorer (or in other
    applications).

    Note: If your data source is an ArcSDE geodatabase, then see the Working
    with ArcSDE Geodatabase Data Sources
    section below for some suggestions
    on how to manage the ArcSDE server connections. There are some additional steps
    that should be taken to ensure ArcGIS Server can connect to the data. Copying
    the SDE connection file to the project folder makes access much easier to
    manage rather than having to set permissions on your personal profile folders.

  2. Set relative paths for the data sources.
    This can be set via File > Document Properties > Data Source Options.
    Relative paths are not necessary if the ArcMap and ArcGIS Server applications
    are on the same machine but they do make it much easier to move a project
    folder from one server to another or to multiple servers all with the same data
    and models. Hard-coded paths are best if there are multiple SOC machines but
    all pointing to a single drive to access data.
  3. Check the “Overwrite the outputs of geoprocessing operations” option.
    This can be set via Tools > Options > Geoprocessing tab. It is important
    that the output file name remain the same after multiple executions. If this
    option is left unchecked the output name would change (i.e., increment by one
    each time the model is run) and the applications would not be able to reference
    the correct output result.
  4. Uncheck the “Results are temporary by default” option.
    This can be set via Tools > Options > Geoprocessing tab. If the results
    are going to be used in subsequent applications, it is important the results
    are not immediately cleared.
  5. Save the MXD to the new folder location and name it “SelectStates.MXD”.
  6. Keep ArcMap open. You will come back to the same MXD in a few steps.

Building a geoprocessing model

  1. In ArcCatalog, create a toolbox called “CustomToolbox”.
    Right-click the SelectTask folder and click New > Toolbox. Your folder
    should look similar to the following:
    Folder with toolbox
  2. In ArcMap, open ArcToolbox.
  3. Add your CustomToolbox.
    Right-click ArcToolbox > Add Toolbox, browse to the Select Task folder, and
    click Open.
  4. Set the ScratchWorkspace environment variable to the location of your project
    folder (Example: C:PublishSelectTask).

    Right-click ArcToolbox > Environments > General Settings > Scratch
    Workspace. It is important to do this at the application level and not just at
    the tool or model level because it is the ArcMap application that needs to
    manage the output data.
    Setting the Scratch Workspace
  5. Create a new model.
    Right-click your CustomToolbox and click New > Model.
  6. Change the Name property to “SelectStates”.
    Select Model > Model Properties > General Tab. The name property cannot
    include spaces or underscores.
  7. Also change the Label property to “SelectStates” and click OK.
    The label property is what gets displayed in the client application and can
    contain spaces.
  8. Drag and drop the States layer onto the model.
  9. Drag and drop the Select tool onto the model. You can find this tool in
    ArcToolbox Analysis Tools > Extract > Select.

    Note: The Select By Attribute tool can’t be used by itself because it
    does not create an output result that gets written to disk that can then be
    accessed by other applications. In a model, Select By Attribute can be
    connected to the Copy Features command to generate the necessary file output.
    The Select command accomplishes the same end result.

  10. Open the Select tool.
    Right-click on the tool and select Open.
  11. Set States to be the Input Features.
    Select the layer with the blue recycle symbol next to it. This represents the
    data source added into the model. The yellow icon represents a layer located in
    the Table of Contents (TOC). If you choose the latter, you will have two
    references to the same data and one will need to be removed.
  12. Set the %SCRATCHWORKSPACE% inline variable for the Output Feature Class. The
    input should look similar to the graphic below:
    Set the SCRATCHWORKSPACE inline variable

    Note: It is best to use the ScratchWorkspace inline variable when output
    results are going to be referenced from other applications. The Managed option
    works great for intermediate results but does not work well on permanent output
    results. The Managed option will automatically increment the output name with
    each successive execution therefore making it difficult for other applications
    to reference the correct data. The ScratchWorkspace inline variable should be
    used instead. In addition to ScratchWorkspace, also use the “Overwrite the
    outputs of geoprocessing operations” option and make sure the “Results are
    temporarily by default” option is turned off. These settings will ensure the
    resulting output is correctly accessed by other applications.

  13. Build a default, optional expression (Example: “STATE_ABBR” = ‘AK’). This
    can be left blank but it makes it a much easier for a user to modify an already
    existing expression (i.e., its SQL structure) than it is to type one from
    scratch. Click OK.
  14. Create Expression variable/parameter for Select tool.
    Right-click the Select tool and choose Make Variable > From Parameter >
    Expression.
  15. Click the Auto Layout and Full Extent buttons.
    This will better organize your view of the model.
  16. Set the Expression as a Model Parameter.
    Right-click the Expression object and choose Model Parameter. This will enable
    a user to enter a custom expression within the published task inside the client
    application.
  17. Right-click the Expression and rename it “Enter Expression:”. How
    the expression is labeled will control how it appears in the client
    application. This is also true for how you label the model. The following
    graphic shows how the task will appear in ArcGIS Explorer.
    The task in ArcGIS Explorer
  18. Right-click the model output States_Select.shp and check Model Parameter.
    This will persist an output result on disk. The result is what will be sent
    back to the ArcGIS Explorer application. Note, when output is set as Model
    Parameter it cannot also be set to intermediate (temporary data). Therefore any
    results you want to send back to ArcGIS Explorer should not be intermediate.
  19. Check Add to Display. This parameter is only necessary for displaying the
    result in ArcGIS Desktop. This is not necessary when publishing the model as a
    task.
    Model options checked

    When completed, the model should look similar to the following graphic:
    The completed model

  20. Save the model.
    Click Model > Save.
  21. Close the model. Click Model > Close.

Designing a Map Document (part 2)

  1. Drag and drop the Select States tool from ArcToolbox into the top portion of the
    ArcMap TOC.

    This is called a Tool Layer and it can be executed from the TOC. The Tool Layer
    will also include inputs and outputs to the model once it is executed.
  2. Run the model from the tool layer.
    Right-click the Select States tool layer and select Open to open the model.
    Then click OK to run the model. A layer named States_Select.shp displaying
    the default selected state of Alaska should have been added to your TOC as part
    of the tool layer.
  3. Set the symbology for the States_Select.shp layer.
    However this layer is symbolized is how it will appear in a client application.
    If you only want the result to appear in the ArcGIS Explorer results list but
    you don’t want the feature(s) to display in the view, then set the symbology to
    be nothing.
  4. Uncheck the top level tool layer. The tool layer is really intended only
    to be used for publishing purposes. If it is turned on while running the model
    from ArcMap, it won’t be consistent with the layer returned from the result via
    Add to Display. The TOC should look like the following graphic:
    Map document TOC

Note: Models can be published in one of two ways:

First, publish an MXD containing geoprocessing tool layers. Each tool layer
becomes a task in the geoprocessing service. This method is a little easier
because all the symbology can be directly set within the MXD. Also,
geoprocessing capabilities are automatically recognized when the MXD is
published to ArcGIS Server. Both the data and tools can be consumed by client
applications. This will be the method used in the following sections.

Second, a geoprocessing toolbox can be directly published. Each tool in the
toolbox becomes a task in the geoprocessing service. If you want to control the
appearance of the input and output results, then you would need to create
additional layer files with the desired symbology that are referenced by the
tools. Publishing stand alone tools does have an advantage over publishing tool
layers.

Publishing the map with geoprocessing capabilities

  1. In ArcCatalog, browse to the SelectTask folder.
  2. Right-click the SelectStates MXD and choose Publish to ArcGIS Server. The
    first panel allows you to select an ArcGIS Server, specify the name of the
    service, and select or create a folder to organize your new service. You can
    optionally change these settings. Click Next.
  3. Verify that the Mapping and Geoprocessing capabilities are checked and click
    Next.

    Optionally, this service could be made available to KML, WMS, or mobile
    clients.
  4. Click Finish.
    To dismiss the summary page. A new map service and a new geoprocessing service
    are added to the GIS Server. Both services should be started and ready to go.
    The GIS Server connection in ArcCatalog should look similar to the graphic
    below:
    GIS server connection in ArcCatalog
  5. Verify the service is running. A quick way to determine the service is
    working before continuing is to make sure you can preview the service in
    ArcCatalog. Simply select the SelectStates map service and click on the preview
    tab. If you can see the States displayed, then the service is running properly.
    If you don’t see anything display, then refer to the Troubleshooting Tips
    section near the end of this document.

Adding the custom task into ArcGIS Explorer

  1. Open ArcGIS Explorer.
  2. Click Select Tools > Manage Tasks
    This dialog allows you to control access to the tasks that can be used within
    ArcGIS Explorer. The tasks listed on the left are all those currently available
    to the application, the tasks on the right are those that are actually being
    used.
  3. Click the Get Tasks button.
    This allows you to browse locally for tasks or to connect to other servers to
    add additional tasks.
  4. Make sure the Servers option is selected and click ArcGIS Server.
  5. Enter the appropriate connection information.
    If you are not sure of the server name or path, then look at the properties of
    the SelectStates map service via a right-click > Service Properties >
    Capabilities tab. Copy the URL information up to and including “/Services”.
    Connecting to the server in ArcGIS Explorer
  6. Expand the resulting SelectStates toolbox, click the Select States tool,
    and click Open.

    The task has been made available to the application but is still not available
    to the current map.
    Adding the task in ArcGIS Explorer
  7. Click the Select States task on the left side and move it right to make it
    available to the current map. Then click OK.

Using the task in ArcGIS Explorer

  1. Click the Select States task in the Tasks list.
    This will open the custom task in the task window. The labels and text display
    exactly how you set them up in the model.
  2. Modify the expression found in the Task Center. Because a default expression was
    created, it is easy to manipulate.

    Keep in mind that this expression could be modified however someone would need
    it to be (Example: “STATE_ABBR” = ‘WA’ OR “STATE_ABBR” = ‘RI’).
    The completed task running in ArcGIS Explorer
  3. Click Run. Then double-click the result in the Results list to zoom to the
    feature extent (ArcGIS Server Service Pack 5 is required to zoom to the results).

Troubleshooting Tips

  • ArcGIS Server must be properly installed using an Administrators account and
    both the GIS Server and Web Applications post installs must be completed.
  • The ArcGIS SOM and SOC accounts (the defaults are ArcGISSOM and ArcGISSOC) must
    be a member of the Users group.
  • The ArcGIS Web Services account (the default is ArcGISWebServices) must be a
    member of the agsadmin group.
  • Be sure that your account (the person publishing the services) is a member of
    the agsadmin group.
  • It may be necessary to reboot your machine in order for permissions to take
    effect.
  • The ArcGISSOC account must have at least read-only access to the file-based
    data, the MXD files, toolboxes, and SDE connection files. If the local Users
    group has access, then the ArcGISSOC account also does because it is a part of
    the Users group. You would need to explicitly add ArcGISSOC if you wanted it to
    have a different set of permissions than Users (e.g., read vs read/write).
  • If these files (above) are on a separate network drive, then the folder must be
    shared and the ArcGISSOC account must have at least read access to this shared
    drive.
  • The ArcGISSOC account must have read/write access to the ArcGISServer
    directories. This is where the cache, jobs and output are written too.
  • Confirm that IIS is properly installed and running.
  • Confirm that the IIS virtual directories (argisoutput, arcgisjobs, and
    arcgiscache) exist and their permissions and local path are correct.
  • Confirm that a folder that matches your ArcGIS Server instance name exists.
    This folder is created during the post install of ArcGIS Server and is found by
    default in C:Inetpubwwwroot.
  • If you’ve made changes to the MXD (e.g., selection symbology), then you will
    need to stop and restart both the map service and the geoprocessing service.
  • If you’ve made changes to the model, then remove the tool layer, rerun the
    model and add the tool layer again. Be sure to follow the steps again as
    indicated in the steps above.

Working with ArcSDE Geodatabase Data Sources

When a connection is made to an ArcSDE geodatabase data source, an SDE
connection file (*.SDE) is typically created in your Documents and Settings
folder under Application DataESRI. This folder would need to be shared and
the ArcGIS Server SOC account would need to be given permissions to the folder.
An easier way to manage this is to create a copy of the *.SDE file and copy it
to the SelectTask folder as outlined below:

  1. Expand Database Connections. This is a node in the ArcCatalog Tree.
  2. Copy the SDE connection file. Right-click the SDE connection file
    and select Copy.
  3. Paste the SDE connection file into the SelectTask folder.
    Browse to and right-click the SelectTask folder and select Paste.
  4. Update all ArcMap ArcSDE geodatabase data sources to use the new connection
    file.
    This can be done via the layer properties Source tab. Right-click
    the layer, select properties. Select the Source tab, click the Set Data Source
    button and browse to the new connection file. Do this for each ArcSDE
    connection.
Posted in Services | Tagged , , , | 3 Comments

Five Best Practices for Maintaining an ArcSDE Geodatabase

This blog entry has been taken from a podcast previously recorded by Derek Law from the geodatabase team. We really liked the podcast and thought there was some valuable info to be shared.  To hear the podcast and check out other useful podcast topics visit: http://www.esri.com/news/podcasts/instructional_series.html

The performance of an enterprise level ArcSDE geodatabase is influenced by many factors, such as hardware configuration, network configuration, network traffic, and the number of concurrent users.

The tips in this blog entry are not database platform-specific, but they are general tips that will hopefully enable you to improve the performance of your enterprise geodatabase.

So, five best practices for maintaining an ArcSDE geodatabase are:

  • Increase the frequency of updating statistics on tables
  • Rebuild indexes on tables
  • Plan parent-child version relationships carefully
  • Compress the geodatabase often
  • Monitor system resources

1. Increase the frequency of updating statistics on tables

Statistics in the database describe the column data stored in tables. They help the database Query Optimizer to estimate the selectivity of SQL expressions, and enable it to accurately assess the cost of different query plans. The optimizer then chooses the most efficient execution plan for retrieving and/or updating data in the database. Having poor statistics is a frequent cause of poor performance. Keeping accurate up-to-date statistics will help improve database performance, because this will enable the Query Optimizer to make more accurate assessments of query execution plans.

The frequency of updating statistics will depend on the editing activity in the geodatabase. Typically, more editing activity means you should update statistics more frequently. This is the responsibility of the database administrator, and not the ArcSDE software, which does not maintain statistics. You can update statistics for a table or feature class in ArcCatalog with the Analyze Component’s dialog box. It updates the statistics for the supporting tables that are associated with the selected object.

You should also update statistics on the SDE repository tables. This can be done with database management software. As a general rule of thumb, we suggest you update statistics at least weekly or monthly, and typically before and after a compress, which can be automated at the database level.

There is one exception: in situations where all users are editing just the SDE.DEFAULT version, you should just keep the statistics you collected before the compress. This will ensure that the query optimizer knows the delta tables are still active.

2. Rebuild indexes on tables

Indexes are used in a database to help speed up the retrieval of rows from a table, and they are also used by the database Query Optimizer when assessing query plans. As tables are modified by updates, inserts, and deletes of records, the corresponding indexes can become fragmented and unbalanced. This leads to increased I.O processing, which affects performance. This tip works in conjunction with the previous one. If you update statistics frequently, in turn you should consider rebuilding indexes if they are fragmented. Both actions will help improve performance.

In general, accurate statistics help to define a good index. You can assess the usefulness of an index with database management tools by monitoring its usage. Another benefit of rebuilding indexes is that you may reclaim disk space that was caused by its fragmentation. In versioned editing environments (where edits are performed daily), you may want to consider rebuilding indexes at regular intervals (for example, weekly or monthly), to keep performance degradation under control. We recommend you rebuild indexes after a compress. You can rebuild indexes within a database management program, or with ArcSDE commands.

For more information, see Knowledge Base (or KB) Article #24518, titled, FAQ: How can ArcSDE performance be improved?

3. Plan parent-child version relationships carefully

The versioning environment within an ArcSDE geodatabase enables users to implement and sustain complex business workflows. Typically the number of versions and how they are interrelated will depend on your business workflow. It is important to properly manage versions in the geodatabase, because poor version management will impact performance. You should keep the following in mind: every edit in the geodatabase is adding a state to the state tree. A state tree represents the total number of edits states stored in a geodatabase. Think of it conceptually like a flow chart diagram of circles and lines that flows from top to bottom. Each represents an edit state, and each state is linked by a line showing the edit history in the geodatabase.

A state tree, typically, has a structure similar to an upside-down tree, starting with one circle at the top (let’s say its zero), and flowing down in many branches. For example, a typical ArcSDE geodatabase may have approximately one million edits per day, resulting in hundreds of thousands of edit states in a state tree.

Ideally, you want to keep the state tree as simple and as small as possible. Versions are pointers to an edit state, and they will “pin” the state tree; in other words, they will keep its structure complicated. This can affect performance, because it may take queries longer to execute. Therefore, the more complex the versioning model (in other words, the more versions you have), means more potential records in the delta tables, which means potentially slower performance.

In general, you should try to do the following;

  • Reconcile versions to the SDE.DEFAULT version as soon as you can.
  • Delete versions when they are no longer needed.
  • Avoid creating versions that will never be reconciled with SDE.DEFAULT.

You could also run multiple reconcile services, to reconcile without posting as many older versions as possible each evening. This operation will simplify the state tree, so that when a compress is finally executed, it will trim the state tree. Version management can be performed in the Version Management dialog box in ArcCatalog or ArcMap.

For more information, read the ESRI technical white paper titled Versioning Workflows on the ESRI support site.

4. Compress the geodatabase often

Compressing an ArcSDE geodatabase helps maintain database performance by removing unused data.

Specifically it does two things:

  • First, it removes unreferenced dates, and their associated delta table rows.
  • Second, it moves entries in the delta tables that are common to all versions into the base tables, thus reducing the amount of data that the database searches through when executing queries. In effect, a compress will improve query performance and system response time by reducing the depth and complexity of the state tree.

When a large volume of uncompressed changes have accumulated in an ArcSDE geodatabase, a compress operation can take hours or even days. This is another very common cause of poor performance. To avoid this, you should compress on a regular basis (daily, weekly, and after periods of high editing activity). Users can stay connected to the geodatabase during a compress, but we suggest that all users be disconnected for the compress operation to be fully effective.

Remember to update statistics before and after a compress, and note the one exception mentioned earlier. The compress command is available in ArcCatalog. You add the command from the Customize dialog box, and you must be connected as the SDE user to execute it, or you could execute a compress with SDE commands.

For more information, see KB Article #29160 titled How to Compress a Version Database to State Zero.

5. Monitor system resources

When experiencing intermittent performance issues, it may be helpful to monitor the memory and CPU usage on both the client and server machines. This may help identify on which machine the performance bottleneck is occurring. For memory, it is important to ensure that the operating system is not running out of available memory and using swap space (in other words, virtual memory). Enterprise level ArcSDE typically needs at least one gigabyte of free disk space to operate efficiently. For CPU, you want to avoid and reduce how often the system hits a hundred percent CPU usage. Some troubleshooting suggestions to improve server performance include:

  • Closing unrelated applications on the server
  • Performing a database trace to examine and review performance (what’s in the database)
  • You could have users switch from application server connections to direct connects (this will put more workload on the client and less on the server)

For tips on improving client performance, refer to the ESRI Instructional Series Podcast titled Performance Tips and Tricks: ArcSDE Client-Side Optimization.

So, just to review, the performance of an ArcSDE geodatabase is influenced by many factors: hardware configuration, network configuration, network traffic, and the number of concurrent users.

The five best practices for maintaining an ArcSDE geodatabase covered in this post were:

  • Increase the frequency of updating statistics on tables
  • Rebuild indexes on tables
  • Plan parent-child version relationships carefully
  • Compress the geodatabase often
  • Monitor system resources

For more information, see the help topic An Overview of Tuning an ArcSDE Geodatabase.

ESRI also offers several instructor-led training classes on the configuration and tuning of ArcSDE geodatabases, based on DB2, Informix, Oracle and SQL Server database platforms.

Posted in Geodata | Tagged , , | 5 Comments

ArcGIS Explorer 600 – A Sneak Peek

The next release of ArcGIS Explorer – Build 480 - is coming up quickly, with some important new features and enhancements.  But at the recent ESRI Developer Summit plenary session we took a longer look ahead, beyond Explorer 480, and showed the first public demonstration of ArcGIS Explorer 600 and its new user interface.

The goal of ArcGIS Explorer 600 is to build on the foundation of previous Explorer releases while improving usability and making it more accessible to non-GIS users through the adoption of a new Microsoft Office-style ribbon-based UI.

Using the new ribbon user interface will make the functions of the application easier to find and experiment with. The ribbon also makes it possible to display only the functions that are applicable to what you are doing and hides controls that are not relevant. Other key aspects of the new user experience include:

  • Better map content management
  • Easier access to ArcGIS Online basemaps
  • Improved window management

Like Explorer 480, Explorer 600 will also include some new key functions. One of the big changes coming up in Explorer 600 is the ability to view your map in both 2D as well as 3D. You’ll be able to easily toggle the view between both modes to visualize your geographic information in different ways.

Stay tuned for more on the upcoming Explorer 480 and 600 releases…

Posted in Uncategorized | Tagged , , , | Leave a comment

Showing parcel hooks/ties with representation symbology

By Charlie Frye, Esri Chief Cartographer

Example of a parcel hook or tie that denotes a connection between two land parcels

Last week we received a good question on Ask a Cartographer: is there a way to create parcel hooks with ArcGIS 9.2? After first discovering what parcel hooks were and finding an example of a typical case (shown at left), the answer seemed like a ‘no-brainer’. Sure, just use the line decoration option on a marker line representation. However, after acquiring some parcel data that included parcel hooks (thanks to Greene County, NC) and seeing the circumstances where parcel hooks (they’re also called land hooks or parcel ties) are placed on maps, I learned there are are several scenarios that need to be handled. Continue reading

Posted in Mapping | Tagged | 2 Comments

Geodatabase Essentials – Part I: What is the Geodatabase?

Now that the dev summit is over we can tone down the amateur journalism and get into some real topics. Seeing as how this blog is titled “Inside the Geodatabase”, I thought a good place to start would be an introductory topic on the geodatabase. So here is the first in a series of posts we’re calling “Geodatabase Essentials”. Future posts tagged with this title will contain introductory information laying the foundations for essential geodatabase topics.

What is the Geodatabase?

The geodatabase is the native data storage and data management framework for ArcGIS. Why would you want to use a geodatabase? Because it acts as an organizational tool to store and manage your data, and is also the gateway into advanced GIS capabilities.

The geodatabase is a container which houses a collection of various geographic datasets.

Geodatabases support all of the different types of data that can be used by ArcGIS. Also, there is a complete set of conversion tools available so you can easily migrate existing geospatial data into the geodatabase.

At face value the fundamental ArcGIS datasets are tables, feature classes, and rasters. These and other more complex datasets, such as topologies and geometric networks, are all contained within the geodatabase. The geodatabase can also add advanced capabilities to these datasets and model behavior. Some examples of this are:

  • Data Validation using domains and subtypes
  • Multiuser editing environment through versioning
  • Topologies to enforce the integrity of your spatial data
  • Networks to model and analyze flows
  • Terrains for modeling surfaces and Lidar data management
  • Distributed geodatabase replication
  • Managing historical archives of your data

There are 3 types of geodatabases: Personal, File, and ArcSDE.

Personal Geodatabases

Personal geodatabases were first introduced in ArcGIS 8.0 and are designed for a single user working with smaller datasets. They are stored and managed in Microsoft® Access™, which ties them to the windows platform.

One thing a lot of users like about the personal geodatabase is the ability to manage the tabular data using Access.

Access based personal geodatabases work well for small datasets and they support all the features of the geodatabase model such as topologies, raster catalogs, network datasets, address locators, etc…. They are single user and therefore do not support versioning and long transactions.

File Geodatabases

File geodatabases, introduced at ArcGIS 9.2, store datasets in a file system folder and are portable across operating systems. They are suitable for single user projects and small workgroups with one editor and multiple readers. Although they do not support versioning, it is possible to have multiple editors with a file geodatabase, providing they aren’t editing the same feature datasets, feature classes or tables.

The file geodatabase is optimized for use in ArcGIS so it provides very fast data use and storage, and can scale to over 1 terabyte in size. Also, the File geodatabase allows you to optionally compress your vector data, reducing the memory footprint used by its storage without affecting performance.

ArcSDE Geodatabases

ArcSDE geodatabases manage spatial data within an RDBMS such as DB2, Informix, Oracle, SQL Server, PostgreSQL and SQL Server Express. Through this architecture, ArcSDE offers a multi-user editing environment and can manage extremely large datasets. ArcSDE geodatabases also support version-based workflows such as geodatabase replication and archiving that are not supported with file and personal geodatabases.

Organizations requiring the full suite of geodatabase functionality and a geodatabase with the capacity for extremely large, continuous GIS datasets that can be edited and accessed by many users should use an ArcSDE geodatabase.

9.3 Beta users can find this information and more on the Geodatabase Resource Center page.

Also for beta users, a few useful topics from the help system covering this information in more detail are: An overview of the geodatabase, Essential readings about the geodatabase, and Types of Geodatabases.

Users not in the beta program can access similar topics covering this information in more detail from the 9.2 web help. See An overview of the geodatabase and Types of geodatabases.

Posted in Geodata | Tagged , | Leave a comment

Get to know your organization's developers

By Charlie Frye, Esri Chief Cartographer

Early last month Chris Sergent forwarded me an email via Mapping Center’s submit a blog entry feature.  Chris, who is a developer, was looking to connect with other Esri developers. The spirit of Chris’s email struck me as something we could all do with a little more of, so we put Chris’s post up on the Esri Developer Summit blog. Continue reading

Posted in Mapping | Tagged , , | Leave a comment

AIRS Curtain Plots Custom Task

One of the custom tasks shown during the Developer Summit plenary was authored by John Grayson, a veteran of ESRI’s Applications Prototype Laboratory. Here’s his description of the task.

This ArcGIS Explorer task has been implemented to visualize data collected by the AIRS satellite, a project of NASA’s Jet Propulsion Laboratory.  The AIRS site provides the following description of the technology:

The Atmospheric Infrared Sounder (AIRS) looks down at the Earth and measures the infrared brightness coming up through and from the atmosphere. The AIRS optical system splits infrared radiation into its constituent “colors”. The effect (but not the technique) is similar to rain drops splitting sunlight into a rainbow. The wavelengths used are sensitive to temperature and water vapor over a range of heights in the atmosphere, from the surface up into the stratosphere. The term “sounder” in the instrument’s name refers to the fact that temperature and water vapor are measured as functions of height. AIRS also measures clouds, abundances of trace components in the atmosphere including ozone, carbon monoxide, carbon dioxide, methane, and sulfur dioxide, and detects suspended dust particles.

The data was preprocessed from its original HDF format into a geodatabase where information about when the data was collected and to which swath it pertained is maintained.  Additionally, curtain plots (vertical rasters) for each of the five attributes were generated as a server-side process.  The task starts out by reading the geodatabase and enabling the user to specify which date and swatch to visualize. The screen capture below shows the custom task interface, which allows the user to load the data and choose from different types of information to display. 

 

Footprints of each swath are drawn, and below they can be seen as blue rectangles on the globe. 

 

Using the custom task UI, the user can choose from a variety of types. The task uses OpenGL to load the curtain plots for the specified type, date, and swath onto the display.  Shown below is the complete set of curtain plots showing carbon monoxide levels for one of the swaths.

A slider control allows the user to quickly visualize one curtain plot at a time, providing a simple exploratory tool to visualize the data in the display.  Shown below is one of the curtain plots, and using the task slider the user can move forwards and backwards in the curtain plot sequence.

Here is another example, this one showing ozone levels. You can compare this to the earlier screen capture above in this post showing carbon monoxide levels.

The task leverages the ArcGIS Explorer SDK and it’s support for integrating custom OpenGL implementations. The task makes use of exposed events that tell us when the application has finished drawing and when OpenGL calls can be made.  OpenGL allows the developer to create tasks that perform visualizations that are not otherwise possible via the Explorer API, and provides a wealth of potential possibilities for the Explorer developer.

Posted in Uncategorized | Tagged , , , | Leave a comment

Video: Creating a web application with ArcGIS Server Manager

This video demonstrates how to use ArcGIS Server Manager to create a web application. You’ll find it useful if you’re getting started with ArcGIS Server, or if you just want to see what ArcGIS Server can do.

If you want to follow the steps in this video, you’ll need to install ArcGIS Server, create at least one map service, and log into Manager.

What other video topics would you find interesting for the blog or for the ArcGIS Server Help? Leave us a comment with your ideas.

Posted in Services | Tagged , , , , | 3 Comments

Search Mapping Center content

By Aileen Buckley, Mapping Center Lead

We’ve added new site-wide search functionality to Mapping Center.  As our content and audience continue to grow, we will continue our efforts to make it easy to quickly find the information you need.  Continue reading

Posted in Mapping | Tagged , | Leave a comment

BPC and Dev Summit – Some Custom Tasks

Last week was a busy week for the entire Explorer team, with lots of interesting things happening and lots of great discussions in the Showcase area and elsewhere at Palm Springs. We appreciate all of your comments and feedback. We’ll be highlighting some of the presentations and those discussions in a series of posts beginning this week.

First, here’s a few things that were shown during the ESRI Business Partner Conference and Developer Summit plenaries. We’ve covered some of the general things that we typically demonstrate in previous posts on the ESRI Federal User Conference, Petroleum User Group meeting, and GITA, so we’ll focus on some of the different things that were shown last week.

One of the custom tasks demonstrated is one you’ll find on the Explorer Resource Center. Just choose File > Resource Center and choose the Tasks tab.

 

The one we chose is the Weather Finder task authored by Michael Waltuch, one of the lead designers on the ArcGIS Explorer development team. Just click to open the NMF file, which will add the task to your current session. To learn more about this task click the Description link on the Resource Center entry, or choose About Weather Finder on the task UI. You’ll learn the following:

Weather Finder locates the nearest weather station to a specified location and reports its most recent weather observations. The task is based on a reverse geocoding web service located at GeoNames.org. To learn more about GeoNames go to their web site and see the service description.

The popup window associated with the weather station includes a link to the National Weather Service’s Telecommunication Operations Center which lists more information and a 24 hour summary for the weather station.

This is a good example of a custom task which has been written using the ArcGIS Explorer SDK to work against an existing Web service – In other words it’s a mashup.

Using one of the Portland bookmarks, we right-clicked the pushpin to use the Send To capability to use this location as the starting point for the Weather Finder task. The bookmark itself is the result of a task (the Create Notes task in this case), and when you take the result of one task and send it to another we refer to that as task chaining, and you can read more about that in the Help.

Here’s the result of the Weather Finder task, the closest weather station to our Portland bookmark. In this case it’s located at the airport. If we click the result pushpin you’ll see the latest weather information in the popup, including links to the NOAA site providing more detailed information and historic data for this location.

We also used a task we obtained from the Showcase area on the Explorer Resource Center. The showcase is a place where anyone can contribute tasks, results, or other items they want to share with the Explorer community. We chose the Flickr Task authored by Rob Dunfey, a member of the ArcGIS Explorer SDK team who also authored a blog post with more details about the custom task. This task is another good example of a mashup with other existing Web-based APIs, in this case it uses the Fickr API.

Here we typed in “portland” to return the first 20 Flickr photos tagged with that keyword. Each of the results can be clicked to open a popup displaying the photo.

Another task that was shown was a custom geoprocessing task, authored using the Model Builder in ArcGIS Desktop and published using ArcGIS Server. The task’s UI was automatically generated by ArcGIS Server, based on parameters specified by the task author. The ArcGIS Explorer SDK can be used to modify its UI, but we just used the default, automatically generated one. 

Here we’ve used Send To again to use our Portland bookmark as the input location for a 1 and 3 minute drive time analysis.

In upcoming posts we’ll cover more of what was shown, including some of the technical details of the plenary and workshop presentations, and also cover some of the things we showed that are coming in future releases.

Posted in Uncategorized | Tagged , , , , , | Leave a comment