Category: Geodata

Direct connections to ArcSDE geodatabases

This post was written by Kim Peter, a product engineer on the geodatabase team, and is a follow up to her post on Connections to 9.3 Geodatabases

There are two types of connections you can make to ArcSDE geodatabases: direct connections or connections through an ArcSDE service. This post deals with making direct connections to ArcSDE geodatabases with ArcGIS clients, where the ArcGIS client and ArcSDE geodatabase are from different releases.

You may have heard that to make direct connections from an ArcGIS client, you no longer need to have the client software and the geodatabase at the same release. This is true for ArcGIS 9.2 service pack 5 (SP5) or SP6 clients making a connection to an ArcSDE 9.3 geodatabase and for ArcGIS 9.3 clients making direct connections to an ArcSDE 9.0, 9.1, 9.2, or 9.3 geodatabase.

Here’s some information you need to successfully set up these inter-release direct connections.

  • You must perform a separate installation to get the direct connect drivers necessary to make these connections.
    • The setup to obtain the 9.0, 9.1, and 9.2 direct connect drivers is called ArcGIS Pre-9.3 GDB Direct Connect Setup and is included on your 9.3 client media along with an installation guide.
    • The setup to obtain the 9.3 drivers needed to make a direct connection from an ArcGIS 9.2 SP5 or SP6 client to an ArcSDE 9.3 geodatabase must be downloaded from the ESRI support site Patches and Service packs download page. This setup is called ArcGIS 9.3 GDB Direct Connect for 9.2 Clients Setup. An installation guide for this setup is also provided with the download.
  • The ArcSDE 9.0, 9.1, or 9.2 geodatabase to which you are making a direct connection must have the latest service packs applied. Check the ESRI support site’s download page for the latest service pack for each release.
  • Direct connections from an ArcGIS 9.3 client to an ArcSDE 9.0, 9.1, or 9.2 geodatabase for Informix is not supported.
  • Direct connections from an ArcGIS 9.3 client to an ArcSDE 9.0 geodatabase for Oracle 8i are not supported.
  • When you connect to a geodatabase on an ArcSDE database server (an ArcSDE geodatabase for SQL Server Express), you always make a direct connection. Be aware that the upgrade procedure for ArcSDE geodatabases for SQL Server Express will differ depending on whether or not you have the 9.2 direct connect drivers installed. See Upgrading geodatabases on ArcSDE database servers in the help to see how to upgrade these geodatabases. (Note that at 9.3 final release there is a bug that will prevent you from upgrading the geodatabase if you have the 9.2 direct connect drivers installed. If you want to upgrade the geodatabase, do it from an ArcGIS 9.3 client that does not have the 9.2 direct connect driver installed.)

For information on connections, see An overview of ArcSDE geodatabase connections and Properties of a direct connection.

Posted in Geodata | Tagged , | 5 Comments

9.3 Webhelp now publically available

The 9.3 Web help is now publically available since the login requirement was lifted yesterday. The web help is a great resource for up to date information. Even after the product is released the web help gets updated weekly with content and corrections.

Check out the What’s new in 9.3 section to see what new functionality 9.3 has to offer.

As for geodatabase related documentation, there are lots of new topics and content, such as:

Merge geometries
Using the Version Changes command
Mosaicking raster datasets
Color correcting using raster data
Raster Clip geoprocessing tool
ArcSDE connection syntax
Compatibility between clients and geodatabases
Inside a geodatabase in PostgreSQL

The Resource Centers will also be publically available soon, we’ll keep you posted on those.

Posted in Geodata | Tagged , , , | Leave a comment

Connections to 9.3 geodatabases

This post was submitted by Kim Peter who is a product engineer and technical writer on the geodatabase development team

In order to take advantage of new functionality added to the geodatabase in a release of ArcGIS, the geodatabase must be upgraded.  This will be the case with ArcGIS 9.3, as we’ve added new functionality to terrains and network datasets:

To see how you can upgrade a geodatabase to the 9.3 release, consult the help topics Upgrading file and personal geodatabases, Upgrading geodatabases on database servers, and Upgrade summary for ArcSDE geodatabases.

However, this doesn’t mean that you need to upgrade your ArcGIS 9.3 clients to 9.3 in order to connect to and use 9.3 geodatabases. You will still be able to connect from a client with ArcGIS 9.2 SP5 or later service pack to a 9.3 geodatabase. For example, if a coworker sends you a file geodatabase he or she created with an ArcGIS Desktop 9.3 client, you can open and edit it in your ArcGIS 9.2 SP5 client.

As with previous geodatabase upgrades, if you decide not to upgrade your geodatabases but you do install the 9.3 release of the client software, you will be able to connect from the 9.3 client to 9.2 geodatabases. If you don’t need to take advantage of the new functionality for terrains and network datasets, the geodatabase upgrade is optional and not required.

Be aware, though, there are a couple of caveats to these inter-release connections.

They are as follows:

  • You have to have at least service pack 5 applied to the client application to be able to connect and edit from a 9.2 client to a geodatabase that has been upgraded to the 9.3 release.
  • If connecting from a 9.2 SP5 or SP6 client to a 9.3 geodatabase, you will have all the 9.2 functionality available to you.
  • If connecting from an ArcGIS 9.3 client to a geodatabase that has not been upgraded from 9.2, ArcGIS will prevent you from using 9.3 functionality in the geodatabase.
  • If making direct connections from the client application to an ArcSDE geodatabase, additional drivers need to be installed. This configuration will be discussed in a future blog post.
Posted in Geodata | Tagged | Leave a comment

Road Ahead – Extending One-Way Replication

We’ve extended the one-way replication model at 9.3 to include file and personal geodatabases. So now when you are creating a one-way replica, the destination doesn’t have to be an ArcSDE geodatabase, it can also be a file or personal geodatabase.

This opens up a host of new replication scenarios since the child replica doesn’t need to be versioned or have SDE technology. This will also leverage one-way implementations such as production-publication scenarios or mobile users working in the field.

Here are a few more sources of information on replication topics:

  • Also, a new video in the 9.3 help system offers an example of how to create and synchronize replicas in a connected environment.

 

Posted in Geodata | Tagged , , | 1 Comment

Geodatabase Essentials – Archiving

Geodatabase archiving provides a mechanism for capturing and managing a record of your edits as features and rows are updated over time. This record is automatically updated as edits are saved and is available for query and analysis. This is handy when you want to go back and see how a feature has changed over time, see how a feature looked at a certain point in time, see how a certain area may have once looked, etc…

Archiving supports the complete geodatabase data model so topologies, geometric networks, feature datasets, relationship classes, tables and feature classes can all participate.

So here’s how it works:

First you enable all or a subset of your data as archiving. Only versioned data may be archiving enabled though, so you may have to first register your data as versioned. When the data has archiving enabled a class is associated with the original class known as the archive class. It is this archive class that maintains a record of the modifications made to the original data. The archive class has the same name as the original data, but with an “_H” appended on to the end. For example, if a feature class named “Roads” had archiving enabled on it, the archive class would be named “Roads_H”.

What’s nice about archiving is that any changes made to a archiving enabled class are recorded automatically. It’s not something that is added to your workflow that you have to remind yourself to do after you make your edits. Once archiving is enabled on a class, any changes are automatically managed by the archive class.

A few new videos were added to the help system at 9.3 to demonstrate geodatabase archiving functionality. The videos offer examples on:

  • Enabling archiving on a dataset
  • Creating an historical marker and connecting to an historical version
  • Using the geodatabase history viewer
  • Working with the archive class

 

You can check the videos out HERE

Posted in Geodata | Tagged , , , | Leave a comment

Developer Tips – Using geodatabase methods with conformant array parameters in .NET and Java

 This post was written by geodatabase Product Engineer James MacKay. James works on the geodatabase development team and is responsible for a lot of the geodatabase SDK that is generated.

There are several methods in the Geodatabase API that use conformant arrays: these are C-style arrays that can be dimensioned at runtime. In most cases, if you see a method with a pair of related parameters – an integer indicating capacity (or something similar) and a pointer to an integer indicating values (or something similar) – it’s a safe bet that the method is looking for a conformant array.

A method with a conformant array parameter: ISelectionSet.AddList

Unfortunately, conformant arrays and Interop don’t jive… although they may work occasionally, eventually they’re going to cause problems. There are two workarounds: the GeoDatabaseHelper class and GEN interfaces.

The GeoDatabaseHelper class implements two interfaces, IGeoDatabaseBridge and IGeoDatabaseBridge2, which implement methods that can’t be used in a straightforward way through Interop. Three of the more common methods are IFeatureClass.GetFeatures, ISelectionSet.AddList, and ISelectionSet.RemoveList. The workarounds are IGeoDatabaseBridge.GetFeatures, IGeoDatabaseBridge2.AddList, and IGeoDatabaseBridge2.RemoveList, respectively.

GEN interfaces are identical to the interfaces with methods that use conformant arrays, but with the array type defined as SAFEARRAY instead of conformant arrays; they are implemented by the same classes. There are four GEN interfaces defined in the esriGeodatabase library: INetTopologyEditGEN, IForwardStarGEN, IUtilityNetworkGEN, and IEnumNetEIDBuilderGEN.

 

Posted in Geodata | Tagged , , , | Leave a comment

Five reasons why you should be using the File Geodatabase

There are many options to the GIS user when deciding what data store to use to store geographic data. At ArcGIS 9.2 we introduced a new type of geodatabase, the File Geodatabase. While all types of geodatabases have their strengths and weaknesses, we thought it would be useful to highlight some of the strengths of using this new type of geodatabase.

Size

The database size is only limited by the available disk space. By default, individual tables and feature classes can be up to 1 TB. With the use of configuration keywords this can be expanded to 256 TB.

Versatility

Works on many different operating systems including Windows and UNIX (Solaris and Linux)

Speed

Provides excellent performance and scalability. For example, to support individual datasets containing well over 300 million features and datasets that can scale beyond 500 GB per file with very fast performance. The file geodatabase out performs shapefiles for operations involving attributes, and scales the data size limits way beyond shapefile limits. Through the use of an efficient data structure that is optimized for performance and storage, File Geodatabases use about one third of the feature geometry storage required by shapefiles and Personal Geodatabases.

Edit Model

The File Geodatabase uses an edit model similar to shapefiles, supporting one editor and multiple readers.  Each standalone feature class, table and feature dataset can be edited by different editors simultaneously but can only have one editor performing edits on them at any given time. This means that User A can edit the Roads Feature Class at the same time as User B edits the Parcels Feature Class.

Compression

File Geodatabases also allow users to compress feature classes and tables to a read-only format to reduce storage requirements even further. This reduces the Geodatabase’s overall foot-print on disk without reducing the performance.

Posted in Geodata | Tagged , | 7 Comments

Road Ahead – New workflow for mosaicking raster datasets

The following post was written by Simon Woo, a product engineer on the geodatabase team specializing in Raster support in the geodatabase.

Previous versions of ArcGIS have provided various mosaicking workflows.  We’ve tried to make the mosaicking experience better in each version, but have had varying feedback on performance, especially when outputting to a file format. 

Prior to ArcGIS 9.3, the workflow to create a mosaic was to create a raster dataset and then use the Workspace to Raster Dataset tool to populate the new dataset.

In ArcGIS 9.3, we have provided a new workflow to make your mosaicking experience faster. 

The new workflow is to:

  • Create an unmanaged raster catalog with the Create Raster Catalog tool.
  • Load all the raster datasets into the unmanaged raster catalog with the Raster to Geodatabase tool.
  • Use the Raster Catalog to Raster Dataset tool to mosaic the datasets together. 

From the tests that we have run, this new workflow is considerably faster.  Below are a couple examples of comparison tests that have been run.   

Mosaic Test 1

Here’s the test data we used:
Type of data: 3-band
Input format: TIFF
Number of files: 20
Average file size: 38.8 MB
Output format: TIFF


 

Using the old workflow of creating a raster dataset and then using the Workspace to Raster Dataset tool, this data took 1 hour and 35 minutes to mosaic.

Here are the steps with the new workflow:

First create an unmanaged raster catalog.

This creates an unmanaged raster catalog. Now load the data into the new unmanaged raster catalog:

Now use the Raster Catalog to Raster Dataset tool to mosaic the rasters in the unmanaged raster catalog into a single raster dataset.

 

 

We conducted a similar test using GRID data and it yielded the following results:

Mosaic Test 2
Type of data: single band data
Input format: GRID
Number of files: 60
Average file size: 5.7 MB
Output format: GRID

Old Workflow – 45 minutes              
Create Raster Dataset time: 1 second
      Workspace to Raster Dataset time: 45 minutes

New workflow – 4 minutes
     
Create Unmanaged Raster Catalog: 1 second
Raster to Geodatabase tool: 1 minute
Raster Catalog to Raster Dataset tool: 3 minutes

The new workflow is shown in model form in the following graphic:

Posted in Geodata | Tagged , , | 2 Comments

Road Ahead – PostgreSQL database support at ArcGIS 9.3

The following post was written by Kasia Tuszynska, a product engineer on geodatabase team specializing in PostgreSQL and the geodatabase.

The release of ArcGIS 9.3 will see an addition to the list of databases supported by ArcSDE. PostgreSQL 8.3.0 is joining the ranks of Oracle, Sql Server Informix and DB2 and is the first open source database to be supported by ArcSDE. I previously worked with SQL Server and Oracle and did not really know what to expect from Postgres, but I found it a very clever and well thought out database, I loved the online community, and I learned more from reading forums than any book out there.

We had an interesting time implementing ArcSDE for Postgres because we wanted to support the PostGIS spatial type and our own spatial type implementation st_geometry. This was a conscious decision so our customers who use both our software and open source software would not have to choose between one and the other but could use both, and potentially even keep it all in one database. We also implemented our own spatial type, st_geometry, it was designed to work with the different geodatabase functionality, we implemented it to make sure that any customer who wants to try out what geodatabases have to offer or who would choose to transition from one database to another would not loose any geodatabase functionality.

ArcSDE for Postgres will be released on Windows, Red Hat Linux and SUSE Linux. Since Postgres is open source and released with a BSD license we could include it in our installation, so if a customer wanted to install Postgres and SDE in one swoop they could do that in windows by executing one installation wizard, or two shell scripts for an install on Red Hat Linux. With SUSE we were halted by the lack of precompiled binaries for Postgres so installation will have to be done the old fashioned way – from source, but the rest of the setup can be done with scripts.

Implementing ArcSDE on Postgres had us learn the ins and out of the MVCC concurrency model, altering the way we send updates to postgres. We spent quite a long time tweaking the sql related to editing, trying to shave seconds off of the performance time. A lot of effort has also been put into the documentation, for folks already familiar with ArcSDE administration moving to postgres will probably be a snap but there a couple of small things to remember for postgres, so when in doubt please check the help. For instance, to avoid having your feature classes looking like tables in ArcCatalog and to make them accessible to everyone the “usage” privilege has to be granted on a schema to the public role. Also, installing Postgres on Windows has to be done through an admin account but the account under which Postgres (postmaster) is to run can not belong to the admin user group … Postgres idiosyncrasies like this can drive one crazy especially if you are not used to it.

This is our initial release and I am hoping that all of our testing caught every annoying little bug, but in case we missed something please do not hesitate to drop us a bug in Nimbus.

Posted in Geodata | Tagged , , | 3 Comments

Road Ahead – Version Changes Viewer

Ever connect to a version of a Geodatabase and ask “How is this Version different” ? At ArcGIS 9.3 we added the Version Changes Viewer to the Versioning Toolbar to answer this very question. The Version Changes Viewer provides the capability to view the changes made to a version since it was created or last reconciled with an ancestor version.

This is useful because:

  • You can quickly see all the changes in a version without doing a Reconcile.
  • The tool can be used to see the changes just within the current edit session or current map extent.
  • You do not have to be editing to see what has changed in the version

How it works:

When the dialog is opened you select a version to compare changes with. The dialog lists all modified classes, inserts, updates and deletes, and allows you to view these in a similar experience to the interactive Conflicts Resolution dialog. It also allows you to view the changes made during your current edit session.

The Version Changes dialog will list all the changes made to the version for all layers in the map document based on the selected workspace. If the layer is not currently visible, changes on that layer will still be present in the dialog. Optionally, the changes can be restricted to only those changes in the present map extent by selecting the “filter changes using map extent” check box in the initial Version Selection dialog.

The Version Changes dialog box displays the number of changes that have been made to the version from the time it and the chosen target version were identical. The number of changes is subdivided by class, and then further subdivided into Insert, Delete, and Update categories. Clicking an ObjectID number shows its changes in the right window. All the attribute values for the two versions being compared and for their common ancestor are displayed. Attribute values displayed in bold signify that a change has been made to the version for that attribute.

Posted in Geodata | Tagged , , | 6 Comments