Tag: schema

The ArcGIS for Water Utilities Approach to Meeting Water Utility GIS Needs

ArcGIS for Water Utilities is an evolutionary step in how Esri’s GIS technology can be deployed at water utilities.  Over the last year we’ve had many conversations about how ArcGIS for Water Utilities enables water, sewer and stormwater utilities to take a better approach to meeting their GIS needs.  We’ve found one of the most effective ways to communicate the “ArcGIS for Water Utilities Approach” is to compare it with two other approaches to meeting water utility GIS needs we’ve seen – “The Legacy Approach” and the “ArcGIS System Approach”.

Legacy Approach

A Legacy Approach to Meeting Water Utility GIS Needs

What we call “The Legacy Approach” to meeting water utility GIS needs was an approach commonly used about 10 years ago.  This approach was typified by water utilities building their GIS from the bottom up, often with many projects over a multi-year period.  With this approach water utilities were spending a lot of time and money assembling a GIS platform and then creating customizations to perform industry common functions.

Continue reading

Posted in Water Utilities | Tagged , , , , , , , | 5 Comments

Benefits of the Local Government Information Model for Water Utilities

Why should a water, wastewater or stormwater utility adopt the Local Government Information Model?

Easier Deployment

One of the biggest benefits of a water utility adopting the Local Government Information Model is that it makes deploying the ArcGIS for Water Utilities maps and apps easier, faster and cheaper.  The further you deviate from the Local Government Information Model, and in particular it’s geodatabase schema, the harder it will be for you to implement the maps and apps that are part of ArcGIS for Water Utilities. It will also be hard and time consuming to upgrade your ArcGIS for Water Utilities implementation when we release updates.

Changes you make to the Local Government Information Model schema may necessitate extensive modifications of the maps documents, and changes to apps (web apps, mobile apps, ArcGIS Desktop, etc.) that are part of ArcGIS for Water Utilities.  So the closer you stay to the core Local Government Information Model, the easier your initial deployment will be and the easier it will be to migrate your ArcGIS implementation to new releases or to deploy updates to the maps and apps.

It’s also important to note that when we say “adopt” the Local Government Information Model we don’t mean that you necessarily have to use it as is (or more appropriately – as downloaded).  You probably will need to configure the Local Government Information to meet the needs of your organization.   But the key thing to keep in mind is you should only be making changes to accommodate the true organizational needs of your utility. For example, instead of changing the field names to the field names you’d like to use in your organization, modify field and map layer aliases.   Bottom line, don’t reinvent the wheel, just make changes that are required to meet specific business needs in your organization.

At the very least you need to change the projection to the appropriate coordinate system and set up the domains to reflect the assets in use at your utility.  Small utilities or utilities that are new to GIS may choose to take the Local Government Information Model as is, while larger utilities, mature GIS implementations, or GIS implementations that are integrated with other enterprise system will undoubtedly need to make more significant configurations or extensions to the schema to reflect their organizational needs.  

Water, Sewer and Stormwater Data Modeling Best Practices

The Local Government Information Model incorporates many best practices for water utility GIS.  One of the most important best practices is how to represent a water, sewer or stormwater system in GIS.  

For years Esri had downloadable data models for water, wastewater and stormwater utility networks.  Those data models were the first freely available water utility GIS data models.  They were stewarded by Esri, but built by the user community and became the industry standard.  Globally thousands of water utilities have built their GIS around Esri’s free data models.  

The Local Government Informational Model is the next iteration of Esri’s water, sewer and stormwater data models.  In essence we’ve modernized the data models to reflect how water utilities have been deploying GIS over the past few years and we’ve also modified the schema to fit the requirements of the ArcGIS for Water Utilities maps and apps.  As water utility GIS continues to evolve Esri will regularly maintain the Local Government Information Model to keep introducing new best practices into the user community and functionality into our apps.

Comprehensive Data Model

There is no doubt Esri’s water, wastewater and stormwater data models were an incredibly valuable starting point for water utilities to get their utility networks into GIS.  Since the original data models focused primarily on a data structure for the assets that comprise utility networks, we received feedback that many utilities wanted more guidance on how to model operational data (workorders, service requests, customer complaints, main breaks, capital improvement projects, etc.) and base data (roads edge of pavement, road centerlines, elevation data, parcels, etc.) in their GIS.  The Local Government Data Model solves this problem because it includes a complete schema for typical water utility base data and operational data.  

Over the years, an observation we’ve made is that water utilities struggle with how to model and manage schemas for datasets that aren’t their utility networks or operational data – simply put managing base data can be a challenge for water utilities. For example we’ve seen a lot of utilities struggle with managing roads, parcel, buildings, etc. in their enterprise GIS, especially when these datasets are coming from other organizations or departments.

This is a particular issue for water utilities that serve multiple units of local government such as authorities, county wide utilities, state wide utilities and private companies.  A good example of this is a water authority whose service territory includes three counties.   The water authority needs parcel data that is maintained by the counties.  County A, County B and County C all use different schemas for their parcels.  So the water utility had two choices – leave the parcels in 3 different data layers and use them as is – which makes analysis, map creation and integration with other systems at the utility that need parcel data (such as a customer information system) difficult.  Or invest time to extract, transfer and load (ETL) the parcels into a common schema so they can be used as a single seamless layer across the service area.  The Local Government Information Model can now serve as the common schema in this example.
Easier Data Sharing

We describe the Local Government Information as a harmonized information model – meaning designed to accommodate typical GIS needs across local government.  If organizations that commonly share data all adopt the Local Government Information Model, it will greatly reduce the time and resources spent establishing a common schema and migrating data to these schemas – thus allowing water utilities to focus on the maintenance and management of their authoritative data.

For example a private water utility may serve two municipalities.  If the water utility and both municipalities all adopt the Local Government Information Model then they can all very easily exchange data.   When the water utility needs road centerline and edge of pavement layers from the municipalities than the utility can just import the new data without having to manipulate the schema and will have seamless layers for their service areas.  The same logic applies to the water utility sharing data with the municipalities – when the water utility updates the location of their upcoming capital projects, the utility can share that data back with the municipalities and the municipalities can use it without any schema manipulation.

Best Cartographic Practices for Water Utility Maps

As we’ve discussed in a previous blog, the Local Government Information Model includes geodatabase schema, map documents and specification for services necessary to deploy the ArcGIS for Water Utilities and ArcGIS for Local Government maps and apps.  

The map documents highlight
best practices for displaying water, wastewater and stormwater data in the context that each map is designed to be used.  For example the map documents included with the Mobile Map Template have best practice cartography for displaying water utility GIS data in the field in both a day and night time use map.  The same goes for the map document included with the Infrastructure Editing Template – this is a best practice map document for editing water utility data with ArcGIS Desktop.

Looking to the Future

The specification for the services (map, feature, geoprocessing, etc) necessary for the ArcGIS Water Utilities maps and apps are also part of the Local Government Information Model.  So if other local government entities in the service area of water utility embrace the Local Government Information Model, ArcGIS for Local Government and start to publish services, then water utilities can consume those services for their maps and apps.  In this scenario the water utility may no longer have to import some data into their own geodatabase and can just consume the services right from the organization that is the steward of the data.

We hope you’ve found this exploration of some of the benefits water, wastewater and stormwater utilities will experience when adopting the Local Government Information Model helpful.  We encourage your feedback on the information in this blog, the Local Government Information Model or ArcGIS for Water Utilities.

Posted in Water Utilities | Tagged , , , , , , , , , , , , , , , , | 3 Comments

Data quality concerns keeping you up at night?

What keeps water, wastewater and stormwater utility GIS professionals up at night?   Could be doubts about your system architecture or capacity, might be fears about data backups and recovery, maybe your backlog of unprocessed as-builts.  A common concern we are hearing right now from the user community is about being sure that your data is good enough to meet the needs of your utility.  This is driving more water utilities to focus on quality assurance (QA) and quality control (QC).

Across the industry water utilities are expanding their GIS quality control procedures or implementing formalized quality control if they don’t have any in place.  Water utilities are also reviewing their existing GIS implementation and workflows for ways to increase quality assurance.  At some water utilities these changes are coming out the GIS department, driven by proactive GIS managers and staff.  At other utilities these changes are coming top down from utility management that recognize GIS data now runs throughout their utility like a steel thread or from the IT department as it assess the state of all utility digital data.

But haven’t we always been concerned about data quality?

Continue reading

Posted in Water Utilities | Tagged , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Updated Water & Wastewater Utility Data Models

Newly updated water and wastewater data models are available for download here – http://resources.arcgis.com/content/water-wastewater-and-stormwater-data-models

These data model updates include a number of changes that reflect our shared knowledge of how water, wastewater and stormwater utilities are currently modeling their utility networks for  in GIS.   So these updates include input from the user community as well as the experience of Esri professional services creating data models for water utilities using the downloadable data models as a starting point, and our knowledge of how water utilities are expanding their use of GIS to go beyond asset management.

The Benefits of the Data Model

So whether you are implementing GIS for the first time or updating your schema to support another business function at your utility with GIS, what are the benefits of starting with the downloadable data models?

Because using the downloadable data models help you (or your consultant) reduce the cost, time and complexity of implementing GIS or of updating your schema.

The data models reduce implementation cost because you aren’t building a data model from scratch, instead you start with a data model that reflects the way most water utilities are storing their data in GIS.  Before the downloadable data model, you had to pay a consultant to build you a data model from the ground up.  So being able to download the data model as a starting place saves you somewhere between $20,000 and $100,000 depending on the size of your utility, the complexity of your utility networks and your GIS needs.  

The data models reduce the time it takes to implement a GIS because you can focus on determining the gaps between the data models and what your GIS needs are.  We’ve even seen some small utilities that have downloaded the data model, changed the spatial reference and started to populate it without doing any customization of the schema (not that we recommend this, but it has been done).

The data models reduce the complexity of implementing a GIS because it gives you something to work against.  You can immediately use the downloadable data model for a pilot to understand how to bring your existing data into GIS.

These same principles hold true whether you are implementing GIS, doing a comprehensive modernization of your schema or just making minor modifications like adding a new feature class or new attributes.

The Relationship Between the Downloadable Data Model and the Template Schemas

Since we’re on the topic of data models, we also wanted to clear up some confusion about the downloadable data model and the schema that is used for the Water Utility Templates.

If you examine the downloadable data model and the schema used for the Water Utility Templates you’ll notice that they are not the same. Simply put the schema the templates are configured for and the sample geodatabases included with them aren’t the same as the downloadable data models.

So, why is the schema for the templates different than the downloadable data models?

It’s because when we created the templates, we followed the exact same process that you should follow when implementing, expanding the role of or deploying a GIS application at a utility.  We downloaded the data models and then created a schema that was customized to meet our needs.  Our needs in this case were to create a set of applications and maps with ArcGIS for data maintenance, planning, mobile GIS, operational awareness and customer interaction.

After downloading the data model, we identified the data we needed to support our maps and applications and then customized the data model until we had the necessary schema.  Once we had designed and created the data model for our templates we then transformed and loaded the sample data from Fort Pierce, Florida into our new template schema. For each of our templates we also did a series of pilots to ensure that maps, applications and our schema met our needs.  

Posted in Water Utilities | Tagged , , , , , , , , | Leave a comment

Take advantage of new Silverlight 4.0 features in the ArcGIS Silverlight API

Our recent 2.0 beta release of the ArcGIS Silverlight API was built on the Silverlight 4.0 platforn, taking advantage of several of its new capabilities. Below are a few of my favorite new features:

Namespace prefixes

In Silverlight 3.0 you had to use multiple namespace prefixes to access classes in XAML that were located in different namespaces. This meant you often ended up with something like this in the page header:


In Silverlight 4 we can now map multiple namespaces to the same xmlns prefix, and the 2.0 API takes full advantage of this both in Silverlight and WPF. This means that you don’t have to declare multiple prefixes for core, geometry, symbols, toolkit, etc… but can just use one for them all:


This is what it looks like in intellisense:

So even though the following four objects are all in different namespaces, the namespace prefix is the same:

<esri:Map />
<esri:Navigator />
<esri:MapPoint />
<esri:SimpleMarkerSymbol />

Graphic.Attributes Binding

Before Silverlight 4, support for binding to dictionaries was lacking compared to WPF, and you had to use a custom converter to access attributes on a graphic feature. In Silverlight 4.0, this is now much easier, and you will never have to use the DictionaryConverter again (round of applause please):

Silverlight 3 (assuming the DataContext is the Graphic.Attributes, for example inside a MapTip):

    <esriConverters:DictionaryConverter x:Key="MyDictionaryConverter" />

<TextBlock Text="{Binding Converter={StaticResource  
    MyDictionaryConverter},ConverterParameter=StateName}" />

Silverlight 4:

 <TextBlock Text="{Binding [StateName]}" />

Even more interesting is that you can now update your attributes inside a map tip by simply using Two-Way binding on a textbox:

<TextBox Text="{Binding [StateName], Mode=TwoWay}" />

From now on consider the DictionaryConverter to be obsolete. 

The attributes type has also been updated (implements IDictionary) so if you bind to the attributes and change any values, it will automatically trigger a rebind.


By far the #1 requested feature, not only for our API, but also for Silverlight in general. Silverlight 4 finally comes to the rescue and provides an API for printing. You simply give it a control to print:

PrintDocument doc = new PrintDocument();
doc.PrintPage += (s, e) =>
   e.PageVisual = MyMap;
   e.HasMorePages = false;

Binding to DependencyObject

Prior to Silverlight 4, you were only allowed to set binding expressions on objects that inherit from FrameworkElement. This limitation has been relaxed so now you can bind to anything that inherits from DependencyObject (like in WPF). This means you can bind an attribute to a symbol and rotate it based on the attribute value. Example:

<esri:MarkerSymbol x:Key="MySymbol">
       <TextBlock Text="=>">
           <RotateTransform Angle="{Binding Attributes[Heading]}" />
</esri:MarkerSymbol >

Design Time editor in Visual Studio 2010

Visual Studio 2010 now supports previewing and editing your XAML in design view, similar to what you did in Expression Blend. Our design time support works the same way in Visual Studio as in Blend, so you can now use Visual Studio to do some quick design-time editing.   However, I do believe Blend is still an invaluable tool and has very powerful support for creating good looking layouts, animations, etcetera. 

Morten Nielsen
Senior Software Engineer
ArcGIS Server.NET, Silverlight/WPF, MapIt

Posted in Web | Tagged , , , | 21 Comments

ArcGIS for AutoCAD Template for Water Utilities

There is a new template in the template gallery showing how to use ArcGIS for AutoCAD at a water utility to engageAutoCAD users in GIS data updates. 

You can download the template here:


And you can download ArcGIS for AutoCAD here:


How can you benefit from ArcGIS for AutoCAD at a water or wastewater utility?

ArcGIS for AutoCAD helps you solve 2 common problems.  The first is sharing GIS content with CAD users quickly without data duplication and wasting disk space.  The second is engaging CAD users in your data GIS data updates.  The template you can download shows you how to solve the 2nd problem.

Sharing GIS Data with CAD users

Because CAD and GIS are different tools intended for different purposes – GIS to manage, analyze and share spatial data and CAD to create designs for construction plans, they store data differently.  To maximize their benefit from GIS, most utilities centralize their data into a single multiuser geodatabase that is then shared among different departments.  A centralized geodatabase costs less to maintain, breaks down departmental information silos and help everyone collaborate at utilities.  The data in the centralized geodatabase can then be published out to users via a web map, a mobile application and also integrated in with other utility systems such as workorder, SCADA, billing or CIS.

For many utilities, design in CAD starts with data from GIS.  A typical workflow for starting a design project would be to export planimetric data (edge of pavement, building footprints, etc), cadastral data (parcel boundaries, ownership information), topography (contours, streams, etc) and utility network data (pipes, valves, manholes, hydrants) from GIS to CAD.  To export the GIS data to CAD (either DGN or DWG) geoprocessing tools are used to create the new CAD file with the utility’s CAD standards.  Some of the data exported from GIS will be used as background layers in the CAD file while some of the data will be used as starting point for the utility design.  Also aerial photos, quad sheets, etc from the GIS may be copied over as image files for use as background data for design.  If a utility doesn’t do their own design work they could provide the exported GIS files and imagery to their consulting engineers.

So for each design project a utility undertakes, DWGs or DGNs exported from GIS and file based aerial photography may be stored in the directory for the design.  Very quickly you may have many data snapshots exported from GIS and the same aerial photos duplicated many times to be used in multiple design projects over time. So this type of workflow will begin to use up a lot of storage space (especially if designers are using aerial photos) and also is creating a file management headache.

ArcGIS for AutoCAD solves this problem by allowing you to serve maps from ArcGIS Server to your CAD users and eliminating much of the file exporting and data duplication; more efficiently providing data for designers and saving disk storage space.  In more technical terms, AutoCAD would be consuming map services from ArcGIS.  So for example, you could publish a map to ArcGIS Server that has your parcels, road edge of pavement and building footprints symbolized to mimic your CAD standards and every time you need to show parcels, edge of pavement & building footprints in a design just use the map service with ArcGIS for AutoCAD.  The same goes for aerial photos (or another good option is to use ArcGIS Image Server to serve aerial photos up to AutoCAD & Microstation). 

Another benefit of using ArcGIS for AutoCAD is that ArcGIS Server is doing the work of generating the map services that AutoCAD consumes, labels can be dynamically created by ArcGIS Server using the GIS labeling engine.  As some of you may have experienced, exporting dynamically drawn labels from GIS to a DWG is a multistep process.  So for example, ArcGIS for AutoCAD can consume a map service that is creating road and parcel labels on the fly.

Engaging AutoCAD users in GIS data updates

The other benefit of ArcGIS for AutoCAD I wanted to highlight is getting the appropriate data from your designs from CAD into ArcGIS.  This is what the downloadable template illustrates.  We already explored how design begins with GIS, now let’s explore how it ends with GIS, meaning that when the design is finished you want to get data back into GIS because that is the system of record for most utilities to store their asset data.

So if you are using ArcGIS for AutoCAD to organize your CAD data into attributed feature classes and configured for your utility’s CAD standards & GIS data model than bringing data design data in a DWG will be much easier. 

For example, after you’ve created a design for a new sewer main extension, you now need to get the new proposed main features and attributes (material, diameter, etc) back into your GIS and in your proposed features dataset.  With the criteria for feature classes configured in your AutoCAD files, you can just open the DWG in ArcMap and then us a geoprocessing tool to append the new main into your geodatabase.  In ArcMap the new sewer main looks and acts like a feature class in GIS because of the configuration file that bridges your GIS data model and CAD standards.  So for the designer the workflow is to draft the new main on the correct level and populate the correct entity information for the new main (diameter, material, etc).

Really, the idea behind ArcGIS for AutoCAD is make it easier for CAD designers to access data from ArcGIS and more easily pass data back to a utility’s enterprise GIS,  yielding a much smoother workflow in a mixed GIS & CAD environment

So, give the ArcGIS for AutoCAD template a try and let us know what you think.

Posted in Water Utilities | Tagged , , , , , , , , , , , , , | Leave a comment

Getting Started with GIS for Small Utilities

A few days ago I was talking to a manager at a small combined water and wastewater utility who wanted to know with a limited budget, how can they implement GIS and where should they get started?

That’s a question we often get.  No doubt, most large and small utilities implement their GIS as a series of ongoing projects.  Most of the time utilities set aside a yearly budget for GIS to fund things like data creation, software purchase & maintenance, hardware and training.  Often small utilities will focus their data creation efforts on creating a few new layers a year.  This holds true whether they do the data creation themselves or hire a consultant to create their data.

The starting place for the majority of water, wastewater and storm water utility GIS projects is asset management.  This means both using your GIS as the system of record to store your assets information and then using that asset information to make decisions about when is the optimal time to repair rehabilitate or replace an asset.  If you are implementing an Enterprise Asset Management System (EAM) than both your GIS and your work orders are part of a system of record for your assets.

So if you are a small utility, what is an example of a high level process to get started with GIS?

  • 1. Download the ESRI Simplified Water Sewer Datamodel http://support.esri.com/index.cfm?fa=downloads.dataModels.filteredGateway&dmid=16 A data model is the structure (GIS people call this a schema) of how to store data in GIS. ESRI with a consortium of utilities and consultants has identified the best practices for storing water, wastewater and stormwater data in GIS and posted them on our website as data models you can download for free. The ESRI W/WW & SW data models were the first freely available data models for these industries and have become the de facto industry standard for how to store these types of utility data in a GIS. By using these data models as a starting place for your GIS you will save money and time and will not be reinventing the wheel.
  • 2. Compare the assets you own with the layers (GIS people call these feature classes) in the data model. Also make sure that data model can hold the descriptive information you need about each of your assets (GIS people call this attributes). So in the sewer data model you’d have a feature class for gravity sewer mains that would have attributes such as diameter, material and installation date. The data models have been in use for years and have most of the common assets in use at W/WW utilities, so more than likely you’d identify features in the data model that your utility does not own and remove them and add some attributes that your utility wants to track.
  • 3. Modify the data model you download to have the exact set of asset and attributes that you’ve identified your utility needs in your GIS. You may want to have a consultant do steps 1 to 3 for you or with appropriate training before attempting this task you can do this in house. Some utilities choose to have a consultant do a data model workshop where all of asset management stakeholders (the people in the utility that will use the information in the GIS) identify what specific information they need. Engaging all of the asset management stakeholders in modifying the data model is good investment for utilities.
  • 4. Get base mapping data – Base mapping data is data that you want to lay your utility asset data on top of in a map. There is a tremendous amount of free GIS data you can get from places like the cities or counties your utility serves. ESRI also provides free base map content through ArcGIS online.
  • 5. Identify data sources that you want to use to populate your GIS – such as hard copy maps, construction plans, as-builts, field maps, CAD files, data from spread sheets, etc. If you don’t have any good source data to start with then you may want to consider doing GPS data collection
  • 6. Do a pilot – Start to put data into your asset data into the data model. If you are doing wastewater assets, select part of your collector system (such as one sewer shed). At the end of creating pilot data use your GIS to make some maps and do some analysis – ask the questions you need answered to make better asset management decisions of your new data. Doing a pilot makes sure that any changes you made to the data model suit your needs and also helps you become comfortable with GIS. Identify any lessons learned through the pilot and then modify your approach if needed. Also identify some steps to do quality control of your new GIS data.
  • 7. After the pilot keep populating your asset data in GIS and maintaining the asset data you’ve already created. Now you have a production GIS for asset management. If you are a small utility with limited budget you may want to break your GIS up into phases – such as doing 1 sewer shed each year or doing your water distribution system in the 1st year and then your wastewater system the next year.

By investing in training or hiring someone with GIS knowledge you can achieve these tasks yourself.  Also many small utilities use an ESRI business partner to help them get started with their GIS.   Understanding the steps above will help you better engage with a GIS consultant.

While asset management has been the traditional starting place of W/WW & Stormwater GIS, we’ve seen an interesting trend emerge over the past few years where some utilities have made vehicle routing or a management dashboard their first GIS project.  By starting with something like vehicle routing, water utilities can have a quick win with GIS (who can argue with the benefit of fuel savings & reducing vehicle miles driven) and become comfortable with GIS as technology before moving forward with getting your assets into GIS.

As always, if you have anything to share on this topic please post a comment or send an email to: ArcGISTeamWater@esri.com

Posted in Water Utilities | Tagged , , , , , , , , , | Leave a comment

Water Utilities Templates Updated

The ESRI Water Utilities Team posted updates to all three templates in the Gallery today.    In summary, these updates include:

  • Updates to the Operations Feature Dataset in the Sample.gdb.
  • Updates to the Getting Started documents.
  • Release Notes for each Template.
  • General improvements to error handling and messaging in the Mobile Map Template.
  • Performance improvement for several tools in the Network Editing Template.
  • Bug fixes in the Network Editing and Mobile Map Templates.
  • New functionality added to the Network Editing Template.

For details on the new functions and the bug fixes, please refer to the release notes in each templates folder.
The Team will post updates from time to time to resolve known issues and add new functionality. If you have specific questions about the updates, please let us know: ArcGISTeamWater@esri.com
The ArcGIS Water Team

Posted in Water Utilities | Tagged , , , , , , , , | Leave a comment

Building and Maintaining Water Utility Geodatabases – Part 3

 Copy and Paste

You probably want to bring in some of your datasets and merge them with the projected schema. Copy and paste is the easiest way to do this, but you may get some error messages about differences in the source/target spatial reference. For example, when you try to copy and paste in ArcCatalog you may get messages like this:

This is due to some subtle differences in the properties of the target feature dataset/spatial reference and should not be a concern. The best way to handle this is to use the Import Feature Class tool in ArcCatalog:

You can also use the equivalent Geoprocessing tool: “Feature Class To Feature Class”.


Geoprocessing and Data Interoperability

In ArcGIS, Geoprocessing (GP) tools and scripts can be used to do the data manipulation and loading tasks. At a high level, the process involves using GP to massage the source data until it matches the data model of the target geodatabase, and then using commands like Append to get the features into your target geodatabase. This works well for most data loading situations, especially if you using Python or other scripting tools for automation. Once you figure out the pattern you can copy/paste scripts and blocks of code and it is generally easier to manage than ModelBuilder Models for data loading.


Another option is the ArcGIS Data Interoperability Extension. This extension provides a visual workbench to connect source and target datasets, and has a useful set of tools called “transformers” that can be used to perform calculations between source and target (for example, LifecycleStatus should be a new field called ACTIVEFLAG and LifecycleStatus=”Active” should be ACTIVEFLAG=”1″). This approach is preferred by many specialist users but does have an associated cost and learning curve.


Simple Example for wFitting, similar to the Python script for wCasing.


Portion of more complex Spatial ETL tool example for wMain


Sample Tools

Attached to this post are sample tools used to Load Data, Creating Reporting Layers and Create HTML Inventory Reports from your Geodatabase.  These tools are a working example of how to use Geoprocessing tools to build a water utility database. These tools can be used to build part of a geodatabase that matches the Fort Pierce template data model. The tools are designed to be used by GIS specialists building GIS Servers. Using this sample is easy, but implementing these tools on your project can be a large project effort. This template is designed to help you get started and to show you how we loaded the Fort Pierce data.  The basic principle is to “cook” or prepare feature classes to simplify application development and improve the performance and scalability of your applications. As an initial step, we suggest you watch the online video named How to Load Data into the Template Geodatabase and How to Build Reporting Layers found on the Water Utility Resource Center. Then, you can follow the instructions below to install and use the template on your own.


Posted in Uncategorized | Tagged , , , , , , , , , , , | 3 Comments

Building and Maintaining Water Utility Geodatabases – Part 1

Part 1: Explanation of the Sample and Template GDB

The Water Utility Templates provide sample datasets and also a template Geodatabase for your use. There are 4 main parts to the Template Geodatabase:

    ReferenceData – Landbase data typically acquired from other organizations

  • WaterDistribution – Water Utility Network Assets
  • PlanningAndOperations – Long Range Planning and Utility Administrative/Engineering Area Boundaries
  • Field Operations – Feature classes include redlines/markups and workorders assigned to field crews
    We would like to once again thank the Fort Pierce Utilities Authority and St Lucie County, Florida for allowing us to include a sample of their data in the templates. You should be aware that we took a copy of the data at a point in time and built additional content on top of those datasets. As a result, the data in the templates has been significantly altered from the original database and also does not reflect real conditions in Fort Pierce Florida. That will be obvious to many readers of this blog, but we want to clarify that for people who are just getting started.


    The Sample.gdb is quite similar to the Template.mdb, but it has more content in the ReferenceData feature dataset. The reason for the difference is that most water utilities do not control the landbase data they use, and we expect that (like Fort Pierce) most of the data loading from partners will essentially be a copy and paste into the target geodatabase. As a result, we only included a few ReferenceData feature classes in the Template.mdb, and depending on your system it is ok to remove those feature classes if you have something different to use.
    Below is a detailed explanation of each Feature Dataset in the Sample/Template geodatabase.


    This feature dataset contains the water network feature classes. ESRI has built template water network data models for about 10 years and most water utility projects have used those designs as a starting point for implementation. This design should look familiar to most longtime users, but there are some significant design changes here that are described briefly later in this document.
    To load data into these feature classes, you should first drop the geometric network. Data loading tools will run faster without the network, and you will also be able to use all of the data loading tools available in ArcGIS. Once you have all of the data loaded you can rebuild the network using the wizard in ArcCatalog.
    The loading process is a bit different than the ReferenceData because you have a target database design to load into. Of course it is ok to use a different design or to use the design you already have, but that will mean more work if you want to use the map documents provided with the templates. You might also want to use this latest design if you have existing data because it improves on previous examples.
    Generally speaking you will need to do more than copy/paste your water network features into a target geodatabase. You will likely need to calculate new values, combine or split datasets into new feature classes, and you should also have a plan for doing QA/QC on the results of the data loading process. This can be a time consuming exercise that might be more practical to contract out to an organization that specializes in this type of work, and even for simple examples many weeks of work can be involved in getting existing data loaded and validated.


    This feature dataset contains administrative and planning content for water utilities. The reason for putting these features in their own feature dataset is that they are edited on a different cycle than the asset data and they may have different permissions/editors.
    The simple part of this feature dataset is the feature classes for EngineeringGrid, Map Sheet boundaries, and other administrative areas for water utilities. This part of the database also contains planned replacement mains – wReplacementProject (Capital Improvement Projects) and also proposed mains – wProposedMain (New Development).
    This feature dataset also contains content that will be new to most users. What we built in the Fort Pierce database was a set of reporting layers that allow us to organize and view asset inventory, consumption, asset condition, and other data in the context of operational management units like map sheets, districts, and political jurisdictions.

    These feature classes were created by spatially joining/intersecting the water network features to sets of polygons like the EngineeringGrid example shown above, and then further calculations for leaks/mile, consumption, and inventory information were performed.
    There are 2 main benefits we see from this work:

  • 1. The ability to provide more granular reporting. For example, consumption and number of valves by map sheet or smaller area rather than company-wide reports.
    2. Applications such as the Operations Dashboard require that we prepare data so that rather than users running general queries on the data they can just click a few times and get the information they are looking for. As a general principle, we want to “cook” the answers to the common questions to improve performance and reduce load on web servers.


    Last but not least the FieldOperations feature dataset contains a number of feature classes that are used in the Mobile and Dashboard Water Utility Templates. These include leaks, redlines, and workorders. While the Mobile Map example only uses the redlines capability in the database, the design includes the ability to manage workorders and inspection information to/from the field as well.
    The one thing people notice right away is that workorders and inspections in this data model are Point feature classes. Yes, we know that most workorder/CMMS systems do not store point features because they are tabular systems, but part of simplifying the data/application for field users is finding the geographic location for work. In addition, ArcGIS Mobile works with simple feature classes so this approach works well from an implementation standpoint.
    The content for this feature dataset will be more dynamic, and will likely only contain assigned/current work that needs to be pushed to/from the field. For example, today’s work can be loaded into the mobile feature classes and updates can be pushed to the field. Similarly, as work happens in the field, data can be pushed back into the office through a mobile data service.
    Ultimately this approach requires some back-end work to get the data from one or more enterprise systems to the field application, and then from the field application back to the enterprise systems. This is not particularly difficult work, but the data loading and management activities will be different than the other feature datasets in the template geodatabase.

    Data Model Notes

    The data model available with this template has some changes from those data models that reflect the evolution of implementation models for geodatabases. These changes include:

    • Uppercase, short field names to ensure that field names are not altered/truncated when moved between different database platforms (and yes shapefiles are still being used for data exchange)
    • Longer, more meaningful alias names on fields
    • Descriptions on fields and feature classes (FGDC stored as FGDC metadata)


    Feature class aliases are plural but table names are singular. When you add a feature class to ArcMap you will get a layer with the plural name (the convention GIS users prefer), but it also works for database people that insist on singular names for tables.



    • LifecycleStatus field replaced by ACTIVEFLAG. Previous designs had proposed, active and abandoned features in the same network feature classes. Over time most utilities have separated proposed and abandoned features into different feature classes so they are not accidentally included in network traces or asset inventory reports. Once we added those feature classes to this design, Lifecycle status just became a flag to indicate if the features are active or not. For example, there are temporary/seasonal services that are only used for part of the year, and there are other assets in the ground that are not active because they are still under construction or are temporarily inactive.
    • OWNEDBY/MANAGEDBY fields indicate the owner of each asset and also the maintenance responsibility. While most domains/list of permitted values in this data model are strings, these domains are integers. The plan here is that “Company owned” assets that should be counted for inventory purposes have a value > 0 and things that should not be counted have a value < 0. In many cases there are multiple companies involved in county/regional scale systems and this strategy will make asset reporting simpler. Of course in those situations people should extend the list/domain of values for their specific situation.



      • You may notice that we also dropped AdministrativeArea and OperatingArea fields because these are better managed as part of the reporting layers we built for the PlanningAndOperations data – which is described in the next section of this document.
        More Information

        In each of the Templates, you will see the sample geodatabase and the template geodatabase in the “Maps and GDBs” folder. If you look in the subfolder “Documentation” there are html documents that provide reports for the geodatabase and map documents.
        These documents will help you to understand the details of the geodatabases and maps, and also help you to plan out how to load data into your Geodatabase and make changes to map documents to work with your data.
        Most projects will start with a source-target matrix spreadsheet that describes the available data and the target datasets for their new Geodatabase. This is a good place to start and it will help you to assess the suitability of the template design as well as the level of effort required to build your Geodatabase.

        Again, this can be a large part of your project, so keep in mind that ESRI and Business Partners are here to help you if you need us. Network with your peers to get their recommendations on who to work with, or email us at ArcGISTeamWater@esri.com and we can get you in touch with someone local to help you to get started.


    Posted in Water Utilities | Tagged , , , , | 1 Comment