This week and the next, we will be posting updates to the templates on the Water Utilities Resource Center.  Our focus was to expand the templates to include data, cartography and examples for sewer and storm water. 

The next release of the templates will have an expanded and updated data model.  There are now feature dataset for both the Sewer System and Storm Collection System.  We have included these datasets in each the of map documents with sample cartography, scale dependency, label expressions, etc. 

We also restructure the Operations and Planning datasets.  All operational data, whether it be data for the field or the office, is now in the Operations Dataset.  This dataset has been expanded to include layers to support typical activities for Sewer and Storm data maintenance.  The Planning dataset is now only used to store and manage the reporting layers.  We have also included the results from the CIP template, both decision support results and the CIP project areas, in the core information model.  We did this so you can see how storing when you store CIP data in your utilities authoritative data repository in GIS, your analytical results and new CIP projects are available for publication to browser based and mobile GIS applications.

Since many public works departments also operate water or wastewater utilities, we've decided that the public works resource center and the water utility resource center should use the same sample data when possible.  So you'll also notice in the newer template sample data some public works feature classes like roads and facilities.  We wanted to leave this dataset in the download to show how one Geodatabase, a central source of information, can support many different divisions or departments in a municipality and to show that these templates can be easily expanded to support different or other datasets.

At the time of this blog, we have already posted the first two updated templates, the Water Utilities Mobile Map template and the Water Utilities Network Editing template.  These templates have been upgraded and improved to handle the changes to the data model mentioned above.  You will see new functions and workflows built around the sewer and storm datasets.  Below I will highlight some of the new functions in each template.

In the Water Utilities Network Editing Template, you will find many new improvements and enhancements.  Most of these changes were a direct result of your requests.  First you will notice that we split up the Attribute Assistant and the ArcMap Toolbars into 2 separate installs.  This makes it easier for us to make future improvements and roll them out faster and also allows you to install just one of the components.  We heard from a few utilities that had built their own editing toolbar previously that they just wanted the attribute assistant.

When you open ArcMap, you will now find two toolbars.  We split the tools into reporting/tracing tools and into editing tools.  If you want more details, review the release notes, or you can click shift +F1 on top of any of the tools on the toolbar...yes per your suggestions, we included compiled help for each of the tools!!!  The new reporting/trace toolbar has commonly used tracing functions.  You can perform an upstream trace, downstream trace, or isolation trace, by just the click of your mouse.   There is also an option to Export to Excel the selected feature, or load the selected features into the ID Box. 

You'll also have notice a new table, GenerateID, in the GDB in the updated data model.  This table is used to support a new option in the Attribute Assistant, GenerateID.  This new option allows you to specify a column in the GenerateID table to use as the ID index.  Yup, you can generate unique ID right in ArcMap using whatever incrementing scheme you want.  The tool uses the value, combines it with a prefix you specified, then increments the table.  There are a few more new options in the Attribute Assistant, so check out the release notes and review the help.  There is also a link for the help in the start menu, under ArcGIS Templates.  Note, Windows 7 does not support .hlp out of the box, please download the fix.

The Water Utilities Mobile Map now shows both the Water dataset, and  the sewer and storm data.  We added a new component that lets you toggle between the different datasets.  So it is easy now to just look at sewer data or water or storm, or turn all three on.  This is presented to the field staff as a single, large button that make toggling between them very easy.  We also improved the ID layer list.  You can now filter which layers are presented to the user for Identification, making it easier to navigate the drop down list.  You will also see the list expands when you click it, again, making it easier for the field personal to select a value.  This new version also includes a module to show how to record new data, such as inspections, leak locations, service request, etc.  This inspection module can linked to a source asset.  Say you are doing a fire hydrant inspection.  When you tap or click the hydrant, the inspection module copies information from the hydrant to the inspection record.  It does this by matching field names.  So it can help automate some of the information that needs to be captured, like ID.  Lastly, you will see a module for workorders.  This is an example of how you can work with a workorder system.  This module read a feature class that stores all the work orders, filters them based on the crew name and present them to the field staff.  The workorder module is linked to the activity module, so by opening a workorder, it starts an inspection. 

We're very happy with these new releases, but we're already looking forward to rolling out more enhancements.  With the expanded tools, symbology, data schema and workflows into Sewer and Storm, you now have a starting point for all assets at a water department, sewer utility or public works department.  

Please keep in mind, these enhancements came directly from your requests and feedback about the templates, so please keep them coming!

 

Thanks

ArcGIS Team Water

Date:  Tuesday, January 12, 2010   
Location:  ESRI Seminar on the web    
Time  1:00 PM - 2:00 PM  EST

Water, wastewater, and stormwater utilities are encouraged to join us for a free webinar exploring the new, freely available ArcGIS Water Distribution Capital Planning (CIP) Template.

See the New Water Distribution CIP Planning Template in Action

Learn how you can implement this free, downloadable template by watching a demonstration of the template as part of a typical water utility workflow. You’ll see how to use and customize geoprocessing models to gain better insight into how your water network is performing and what assets you should consider replacing or rehabilitating. You’ll also see how you can use your existing GIS data to estimate CIP project costs and cost out main extensions.

Learn About the ESRI Water Utility Resource Center.

Whatever your GIS experience, you’ll want to know how to download and leverage our free templates including the recently released CIP template. Plus, hear how to share knowledge with your peers and ESRI’s Water Team.

Find Out How Water Utilities Benefit from Using GIS in CIP

Discover the benefits of GIS-supported decision making for CIP and geodesign for project costing.

Get Your Questions Answered

The ESRI team that created the CIP template will answer your questions on how you can modify and customize the template to fit your utility’s needs.

 

Register - http://events.esri.com/info/index.cfm?fuseaction=showSeminar&shownumber=13106

In the upcoming months, we will try to pull out some key pieces from 9.4 and start discussing how the templates will take advantage of the new functionality and how they will affect the water community.  Our first post, we will take a look at ArcGIS Mobile and the Mobile Map Template that was based on this platform. 

At 9.3 and 9.3.1, the ArcGIS Mobile platform included a SDK or developer kit and an out of the box application for Windows Mobile 5/6 handheld devices.  We used this SDK for the Mobile Map template that you can download from the water resource center.  At 9.4, ArcGIS Mobile is also going to include an out of the box tablet based application.  This application has been designed for the field personal using a touch base PC.  So it is very easy to navigate and interact with.  This application is also extendable, so you can use the base application and add extensions that provide custom functions or workflows.
 
At 9.4, we are going to release a configuration of the Out of the Box ArcGIS Mobile Tablet application with several Add-Ins focused on the field workflows for the water community.  The Add-Ins will emulate the functionality in the current Mobile Map Template. They will be starting point to show you how to extend the Out of the Box mobile application to fit into your utilities workflows.  The goal is that you will be able to use a core, supported mobile application, and just provide the add-ons to support your needs. 

The ArcGIS Mobile framework has many new enhancements that the water community will be able to take advantage of.  The most intriguing is the new supported data formats.  ArcGIS Mobile now supports both Operational Layers and Basemaps.  This means, you can separate the data into two different storage types. 

Your operational data is the data that you interact with, so data that you search, identify, and edit.  This is your water mains, sewer lines, valves, manholes, catch basins, etc.  This operational data is stored in the mobile cache format, which is a representation of your geodatabase.  This representation or cache stores the geometry, attributes and symbology.  By caching the data on the device, it allows the field personal to work disconnected from the office, but anytime you have a connection to the office, this data can be updated, and changes made in the field can be pushed back to the office. 

Your basemap is the data that helps your field personnel orient themselves, locate a particular asset or facility, and it provides a reference for the operational data.  In the past, the basemap data was included in the operational data cache and typically has been larger than the operational data.  This made managing the cache a lot harder. 

At 9.4, the basemap data can be stored or delivered in a number of ways.  One way basemap data can be delivered to your field personnel is directly from ArcGIS Server in the form of a tiled map service.  This means that none of the basemap data has to be deployed to the device.  ArcGIS mobile leverages the internet to retrieve the tiles and displays them for the user.  Those tiles are stored on the device for your session, so once they are retrieved, that can be used again and again, until the application shuts down.  This can be ArcGIS Online tile map services or map services that your organization authors.  The upside here is that only the operational data, or mobile cache, has to be managed on the device.  All the basemap data is provided by a map service.  That map service can deliver a tremendous amount of information to the user for the area they are working in.  Those tiles retrieved for the mobile worker persist for the user’s session, so once a tile is retrieved once, it saved on the device so it can be reused in that session.  The downside with this approach is that a data connection is required.  So you will want to look at your network coverage in your area and data fees before settling on this approach for your basemap data.

If you do not have a persistent internet connection but want to provide a large amount of basemap information on your mobile devices, there is another option at 9.4 that allows you to deliver content in a compressed format.  Those same tiles that ArcGIS server is reading and delivering to the field personnel through the map service, can be copied local to the device and used just like any raster dataset.  This allows you to extract out an area of interest at a series of scales and provision the device with this content.  If you worked with an ArcGIS Server tiled cache in the past, you know that there can be lots of files that make up the server cache and moving this number of files around can take a long time.  ArcGIS 9.4 has a new cache format called compact.  This compact cache format bundles up a large number of tiles into one set of files.  It significantly reduces the number of files that need to be copied and reduces the amount of disk space required.  There are also geoprocessing tools that allow you to extract out a section of the cache.  So you can build one large cache, covering your entire service area and pull out sub areas to reduce the amount of data that you would have to deliver to support a field application.

ArcGIS Mobile at 9.4 has many improvements and enhancements, we focused on the application and the data because we see these as important changes that the community will want to take advantage of.  The new application and the supported data formats will allow you to deliverer both a better application and better maps to you field users.  With an out of the box application that is extendable, you can focus on the workflows for the field personal and simplifying them with custom Add-ins without developing an entire application.  The new data formats will allow the field to use better basemaps and reduce the data that needs to be managed on the device. 

We recently completed our first revisions to the Training Plan for Water, Wastewater and Stormwater Utilities that we initially released in early November. 

We’ve made some minor content changes based on feedback from water utilities, done some formatting clean up and also significantly reduced the file size by optimizing the graphics in the document.  So it should be much easier to download and email the plan now.  You can download the updated Training Plan below.

Over the past few years, we've seen water and wastewater utilities increasingly recognize that they can leverage GIS to reduce fuel consumption in their fleet vehicles and more efficiently carry out their work through route optimization. 

No doubt the increased interest in routing solutions for water utilities is driven by 3 factors, first that regulated water and wastewater utilities have a fixed rate structure which leads to rigid budgets.  When the cost of things like fuel, water treatment chemicals, electricity, raw construction materials, etc. fluctuate than a utility has to look at its entire operations for place where cost can be reduced to keep budgets balanced. 

The second factor, which arguable was the wake up call to water and wastewater utilities, was the fuel price spike around 2 years ago.  When fuel prices were spiking, ESRI had an exponential rise in interested from water and wastewater utilities to optimize their routing.  That interest has not subsided with the decline in gas prices.  Many forward thinking utilities are using the economic pullback to prepare for when full prices do rise again in the future. 

The third factor is that water utilities view themselves as stewards of the environment.  The quality of our source water supplies is directly linked to the health of our environment, and many water utilities are taking proactive steps to be more environmental friendly.  We've heard a lot of talk about the water-energy nexus recently and also about carbon footprints.  Water utilities can reduce their overall energy usage (in the form of less fossil fuel usage) through fleet optimization and also can reduce their carbon footprint.

From discussions with ESRI's water utility customers, we increasingly hear about the need to optimize vehicle routing for maintenance activities for fixed meter reading routes and for maintenance activities.  For the purpose of our discussion, we'll lump workorders, off cycle final meter reads, customer service visits, in person billing dispute resolutions and emergency routing together as ad-hoc routing.

Whatever type of routing you are using,  spatially, descriptively and temporarily accurate GIS data about the location of your assets and your customer locations (premise locations in particular), are critically important.  More accurate destination information will yield better routing.  We've seen a lot of technology demonstrations for utility routing, and one thing we've seen lacking in many utility routing solutions is the ability to route to an asset.  The ability to route to an asset is often missing when routing solutions intended for the general public are proposed to utilities.  For example, if a utility crew needs to turn off valve number V-2421 during an emergency, they need to be routed right to valve itself, not the nearest property with a valid address near the valve.

For utilities, route optimization isn't just about the fastest way to get from point A to B to C.  It's also about optimizing the sequence of how you deliver your work.  Meaning that it to be truly beneficial a route optimization solution needs to be able to do things like honor time windows, handle routing and work sequencing for multiple crews some of which have specialty equipment or knowledge, allow you to add more stops and reroute all of your field vehicle on the fly and be able to leverage you existing GIS asset and customer premise locations.

Meter Reading Routing

First, let's examine fixed routing for meter reading.  If your utility currently uses meters that require either proximity (drive by meter reading) or premise visits (manual reads), you've most likely created a fixed set of meter reading routes.  If your utility has to visit a customer premise for routing meter reads, than your meter reading route probably include a mix of vehicle route and then on foot routes (this require multimodal routing). 

We often hear from utilities that want to either optimize their existing routes because they are outdated or they want to establish formal routes for the first time.  From experience, we view meter reading routing as a specialty application of GIS and we suggest that you work an ESRI business partner such as Routesmart who understand how to deploy ESRI technology to overcome some of the special challenges of both drive by and manual meter reading route optimization.

Ad-hoc Routing

I use the term "ad-hoc routing" to describe routing for maintenance activities, emergency activities, customer service and bill resolution done on the customer premise, final meter reads that are off the normal meter reading cycle and inspections.  I lump these activities together into the category of ad-hoc routing because these are not rigid routes like fixed meter reading routes. 

Why do I consider maintenance routing to be ad-hoc routing when some of it is planned well in advance?  Because even though you plan for maintenance proactively and track that in your CMMS or workorder system, if a utility sequences the work to be done, assigns to a crew and creates a route that is often done the day before or the day of the actual work happening.  Utilities often mix in proactive work with reactive work (you didn't plan ahead to do that task) into a crews daily work, so this is really ad-hoc routing.  It's done the day of or day before doing the work and you may also need to reshuffle this on the fly based on the events of the day.

ESRI has a powerful core technology solution (of course this can also be extended by our business partners) for routing.  Our core solution is ArcLogistics.  

A few things to keep in mind about route optimization

We often get asked how to quantify the return on investment (ROI) for route optimization.  ESRI has recently released an ArcLogistics Cost Saving Calculator that you can plug in variable from your utility to estimate the ROI for ArcLogistics - http://roi.esri.com/costsavings2009/index.cfm

If you are selecting a new workorder, CMMS or EAM system think about how this will integrate with a GIS to enable route optimization.  Your workoder system is where you will create and track your proactive and reactive maintenance activities, but when you allocate crews and dispatch work route optimization will help you become more efficient. 

Like geocoding, route optimization relies on an underlying dataset to use for route generation.  Make sure that if you are routing, you have the appropriate dataset with the accuracy level and routing capabilities that fit the business needs of your utilities.  For example some datasets for routing are able to take into left hand turn restrictions, underpass clearances and road weight limits, those can be very important factors if you are moving large or heavy equipment around. 

Think about who will be generating the routes.  Do they need to be automatically generated, will someone create routes in a desktop application or will many people need to create routes (for themselves or others) with a web based or mobile application?  Also think about whether field crews should be empowered to route themselves, that is really a business decision at a utility.  In actuality, route optimization should be available to any system or employee that needs to optimize routes.  So it is really part of enterprise GIS at water utilities.

Understand how you will share routes once they are created.  Do you give field crews their daily routes and sequenced work orders on paper print outs or do you have computer in the field that you can use a mobile application like the ArcGIS Mobile or ArcLogistics Navigator to push routes to and enable turn by turn navigation.  When looking at a route optimization solution, you should assess how you can disseminate the routes and use them around your utility.

Feel free to comment or share any experience you've had with route optimization at your utility.

We’ve decided to repeat our webcast that explored the Water Utility Resource Center and gave an overview of ESRI Enterprise License Agreements for water, wastewater and stormwater utilities.  When we offered this webcast last month we had to 2 completely full sessions and hand a good number of people registered on a waiting list.  So we thought it would be best if we gave everyone another opportunity to participate in these webcasts.

On the webcast we'll briefly touch on the business drivers for water utility GIS and then demonstrate the templates currently available on the Water Utility Resource Center.  We'll than explore the ELA as an licensing mechanism for water utilities.

We'll also have time to answer your questions at the end of the webcast.  We had some excellent discussions about water utility GIS during our November webcasts, so we encourage you to bring your questions.

You can sign up for the December 2^nd webcast here: http://events.esri.com/info/index.cfm?fuseaction=showSeminar&shownumber=13107

We've had a lot of great comments from the water utility GIS community saying that we should do water utility GIS focused webcasts more often.  So starting in January 2010, we are also planning on doing a series of monthly webcasts focused on water, wastewater and stormwater GIS.  We tentatively planning on a webcast focused on ArcGIS Water Distribution Capital Planning template in January.  We'll post more details shortly.

Whether you are implementing GIS in your water, wastewater or stormwater utility and creating a data model for the first time or you are updating your existing GIS datamodel, you will no doubt ask yourself this question -

How should I model my utility's asset in a geometric network and why should I use a geometric network?  You should model a Geometric networks can enable utility system tracing, error checking, and better productivity while editing.

But how should I build it.  That one simple question spawns many sub questions.  Should I use complex edges, what edge to junction or edge to edge rules should I implement?  What are weights?  Should I worry about cardinality?  We get these questions or have this conversation all the time with utilities and partners. 

Below, you will find information and some guidance to help you answer these questions.  Also, we always recommend that you read through ESRI's webhelp as a starting point on geometric on networks:

http://webhelp.esri.com/arcgisdesktop/9.3/body.cfm?tocVisable=1&ID=6764&TopicName=What%20is%20a%20geometric%20network?

First, you will need to create your network.  When creating your network, you have a few options.  The most important are choosing which layers participate in the geometric network and what layers, if any, are sources or sinks.

So what layers should you include in your geometric network?  Keep in mind that the geometric network should encapsulate how your distribution or collection systems actually operate.  So include only layers that participate in the logic network - or to think of it another way the layers that include assets that determine how your collection or distribution system function.

These typically are mains, valves, fittings, hydrants, laterals, virtual lines, manholes, catch basins, etc.  Since the geometric network should only contain layers that affect the network, a change in geometry or information can affect analysis on that network.  Data like Leaks, or SCADA sensor locations, are operational data sets.  It is showing some incident on the network or some value or reading of the network.  So these operational layers should be included in the operations dataset and not in the geometic network.

In order to create a geometric network, you'll need to have a feature dataset which contains all of the feature classes for that network.  You'll also need at least the ArcEditor level of ArcGIS Desktop.  When prompted to build the geometric network from existing features or  an empty network or to create an empty network, you'll typically choose the first option.

After you determine the layers that should be part of your geometric network, you need to think about how you are going to model flow.  When setting sources or sinks, make sure to only set one of these.  This is critically important, and a mistake that we see all too often.  Do not set one feature as source and another as sinks.  You only need one and having both in a geometric network for water, wastewater or stormwater will lead to odd network behaviors.  Typically the NetworkStructure feature class or similar feature class containing a relatively small number of points is used.  While creating the network you don't specify whether it is a source or sink just that it could be one or the other.  This adds a field called AncillaryRole to any feature classes you specify.   Later, in ArcMap you can set the value of this field for individual features to source, sink or none.  These values can be used to establish flow direction for the non-looped portion of your network.  You could instead choose to use the digitized direction of your lines to establish flow direction, in this case sources and sinks are not used.  

After the geometric network has been created, you need to set up the core properties of the network. 

Let's first think about complex edges versus simple edges.  This is an easy one to make a decision on. In a geometric network, a simple edge must be split at every junction, so every valve, manhole, fitting, etc, on an edge, splits that edge.  Complex edges allow one segment to have many junctions on top of it and it does not require that segment to be split.  And by split, I mean separate records in the geodatabase.  Usually, complex edges only are used on your mains and laterals for water, sewer, and storm.  This allows you to model your segments by the method defined by your utility - meaning that there is no standard industry definition for what a "pipe segment" is and we often see utilities making a conscious decision for how they want to define a pipe segment and use that definition across all of their operation and business systems.  We have seen people model laterals as simple edges.  Typically, simple edges are used here because the lateral and the connection point (meter, service connection)  is a representation of the actual meter and lateral.  The representation allows you to perform tracing through the network to the meter and connect the meter id to a billing or customer system. 

Next we need to determine whether to set connectivity rules.  Keep this in mind - if you set just one connectivity rule in your geometric network and wish to use the validation tools, then you need to set up all the rules.  This can be a complex process to figure out how all your assets connect in every situation-.  Despite being complex to implement and maintain, there are certainly large benefits to using connectivity rules.

Connectivity rules allow you to model the logic connection of your network.  To support all connection types, you need to make sure your datamodel will support this.  Connectivity rules can leverage a geodatabase design element called subtypes. Subtype allow more complex modeling of your data so that within a feature class, features are assigned to a subtype which may have different default values, different domains, and different connectivity rules than the other subtypes within that feature class.  The example template geodatabase is simplified and doesn't include any subtypes.  This means that for connectivity purposes a fitting is just a fitting not a tee, bend, or cap.  Likewise for connectivity purposes a lateral line is just lateral line not a hydrant lateral or service lateral.  With a more detailed design which includes subtypes you can make more extensive use of connectivity rules.  That is you could have a rule that says a hydrant must connect to a hydrant lateral line and that a hydrant lateral line must connect to one hydrant.  You could also specify that a hydrant feature be added by default at the free end of a hydrant lateral and that a tap fitting be placed automatically where the lateral line connects to the main.  You could set up a similar set of rules for service lines and meters. 

Within connectivity rules, there is an option to set cardinality.  So you can go beyond just how your assets can connect and you can define how many assets can connect to each other.  Let's think about fittings again, with subtypes for fittings, you could specify that a tee fitting must connect to 3 pipes, an end cap to 1 pipe, etc.  So you can see that to model proper cardinality, you need to model your data in a way to properly define the number each asset can connect to.

With a simple data model, like the data model that is included with the Water Utility Resource Center templates, you can still set a connectivity rule if desired.  For instance, you can specify that wLateralLine should connect to a wMain and by default a fitting must be added. 

Some of the Edit Tools in the Network Editing Template are designed to assist with automation and basic connectivity testing without the use of geodatabase connectivity rules.  For example, the connectivity checker tool merely looks at feature types and makes sure they logically connect to each other.  So if you want to use connectivity to enhance your editing experience, you can do so, without modeling connectivity to represent every asset's connectivity restrictions in the geodatabase.  For instance, you can model a hydrant to a lateral to a main and not worry about modeling everything, you will just have some connectivity errors when you validate, which you can choose to ignore.

Next, you will see an option for setting weights.  Geometric network weights can be used in two ways.  Weights can be a filter, tracing only features with matched values.  This is somewhat advanced and is used primarily in telecom and electric networks.  Weights can also be used to aggregate flow.  This is second usage is helpful for wastewater and storm water networks where flow direction is known - that is the non-looped portion of the network.  Using trace weights, we can accumulate flow upstream from a specified location.  You might add a trace weight on the length of your gravity mains and laterals in order to later obtain the total length of pipes upstream from a given location.  You might also add a field to your wastewater lateral points representing estimated gallons entering the system.  By creating a trace weight on this field, you can summarize gallons at any point in your network using the Find Upstream Accumulation trace task on the Utility Network Analyst toolbar. If desired, the system could then store these values of accumulated flow along your network in the manholes and gravity mains. For an example of this, see the Calculate Accumulation script:

http://arcscripts/details.asp?dbid=14481

In short, weights are typically not used for most utilities.  You can see that they do provide some advance functions but are not required to model and work with a geometric network.

Lastly, we like to recommend that if you are tackling some of these issues, that you take ESRI training so you can understand all of the implications of what we've discussed in this blog.  The proper training or consulting help with creating your datamodel or implementing the geometric network will undoubtedly save you a lot of time and money when your data model is in production.

One of the things we can’t stress enough is the importance of GIS training for water, wastewater and stormwater utilities.  The proper skill sets are critical to the success of any GIS implementation or operation and training is one of the ways a water utility can ensure they have the proper organizational skill sets to capitalize on their investment in GIS.

As a best practice, ESRI recommends that our water, wastewater and stormwater utility customers plan for their training needs.  This includes both short term training needs to support a GIS implementation or a specific GIS project and long term training needs to ensure that a water utility can support their enterprise GIS progression. 

Some of the typical benefits of GIS training for water utilities are faster GIS deployment, significantly reduced potential for mistakes (both GIS software use mistakes and GIS deployment mistakes), best practice enterprise architectures, better workflows and improved data QA & QC which ultimately yield a better return on your investment in GIS.  We seen many utilities that continue to develop their employee’s GIS skills reap the benefits of faster enterprise deployments that are more soundly planned for.

During the past few years, ESRI’s Training Group has created a large number of training plans for water utilities.  The plans were created through a consultative process where our training group would spend time working with a utility to ensure they had the necessary organizational skills to support their GIS plans.  Through this process ESRI identified some patterns in the training needs of water utilities.  These patterns included common GIS training classes taken by certain water utility staff roles, the relationship between size of a water utility and training needs and the sequencing of water utility GIS training.

From the experience of our training group coupled with typical GIS deployment patterns, core water utility functional areas, and best practices for water utility GIS, ESRI has now created a Generic GIS Staff Development Plan for Water Utilities.  You can download the plan from the attachment link below this post.

After you download the Generic Staff Plan, we suggest you first read the Document Purpose section.  This explains how the document is intended to be used.  Since this is a generic document, we suggest that you contact an ESRI Training Consultant to help you customize this training plan to your utility.  We’ve also created the plan to be a living document, so we’ll be updating the plan regularly as ESRI continues to expand our training offerings and the field of water, wastewater and storm GIS continues to evolve.

You can sign up now for a webcast ESRI will be giving that explores the templates on the Water Utility Resource Center and also how water, wastewater and stormwater utilities can benefit from an ESRI Enterprise License Agreement (ELA).  We’ll touch on how you can leverage the GIS best practices from the Water Utility Resource Center and then demonstration of each of the templates in action, including the newly released Water Distribution Capital Planning Template.  The webcast will also explore how water utilities are benefiting from both negotiated Enterprise License Agreements and Small-Utility Enterprise License Agreements.  At the end of the webcast we’ll answer any of your questions about the Water Utility Resource Center, the templates or ELAs.

You can sign up here:

Session 1 – November 5^th 2009 1 to 2 PM EST - http://events.esri.com/info/index.cfm?fuseaction=showSeminar&shownumber=12971

Session 2 – November 5^th 2009 3 to 4 PM EST –

http://events.esri.com/info/index.cfm?fuseaction=showSeminar&shownumber=12972

In the 8 months since the Water Utility Resource Center went live, we've gotten a lot of great feedback from our water, wastewater and stormwater users and been very encouraged by the number of times the templates have been downloaded.

Just this week the Water Network Editing Template went over 2,000 downloads, so we figured we should take a break from building templates, blogging & answering your emails to celebrate.  Of course every good celebration needs a cake, so here is ours:

We're already talking about what to do at 5,000 downloads, so far ice cream cake is edging out cupcakes...

 

Come to WEFTEC!

ESRI’s Team Water/Wastewater is going to Orlando.

This year’s conference is October 10-14 at the Orange County Convention Center. We will be in booth #3857 along with 6 of our business Partners: Azteca, ID Modeling, Trimble, ESEA, Geo Cove, and Westin. Dave Wachal from our professional services group will also be in the booth to answer all your questions. We will be featuring some great new solutions including ESRI’s Water Utilities Resource Center, wastewater models using ArcGIS Explorer, cool tools for data production, handheld GPS devices, mobile GIS, and work management solutions for utility asset management, inspection, mapping and data collection and showcasing numerous case studies. Plus don’t forget to ask us about our new Small Utilities Enterprise License Agreement (SU-ELA) to see if you qualify.

Last year’s conference broke attendance records with over 21,000 attendees and 1000 exhibitors. So be sure to take time out of your busy conference schedule to visit our other business partners on the exhibit floor including DHI, Wachs Utility Services, GBA Master Series, RJN, MWH, VUEWorks and Wallingford Software just to name a few.

Check it out at http://www.weftec.org/home.htm

WEFTEC is a great event that must not be missed. Stop by and say “hi”.
See you in Orlando!

As you may have seen, we released the Water Distribution Capital Improvement Planning (CIP) Template  a last week.  First, we wanted to say a big thank you to all of our users and business partners who helped us to refine the initial geoprocessing models and the toolset also shared their workflows for capital planning.   

We've already had a few questions about why we chose the term Capital Improvement Planning (CIP) to describe this template, since not all utilities use that term.  So when we use the term CIP, what we mean is the long term plans of a utility to manage their assets and/or to expand their system, what you may also call a "Capital Plan", "Long Term Plan" or "5 year plan".

Personally, I think the CIP Template is great example of how ESRI listens to our water utility customers and responds to their needs.  We've had numerous customers over the past few years tell us that they want to be able to leverage their asset data in GIS as well as their operational data (workorders, CIS, water quality) better to support their long term plans.  Of course, we thought that giving our customers a geographic view of all that asset and operational data was the best place for them to start.  We also heard from many of our water and wastewater customers that their long term planning has evolved from an occasional event to a continual process; because of funding issues, grant availability, coincidence with other projects that a utility could share costs with and the desire to be quick and proactive to eliminate the risk of future critical asset failures.

Also, we are excited, because the CIP template is great example of GeoDesign.  We'll be doing a blog shortly that explores the principals of GeoDesign and relates them back to the CIP template.

2 Parts of the CIP Workflow

As we dug into the CIP process, we observed 2 distinct, but related workflows happening.  The first part of the workflow was to assemble data from many sources and analyze that data to look for where projects are needed.  This part of the process is tailor made for the benefits of GIS - to use GIS as the place where different types of data are assembled together into a common view and also to use the analytical capabilities of GIS to gain better insight into the aggregated data.  Because this analysis needs to be iterative (looking at multiple data layers with different weighted criteria), an auditable process (you have to be able to defend your findings to a PUC and your ratepayers) and an automated workflow (to save time, money and resources) this is a perfect match for Geoprocessing Models in ArcGIS.

GIS Analysis for CIP Decision Making

At first we took the approach that ESRI should try and build a few geoprocessing models that all water and wastewater utilities could use to score and rate their assets by estimated remaining asset life, condition or criticality.  We figured that we could do some research, interview some of our users and figure out these geoprocessing models (our inner geography geek begged us to take this approach first).  What we quickly realized was that there isn't a silver bullet set of geoprocessing models we could build because every utility system has their own approach to long term asset management and their own priorities (KPIs, level of service they want to provide, hot button issues, fiscal condition, etc) that drive their long term planning. 

This was also a great reminder that even though we have the ability to use technology to automate a process, the human element is still critical, meaning that the more we talked with the engineers who are creating these CIP plans, the more we realized they need a better way to manipulate and process data so they could apply their engineering expertise to make decisions about capital projects.  We also noticed when talking to engineers doing capital planning, that while they were somewhat aware of the analytical capabilities of GIS, they weren't aware of the geoprocessing framework core to ArcGIS and how to use ModelBuilder to automate analysis and create a reusable toolset. 

So we decided that we need to focus our CIP template on showing the water utility community how they could benefit from automating spatial analysis with the ArcGIS geoprocessing framework by providing some generic models.  So, please keep in mind that the intent of the models we've provided in the CIP template is to show you how geoprocessing and ModelBuilder work within ArcGIS so you can create geoprocessing models that reflect how your utility wants to manage assets and plan for the long term.  Incidentally, if you want to learn more about GIS analysis, Geoprocessing or Model Builder within ArcGIS, ESRI has lots of great resource including on-line training, books and class room instruction.

Estimating Project Costs

The second part of the CIP workflow we observed was estimating CIP project costs.  Basically this workflow was estimating the cost of a project based on either replacing existing infrastructure or adding new infrastructure (main extensions, interconnections, extending service to new sub-divisions, etc).  It's important to note that all of the functionality in this part of the CIP process is core to ArcGIS and the geodatabase, all we've done is customized the application to automate and simplify this part of the workflow.  This is what we decided to call the Costing Estimating Tools.

The first step in estimating project costs is to create projects by grouping assets together into projects.  In this part of the process you are visualizing the data you brought into GIS and also the results of your analysis and then determining what assets you want to include in a project, your rehab or replacement strategy for those assets  and then saving that information.  So you are literally visualizing data in GIS (most likely working with many data layers of data, including the same feature datasets symbolized different ways) and doing some spatial and attribute queries to come up with candidate assets to include in CIP projects.

From there, assets that are in need of replacement or rehabilitation and spatially close to together are grouped in projects.  We've heard from many water utilities that without a spatial context it was a real challenge for them to group assets together into appropriate projects without and also it was a challenge for them to track and manage information about candidate assets for CIP projects throughout the CIP planning process.  Water utilities were struggling with supporting their CIP process with paper maps and tracking assets that were part of a project, including costs to replace those assets, in spreadsheets. 

So traditionally, this CIP process took a lot of staff time and also lead to uncertainty about whether utilities were actually spending their money on the most appropriate capital projects.  We also heard that utilities were struggling with how to update data when they tried to refine a large candidate list of CIP projects down to just a few to carry forward into design and that it was next to impossible to look at multiple scenarios for the same project area (assets grouping and rehab or replacement approach) because so much of this process was manual or spreadsheet driven.

We took the approach that if a utility has their assets (water distribution, wastewater collection or stormwater) in GIS, they should use their GIS asset data to group into CIP projects and then to store information about the CIP projects (like the extent and also all of the assets that are part of the project) as new data layers in GIS.  This enables a utility to create an authoritative source of data about their proposed capital projects in GIS.   So this drove us to create the Cost Estimating Tools. 

As we began to demonstrate early versions of the Cost Estimating Tools to our utility users, we got a lot of great feedback that helped us to refine the tools.  We were told that to be really useful, the tools should include the ability to either rehab or replace existing assets and to extend mains, so we programmed that functionality into the tools.  We also were told by our users that they needed to be able to compare the costs of different replacement strategies (open cut, trenchless, etc) for the same set of assets so we designed the tools to make it easy to compare the costs of use using different rehab methodologies.  Also we knew that the costing element of the tools needed to be flexible, because individual utilities favor different pipe materials which can be set as defaults and that unit costs are often specific to a utility and those can be easily configured in a simple table.

So what we wanted to do with this blog was to explain how we arrived at version 1 of the Water Distribution CIP Template.  We are very interested in your feedback so we can incorporate more useful changes in version 2.  Also we'd like to hear about any geoprocessing models that you would like to use for CIP planning.  So, please leave us feedback here - http://forums.esri.com/forums.asp?c=55&s=426#426

In the next few weeks we'll be recording a video of the Water Distributions CIP Template in action and we are also going to do a webcast in December that takes a deep dive into the CIP Template.

If you have been following us on twitter, you already know that we released the ArcGIS Water Distribution Capital Planning template (we are calling this the CIP Template for short) yesterday.  The CIP template includes a set of models to help you understand how to you can use GIS to score and rank your infrastructure and a set of tools to provide cost estimates for rehabilitating, replacing or building new infrastructure. 

 

We will film a video at the end of October that shows you in detail how these tools work, and we’ll be doing a live webcast in December to explore the CIP template in depth. So in the mean time, below is a little help with the CIP Template.

 

Models:

                We included 6 models that show different ways you can analyze your data.  To run these models, you will need to create a temporary file geodatabase and set the environmental variables for each model.  The two variables you need to set are Current Workspace (the folder that has the CapitalPlanning.gdb in it) and the Scratch Workspace (the folder that has the temporary File GDB you just created). 

 

Tools:

                The Project Cost Estimating tools use three tables in the CapitalPlanning.gdb for configuration.  These tables are shipped to work with the data in the Sample.gdb.  If you want to start changing cost or the configuration, you will need to change these tables.

                CIPDEFINITON – This tables defines which featureclass’s to cost, the fields to look at(such as Diameter and Material) and a few other parameters.

                CIPCOST – This table defines the cost for a particular asset.  When costing an asset, you can define a Strategy, like Replacement or Rehabilitate, then an action for that Strategy, like Open cut for a Replacement .  For each Strategy and Action, then you define the cost based on the fields you set up in the definition table.  So if you are looking at wMains as a layer, you are interested in the Field’s, Diameter and Material, you would select your Strategy, the Action for that Strategy, the Material(say PVC), then Diameter(say 12) and define a cost for what each foot would cost.  So by using a Strategy, Action and two filter fields, you can provide very detailed cost estimates.

                CIPREPLACEMENT – This table allows you to provide lookups for replacement.  If you are going to replace a 6” DI, you may have a rule saying that each 6” DI is going to be replaced with a 8” PVC.  This table allows you to define this replacement.  So that costing is preformed an 8” PVC, not a 6” DI.

 

Since this is our initial release of the CIP Template, we want your feedback.  So please post any questions or feedback to our forums under the Water Utilities Template section: http://forums.esri.com/forums.asp?c=55

We have had a lot of talk about Unique ID’s in the last few weeks and GUID’s have came up a few times.  We wanted to add some more information to this topic.  GUID’s or Globally Unique Identifier are essentially unique values.  These are used by the geodatabase for replication and editing in ArcGIS Mobile in a special field called Global ID.  This Global ID field is managed by ArcGIS.  When a feature is created, it is auto populated with a GUID, either in ArcGIS Desktop, ArcGIS Server or in ArcGIS Mobile.  This should not be used for a Unique ID.  These values can change if you have to reload the data at some point or if you drop the Global ID field and re-add it.  I would say that you should not use any field managed by something as a unique ID.  But there is a GUID Option.

There is also a GUID field type.  This allows you to use the GUID data type as a unique ID.   It is up to you to manage it, but there are some tools to help.  We are looking at adding an option in the Dynamic Value table so the Attribute Assistant that is part of the editing template can add a GUID for you.  I have included a VBScript function that can be used in the Field Calculator.  Please see the attach GUID.Cal file.

You can read more about GUID’s on this Wiki article. - http://en.wikipedia.org/wiki/Globally_Unique_Identifier

At first glance, that might seem like a silly question to a water, wastewater or storm water utility.  After all, how hard is it to find your customers…. they are in your service area, connected to your infrastructure and you have an address to send them bills.  But do you really accurately know where you are providing service to?

We are seeing a trend where water utilities are recognizing the importance of accurately knowing where they are providing service to (your real customer locations) and also understanding that there are many facets to accurately establishing your customer locations. 

So what do we mean by customer location and how do you store that in your GIS?

For the purpose of our discussion here, by customer locations we mean the location where you as a utility are providing service to. This is the location where you are distributing flow to in a water system or where you are accepting flow in a sanitary sewer system

There are some common approaches that we see utilities using to store customer information in their GIS.  Just like any GIS data model, you should pick an approach to store your customer locations that fits your utility’s specific needs.

For water utilities we commonly see customer location stored as a meter feature class (if you have meters) or with a feature class called customer, premise location or service location.  With a geometric network, these feature classes are snapped to lateral which are snapped to mains.  An important distinction for many utilities is that billing location of a customer, where the bill is sent to, is often different than the location you are serving that customer (premise or customer location). 

For wastewater utilities we commonly see customer locations stored with a cleanout feature class, a customer or premise feature class or if a combined water/wastewater utility then water and wastewater both may use the meter feature class from the water distribution network.

Of course if you don’t have your water or wastewater networks in GIS yet, or don’t even have a GIS, customer locations are a great starting point for building a GIS system.  It is relatively cheap to record them and you’ll immediately get high value from having those location accurately measured. 

Some common attributes we see for customer location feature classes are: unique ID, customer type, active, customer name, premise address, customer phone number.  Unique ID is should be an ID that will allow you to join your customer locations with your billing system so you can visualize consumption patterns. 

Benefit of accurately locating your customers:

Some of the ways we’ve seen water and wastewater utilities benefit from accurately knowing their customer locations are:

• Reducing non-revenue water – We’ve seen accurate customers as a critical data component to reducing non-revenue water.  A simple example of reducing non-revenue water with accurate customer locations is to create a map that shows all of your customer locations and then to look for where places (such as a buildings) that should be a customer location but are not.  Another way to use customer locations to reduce non-revenue water is to join your billing data to customer locations and visualize customer consumption by creating a thematic map of graduated symbol sizes or colors.  In this case you are looking for active customers that have abnormally low consumption and may have a defective meter.  Anecdotally we’ve heard from a number of ESRI customers that the simple actions above have made significant reductions in non-revenue water.  We’ve also seen some very sophisticated analysis using consumption data linked to customer locations and metering data to try to identify zones within a water distribution system that may a high amount of water loss due to leaks.
• Allocating demands – More accurate customer locations will yield better demand allocation for hydraulic models, especially when linked to consumption data.
• Estimating flow – For wastewater utilities, more accurate customer locations can be used to better estimate flow through being able to more precisely calculate the EDUs flowing into pipes from upstream. 
• Better Customer Service – For example you can gain better insight into how customer complaints and customer service requests track back to the actual infrastructure that they are served by.  A good example of this is water utilities that are using their customer locations in GIS to track the location of water quality complaints over a multiyear period back to common pipes or water sources that could be the cause of an issue.
• Improved routing – More accurate customer locations will yield better routes for field crews, saving fuel and time.
• Validating premise addresses in other utility systems – We often hear from utilities that while they have very good billing address data for customers in their CIS or billing system, that premise locations (stored as an address) in these systems is often wrong.  So better premise address locations generated with GIS can be used to fix bad premise locations in your CIS (you should have one system of record for premise locations, but we’ll save that discussion for another day).
• Better emergency notifications – We’ve heard a number of horror stories from utilities that have an emergency notification system that has not notified customers during an emergency because their customer locations are bad.  2 common ways to notify customers during emergencies are doing a broadcast notification (notifying all customers in a service area or a municipal boundary) or doing a target notification based on the pipes that serve a customer (just notify customers that are affected by a broken main because you know they are served by that main).  In both of these examples accurate customer locations are key to being able to perform emergency notifications.  

So how do you get your customers located accurately?

The most common way that utilities initially get their customer locations into a GIS is through geocoding their billing roster.  When geocoding any address data, the 2 critical components that determine the quality of your geocodes is the input address data and the dataset that you are geocoding against. 

We’ve heard from a lot of water utilities that the premise locations in their CIS or billing system are of dubious quality and often time there is not a lot of consistency in how the address fields for premises were used.  So while billing address data is usually of high quality (otherwise you’d never get paid) premise location is not accurately stored because it was perceived to be of less importance.  In this case, step one would be to try and fix some of the issues in the address data you are trying to geocode before you geocode the address data.  This may include trying to standardize the input address data (street abbreviations, data structure, etc).

You also want to use the best dataset available to geocode against.  Increasingly we are seeing water utilities licensing commercial dataset for geocoding.  Particularly they are choosing to license datasets that frequently updated and have the ability to geocode down to a rooftop level.

We somtimes see water and wastewater utilities use parcel centroids as the first step to establishing customer locations.  So if you can get a good dataset of parcels in your service area you can use GIS to calculate the centroid of each parcel as the first step to establishing customer locations.

Also we are increasingly speaking with water utilities that are GPSing their meter locations and curb stops during meter replacement projects or wastewater utilities that are GPSing clean outs during field data collection projects.

No matter what automated process you use to get your customers on the map initially, no doubt you’ll have to do some data clean up.  For example, you will probably have to do some manual data creation and editing to establish customer locations at commercial and industrial locations or for multi-unit housing.  Also you must develop a workflow to keep your customer location dataset in GIS up to date.