We just posted two more updated templates, The Water Utilities Operations Dashboard and the Water Utilities Customer Interaction(formerly Citizen Service) template. You may have noticed we changed the names of the templates slightly. We switched to Water Utilities because these templates now cover more than just the Water Distribution network.
In the new version of the dashboard, you will find an updated basemap document with improved cartography. All the operational map documents have been updated to include layers for sewer and stormwater. You will also see a new set of widgets, some configured for the new data and some that were included in the most recent release of the sample flex viewer. Take some time and explore the new widgets and give us your feedback. We’re really happy about how user feedback is shaping this template into a true utility dashboard.
The Citizen Service template, now called the Customer Interaction template, underwent a big overhaul. The first release of this template was focused on getting information from the public. In this release, we wanted to expand how and what information can be captured. In the submit request web page, you can now overlay a map service from your utility. A user can click on an asset in that service and use the selected asset to power the request. The selected assets ID is silently submitted with the request, allowing you better identify the asset the request is tied to.
Not only did we want to provide a better way of capturing information, but we wanted to help you share information with your customers. There is a new web page allowing you to do just that. You can list any layers that you want to share with the public in the configuration file. We included two different configurations of this web page with the template. One that share main breaks, out of service hydrants and location of capital projects, the other is used to share boil water notices. The web page can also be used to summarize information by area and then display that to the public, so you can give the public a high level view of information by an operating district or administrative area. As they look closer, the overview will fade away and have access to the detailed feature locations.
Again, we are very happy and pleased with being able to roll out these enhancements. Which, came from all of you. So, please let us know what you like, do not like, what enhancement requests you may have, etc. You feedback drives the development of these. Thanks
This week and the next, we will be posting updates to the templates on the Water Utilities Resource Center. Our focus was to expand the templates to include data, cartography and examples for sewer and storm water.
The next release of the templates will have an expanded and updated data model. There are now feature dataset for both the Sewer System and Storm Collection System. We have included these datasets in each the of map documents with sample cartography, scale dependency, label expressions, etc.
We also restructure the Operations and Planning datasets. All operational data, whether it be data for the field or the office, is now in the Operations Dataset. This dataset has been expanded to include layers to support typical activities for Sewer and Storm data maintenance. The Planning dataset is now only used to store and manage the reporting layers. We have also included the results from the CIP template, both decision support results and the CIP project areas, in the core information model. We did this so you can see how storing when you store CIP data in your utilities authoritative data repository in GIS, your analytical results and new CIP projects are available for publication to browser based and mobile GIS applications.
Since many public works departments also operate water or wastewater utilities, we’ve decided that the public works resource center and the water utility resource center should use the same sample data when possible. So you’ll also notice in the newer template sample data some public works feature classes like roads and facilities. We wanted to leave this dataset in the download to show how one Geodatabase, a central source of information, can support many different divisions or departments in a municipality and to show that these templates can be easily expanded to support different or other datasets.
At the time of this blog, we have already posted the first two updated templates, the Water Utilities Mobile Map template and the Water Utilities Network Editing template. These templates have been upgraded and improved to handle the changes to the data model mentioned above. You will see new functions and workflows built around the sewer and storm datasets. Below I will highlight some of the new functions in each template.
In the Water Utilities Network Editing Template, you will find many new improvements and enhancements. Most of these changes were a direct result of your requests. First you will notice that we split up the Attribute Assistant and the ArcMap Toolbars into 2 separate installs. This makes it easier for us to make future improvements and roll them out faster and also allows you to install just one of the components. We heard from a few utilities that had built their own editing toolbar previously that they just wanted the attribute assistant.
When you open ArcMap, you will now find two toolbars. We split the tools into reporting/tracing tools and into editing tools. If you want more details, review the release notes, or you can click shift +F1 on top of any of the tools on the toolbar…yes per your suggestions, we included compiled help for each of the tools!!! The new reporting/trace toolbar has commonly used tracing functions. You can perform an upstream trace, downstream trace, or isolation trace, by just the click of your mouse. There is also an option to Export to Excel the selected feature, or load the selected features into the ID Box.
You’ll also have notice a new table, GenerateID, in the GDB in the updated data model. This table is used to support a new option in the Attribute Assistant, GenerateID. This new option allows you to specify a column in the GenerateID table to use as the ID index. Yup, you can generate unique ID right in ArcMap using whatever incrementing scheme you want. The tool uses the value, combines it with a prefix you specified, then increments the table. There are a few more new options in the Attribute Assistant, so check out the release notes and review the help. There is also a link for the help in the start menu, under ArcGIS Templates. Note, Windows 7 does not support .hlp out of the box, please download the fix.
The Water Utilities Mobile Map now shows both the Water dataset, and the sewer and storm data. We added a new component that lets you toggle between the different datasets. So it is easy now to just look at sewer data or water or storm, or turn all three on. This is presented to the field staff as a single, large button that make toggling between them very easy. We also improved the ID layer list. You can now filter which layers are presented to the user for Identification, making it easier to navigate the drop down list. You will also see the list expands when you click it, again, making it easier for the field personal to select a value. This new version also includes a module to show how to record new data, such as inspections, leak locations, service request, etc. This inspection module can linked to a source asset. Say you are doing a fire hydrant inspection. When you tap or click the hydrant, the inspection module copies information from the hydrant to the inspection record. It does this by matching field names. So it can help automate some of the information that needs to be captured, like ID. Lastly, you will see a module for workorders. This is an example of how you can work with a workorder system. This module read a feature class that stores all the work orders, filters them based on the crew name and present them to the field staff. The workorder module is linked to the activity module, so by opening a workorder, it starts an inspection.
We’re very happy with these new releases, but we’re already looking forward to rolling out more enhancements. With the expanded tools, symbology, data schema and workflows into Sewer and Storm, you now have a starting point for all assets at a water department, sewer utility or public works department.
Please keep in mind, these enhancements came directly from your requests and feedback about the templates, so please keep them coming!
ArcGIS Team Water
In the upcoming months, we will try to pull out some key pieces from 9.4 and start discussing how the templates will take advantage of the new functionality and how they will affect the water community. Our first post, we will take a look at ArcGIS Mobile and the Mobile Map Template that was based on this platform.
At 9.3 and 9.3.1, the ArcGIS Mobile platform included a SDK or developer kit and an out of the box application for Windows Mobile 5/6 handheld devices. We used this SDK for the Mobile Map template that you can download from the water resource center. At 9.4, ArcGIS Mobile is also going to include an out of the box tablet based application. This application has been designed for the field personal using a touch base PC. So it is very easy to navigate and interact with. This application is also extendable, so you can use the base application and add extensions that provide custom functions or workflows.
At 9.4, we are going to release a configuration of the Out of the Box ArcGIS Mobile Tablet application with several Add-Ins focused on the field workflows for the water community. The Add-Ins will emulate the functionality in the current Mobile Map Template. They will be starting point to show you how to extend the Out of the Box mobile application to fit into your utilities workflows. The goal is that you will be able to use a core, supported mobile application, and just provide the add-ons to support your needs.
The ArcGIS Mobile framework has many new enhancements that the water community will be able to take advantage of. The most intriguing is the new supported data formats. ArcGIS Mobile now supports both Operational Layers and Basemaps. This means, you can separate the data into two different storage types.
Your operational data is the data that you interact with, so data that you search, identify, and edit. This is your water mains, sewer lines, valves, manholes, catch basins, etc. This operational data is stored in the mobile cache format, which is a representation of your geodatabase. This representation or cache stores the geometry, attributes and symbology. By caching the data on the device, it allows the field personal to work disconnected from the office, but anytime you have a connection to the office, this data can be updated, and changes made in the field can be pushed back to the office.
Your basemap is the data that helps your field personnel orient themselves, locate a particular asset or facility, and it provides a reference for the operational data. In the past, the basemap data was included in the operational data cache and typically has been larger than the operational data. This made managing the cache a lot harder.
At 9.4, the basemap data can be stored or delivered in a number of ways. One way basemap data can be delivered to your field personnel is directly from ArcGIS Server in the form of a tiled map service. This means that none of the basemap data has to be deployed to the device. ArcGIS mobile leverages the internet to retrieve the tiles and displays them for the user. Those tiles are stored on the device for your session, so once they are retrieved, that can be used again and again, until the application shuts down. This can be ArcGIS Online tile map services or map services that your organization authors. The upside here is that only the operational data, or mobile cache, has to be managed on the device. All the basemap data is provided by a map service. That map service can deliver a tremendous amount of information to the user for the area they are working in. Those tiles retrieved for the mobile worker persist for the user’s session, so once a tile is retrieved once, it saved on the device so it can be reused in that session. The downside with this approach is that a data connection is required. So you will want to look at your network coverage in your area and data fees before settling on this approach for your basemap data.
If you do not have a persistent internet connection but want to provide a large amount of basemap information on your mobile devices, there is another option at 9.4 that allows you to deliver content in a compressed format. Those same tiles that ArcGIS server is reading and delivering to the field personnel through the map service, can be copied local to the device and used just like any raster dataset. This allows you to extract out an area of interest at a series of scales and provision the device with this content. If you worked with an ArcGIS Server tiled cache in the past, you know that there can be lots of files that make up the server cache and moving this number of files around can take a long time. ArcGIS 9.4 has a new cache format called compact. This compact cache format bundles up a large number of tiles into one set of files. It significantly reduces the number of files that need to be copied and reduces the amount of disk space required. There are also geoprocessing tools that allow you to extract out a section of the cache. So you can build one large cache, covering your entire service area and pull out sub areas to reduce the amount of data that you would have to deliver to support a field application.
ArcGIS Mobile at 9.4 has many improvements and enhancements, we focused on the application and the data because we see these as important changes that the community will want to take advantage of. The new application and the supported data formats will allow you to deliverer both a better application and better maps to you field users. With an out of the box application that is extendable, you can focus on the workflows for the field personal and simplifying them with custom Add-ins without developing an entire application. The new data formats will allow the field to use better basemaps and reduce the data that needs to be managed on the device.
Whether you are implementing GIS in your water, wastewater or stormwater utility and creating a data model for the first time or you are updating your existing GIS datamodel, you will no doubt ask yourself this question –
How should I model my utility’s asset in a geometric network and why should I use a geometric network? You should model a Geometric networks can enable utility system tracing, error checking, and better productivity while editing.
But how should I build it. That one simple question spawns many sub questions. Should I use complex edges, what edge to junction or edge to edge rules should I implement? What are weights? Should I worry about cardinality? We get these questions or have this conversation all the time with utilities and partners.
Below, you will find information and some guidance to help you answer these questions. Also, we always recommend that you read through ESRI’s webhelp as a starting point on geometric on networks:
First, you will need to create your network. When creating your network, you have a few options. The most important are choosing which layers participate in the geometric network and what layers, if any, are sources or sinks.
So what layers should you include in your geometric network? Keep in mind that the geometric network should encapsulate how your distribution or collection systems actually operate. So include only layers that participate in the logic network – or to think of it another way the layers that include assets that determine how your collection or distribution system function.
These typically are mains, valves, fittings, hydrants, laterals, virtual lines, manholes, catch basins, etc. Since the geometric network should only contain layers that affect the network, a change in geometry or information can affect analysis on that network. Data like Leaks, or SCADA sensor locations, are operational data sets. It is showing some incident on the network or some value or reading of the network. So these operational layers should be included in the operations dataset and not in the geometic network.
In order to create a geometric network, you’ll need to have a feature dataset which contains all of the feature classes for that network. You’ll also need at least the ArcEditor level of ArcGIS Desktop. When prompted to build the geometric network from existing features or an empty network or to create an empty network, you’ll typically choose the first option.
After you determine the layers that should be part of your geometric network, you need to think about how you are going to model flow. When setting sources or sinks, make sure to only set one of these. This is critically important, and a mistake that we see all too often. Do not set one feature as source and another as sinks. You only need one and having both in a geometric network for water, wastewater or stormwater will lead to odd network behaviors. Typically the NetworkStructure feature class or similar feature class containing a relatively small number of points is used. While creating the network you don’t specify whether it is a source or sink just that it could be one or the other. This adds a field called AncillaryRole to any feature classes you specify. Later, in ArcMap you can set the value of this field for individual features to source, sink or none. These values can be used to establish flow direction for the non-looped portion of your network. You could instead choose to use the digitized direction of your lines to establish flow direction, in this case sources and sinks are not used.
After the geometric network has been created, you need to set up the core properties of the network.
Let’s first think about complex edges versus simple edges. This is an easy one to make a decision on. In a geometric network, a simple edge must be split at every junction, so every valve, manhole, fitting, etc, on an edge, splits that edge. Complex edges allow one segment to have many junctions on top of it and it does not require that segment to be split. And by split, I mean separate records in the geodatabase. Usually, complex edges only are used on your mains and laterals for water, sewer, and storm. This allows you to model your segments by the method defined by your utility – meaning that there is no standard industry definition for what a “pipe segment” is and we often see utilities making a conscious decision for how they want to define a pipe segment and use that definition across all of their operation and business systems. We have seen people model laterals as simple edges. Typically, simple edges are used here because the lateral and the connection point (meter, service connection) is a representation of the actual meter and lateral. The representation allows you to perform tracing through the network to the meter and connect the meter id to a billing or customer system.
Next we need to determine whether to set connectivity rules. Keep this in mind – if you set just one connectivity rule in your geometric network and wish to use the validation tools, then you need to set up all the rules. This can be a complex process to figure out how all your assets connect in every situation-. Despite being complex to implement and maintain, there are certainly large benefits to using connectivity rules.
Connectivity rules allow you to model the logic connection of your network. To support all connection types, you need to make sure your datamodel will support this. Connectivity rules can leverage a geodatabase design element called subtypes. Subtype allow more complex modeling of your data so that within a feature class, features are assigned to a subtype which may have different default values, different domains, and different connectivity rules than the other subtypes within that feature class. The example template geodatabase is simplified and doesn’t include any subtypes. This means that for connectivity purposes a fitting is just a fitting not a tee, bend, or cap. Likewise for connectivity purposes a lateral line is just lateral line not a hydrant lateral or service lateral. With a more detailed design which includes subtypes you can make more extensive use of connectivity rules. That is you could have a rule that says a hydrant must connect to a hydrant lateral line and that a hydrant lateral line must connect to one hydrant. You could also specify that a hydrant feature be added by default at the free end of a hydrant lateral and that a tap fitting be placed automatically where the lateral line connects to the main. You could set up a similar set of rules for service lines and meters.
Within connectivity rules, there is an option to set cardinality. So you can go beyond just how your assets can connect and you can define how many assets can connect to each other. Let’s think about fittings again, with subtypes for fittings, you could specify that a tee fitting must connect to 3 pipes, an end cap to 1 pipe, etc. So you can see that to model proper cardinality, you need to model your data in a way to properly define the number each asset can connect to.
With a simple data model, like the data model that is included with the Water Utility Resource Center templates, you can still set a connectivity rule if desired. For instance, you can specify that wLateralLine should connect to a wMain and by default a fitting must be added.
Some of the Edit Tools in the Network Editing Template are designed to assist with automation and basic connectivity testing without the use of geodatabase connectivity rules. For example, the connectivity checker tool merely looks at feature types and makes sure they logically connect to each other. So if you want to use connectivity to enhance your editing experience, you can do so, without modeling connectivity to represent every asset’s connectivity restrictions in the geodatabase. For instance, you can model a hydrant to a lateral to a main and not worry about modeling everything, you will just have some connectivity errors when you validate, which you can choose to ignore.
Next, you will see an option for setting weights. Geometric network weights can be used in two ways. Weights can be a filter, tracing only features with matched values. This is somewhat advanced and is used primarily in telecom and electric networks. Weights can also be used to aggregate flow. This is second usage is helpful for wastewater and storm water networks where flow direction is known – that is the non-looped portion of the network. Using trace weights, we can accumulate flow upstream from a specified location. You might add a trace weight on the length of your gravity mains and laterals in order to later obtain the total length of pipes upstream from a given location. You might also add a field to your wastewater lateral points representing estimated gallons entering the system. By creating a trace weight on this field, you can summarize gallons at any point in your network using the Find Upstream Accumulation trace task on the Utility Network Analyst toolbar. If desired, the system could then store these values of accumulated flow along your network in the manholes and gravity mains. For an example of this, see the Calculate Accumulation script:
In short, weights are typically not used for most utilities. You can see that they do provide some advance functions but are not required to model and work with a geometric network.
Lastly, we like to recommend that if you are tackling some of these issues, that you take ESRI training so you can understand all of the implications of what we’ve discussed in this blog. The proper training or consulting help with creating your datamodel or implementing the geometric network will undoubtedly save you a lot of time and money when your data model is in production.