Hopefully you’ve had a chance to try out the first version of “Water Utilities Citizen Service Template” in the template gallery. We’re about to release a new update to the template that includes some great enhancements. We’ve also decided to rename the template to something more descriptive, the “Water Utilities Customer Interaction Template”. Currently we’re testing the installation and configuration instructions, but in the meantime you can try the new template here – http://waterutilitiestemplates.esri.com/CustomerInteraction/
The intent of this template is to demonstrate how a water or wastewater utility can use a web based mapping application for better customer self service. Web based self service is nothing new, many companies and governmental entities have include forms, information and applications on their websites designed to empower customers or constituents to do some form of self service. The benefits of web self service are well known and proven – cost reductions, faster response and increased customer satisfaction. Of course to make that happen you have to use the right technology and also the right workflow behind the technology.
Another way that we’ve been thinking about web based self service for water utilities is that it’s a bi-directional sharing of information. Meaning that customers may have some information they need to share with a utility and the utility may also have some information that it needs to share with customers.
The first version of this template was really focused on the customers sharing information with the utility, or more appropriately making a request of the utility for some service. For example notifying the utility of a leaking valve in the street or a hydrant that appears like it’s damaged.
The second release of this template introduces ways that utilities can use a web based map to share information better with their customer (and that’s why we changed the name). We’ll save a more detailed exploration of why and how water utilities should be sharing information back with customers and the general public for another blog. We think it’s a compelling enough topic that we want to explore that on its own.
Now let’s get back to the topic at hand, the Geographic Approach to Water Utility Customer Service. It’s a simple premise, and builds on the general concepts of the “Geographic Approach”, just about everything that a water utility owns, operates or serves has a geographic location and can be mapped. Through mapping your assets, customer and operational data you’ll gain a better understanding of how they impact each other. Service request and the work orders they generate are a key pieces of operational data that have a spatial component and should be mapped or tied back to an asset that has a location.
Part of the geographic approach for water utilities is the fact that it’s easier for human beings to understand information when it’s in the form of a map. Maps provide a richer context then information that is just textual or tabular.
So, what does the geographic approach mean for customer self service at water utilities? It means if you are deploying customer self service capabilities to your website, you should include a web based interactive map if there is a spatial component to the information being communicated.
Some examples of where a water utility could benefit from an interactive map for customers self service are:
- Service requests – A well designed and simple to use web map for service request should be less confusing to customer than a text based form. For example, we’ve seen a number of customer service request web pages that rely on some text based forms that key off an address. But most water utility assets don’t have an address, what they have is a spatial location. It’s confusing the customer to make them try and figure out the nearest address for something they are requesting service for.
- Asset status – while the thought of doing this still keeps some utilities up at night, we are seeing the start of a trend in the industry towards giving a limited view into the location and operational status of certain water utility assets to the public. A good example of this is fire hydrant operational status maps.
- Operational information – Leaks, workorders, project work areas, and service outages. This is dynamically changing information that also has a spatial component. You’ll sometimes see this type of information posted on a water utility webpage as text or maybe with a polygon on a static map in a PDF.
- KPIs – Some utilities track KPIs by spatial areas such as operational area or by city council district or the like and share this information back out with the public. You’ll often see this information presented in a table, but its spatial information and can be better presented to the public in an interactive map.
Lastly, if you’ve been looking at both the Water Utility and the Public Works Resource Centers, you may have noticed that we have the same Customer Self Service template posted on both. No, that’s not because we are lazy… it’s because any entity that needs to take service requests around assets can benefit from this template. That’s also why when you download this template you’ll find service requests for general public works as well as water, sewer and stormwater. When you configure the template you can just remove service requests that aren’t applicable to your organization and add additional ones that are.
This week and the next, we will be posting updates to the templates on the Water Utilities Resource Center. Our focus was to expand the templates to include data, cartography and examples for sewer and storm water.
The next release of the templates will have an expanded and updated data model. There are now feature dataset for both the Sewer System and Storm Collection System. We have included these datasets in each the of map documents with sample cartography, scale dependency, label expressions, etc.
We also restructure the Operations and Planning datasets. All operational data, whether it be data for the field or the office, is now in the Operations Dataset. This dataset has been expanded to include layers to support typical activities for Sewer and Storm data maintenance. The Planning dataset is now only used to store and manage the reporting layers. We have also included the results from the CIP template, both decision support results and the CIP project areas, in the core information model. We did this so you can see how storing when you store CIP data in your utilities authoritative data repository in GIS, your analytical results and new CIP projects are available for publication to browser based and mobile GIS applications.
Since many public works departments also operate water or wastewater utilities, we’ve decided that the public works resource center and the water utility resource center should use the same sample data when possible. So you’ll also notice in the newer template sample data some public works feature classes like roads and facilities. We wanted to leave this dataset in the download to show how one Geodatabase, a central source of information, can support many different divisions or departments in a municipality and to show that these templates can be easily expanded to support different or other datasets.
At the time of this blog, we have already posted the first two updated templates, the Water Utilities Mobile Map template and the Water Utilities Network Editing template. These templates have been upgraded and improved to handle the changes to the data model mentioned above. You will see new functions and workflows built around the sewer and storm datasets. Below I will highlight some of the new functions in each template.
In the Water Utilities Network Editing Template, you will find many new improvements and enhancements. Most of these changes were a direct result of your requests. First you will notice that we split up the Attribute Assistant and the ArcMap Toolbars into 2 separate installs. This makes it easier for us to make future improvements and roll them out faster and also allows you to install just one of the components. We heard from a few utilities that had built their own editing toolbar previously that they just wanted the attribute assistant.
When you open ArcMap, you will now find two toolbars. We split the tools into reporting/tracing tools and into editing tools. If you want more details, review the release notes, or you can click shift +F1 on top of any of the tools on the toolbar…yes per your suggestions, we included compiled help for each of the tools!!! The new reporting/trace toolbar has commonly used tracing functions. You can perform an upstream trace, downstream trace, or isolation trace, by just the click of your mouse. There is also an option to Export to Excel the selected feature, or load the selected features into the ID Box.
You’ll also have notice a new table, GenerateID, in the GDB in the updated data model. This table is used to support a new option in the Attribute Assistant, GenerateID. This new option allows you to specify a column in the GenerateID table to use as the ID index. Yup, you can generate unique ID right in ArcMap using whatever incrementing scheme you want. The tool uses the value, combines it with a prefix you specified, then increments the table. There are a few more new options in the Attribute Assistant, so check out the release notes and review the help. There is also a link for the help in the start menu, under ArcGIS Templates. Note, Windows 7 does not support .hlp out of the box, please download the fix.
The Water Utilities Mobile Map now shows both the Water dataset, and the sewer and storm data. We added a new component that lets you toggle between the different datasets. So it is easy now to just look at sewer data or water or storm, or turn all three on. This is presented to the field staff as a single, large button that make toggling between them very easy. We also improved the ID layer list. You can now filter which layers are presented to the user for Identification, making it easier to navigate the drop down list. You will also see the list expands when you click it, again, making it easier for the field personal to select a value. This new version also includes a module to show how to record new data, such as inspections, leak locations, service request, etc. This inspection module can linked to a source asset. Say you are doing a fire hydrant inspection. When you tap or click the hydrant, the inspection module copies information from the hydrant to the inspection record. It does this by matching field names. So it can help automate some of the information that needs to be captured, like ID. Lastly, you will see a module for workorders. This is an example of how you can work with a workorder system. This module read a feature class that stores all the work orders, filters them based on the crew name and present them to the field staff. The workorder module is linked to the activity module, so by opening a workorder, it starts an inspection.
We’re very happy with these new releases, but we’re already looking forward to rolling out more enhancements. With the expanded tools, symbology, data schema and workflows into Sewer and Storm, you now have a starting point for all assets at a water department, sewer utility or public works department.
Please keep in mind, these enhancements came directly from your requests and feedback about the templates, so please keep them coming!
ArcGIS Team Water
Date: Tuesday, January 12, 2010
Location: ESRI Seminar on the web
Time 1:00 PM – 2:00 PM EST
Water, wastewater, and stormwater utilities are encouraged to join us for a free webinar exploring the new, freely available ArcGIS Water Distribution Capital Planning (CIP) Template.
See the New Water Distribution CIP Planning Template in Action
Learn how you can implement this free, downloadable template by watching a demonstration of the template as part of a typical water utility workflow. You’ll see how to use and customize geoprocessing models to gain better insight into how your water network is performing and what assets you should consider replacing or rehabilitating. You’ll also see how you can use your existing GIS data to estimate CIP project costs and cost out main extensions.
Learn About the ESRI Water Utility Resource Center.
Whatever your GIS experience, you’ll want to know how to download and leverage our free templates including the recently released CIP template. Plus, hear how to share knowledge with your peers and ESRI’s Water Team.
Find Out How Water Utilities Benefit from Using GIS in CIP
Discover the benefits of GIS-supported decision making for CIP and geodesign for project costing.
Get Your Questions Answered
The ESRI team that created the CIP template will answer your questions on how you can modify and customize the template to fit your utility’s needs.
12/08/09–ESRI Data & Maps is a collection of preconfigured data and maps for ArcGIS. It is freely available for all ArcGIS customers as a set of DVDs or layer packages you can download from ArcGIS Online.
An update to the 2009 version is now available. For details about what’s new, see the ArcGIS Data blog.
Whether you are implementing GIS in your water, wastewater or stormwater utility and creating a data model for the first time or you are updating your existing GIS datamodel, you will no doubt ask yourself this question –
How should I model my utility’s asset in a geometric network and why should I use a geometric network? You should model a Geometric networks can enable utility system tracing, error checking, and better productivity while editing.
But how should I build it. That one simple question spawns many sub questions. Should I use complex edges, what edge to junction or edge to edge rules should I implement? What are weights? Should I worry about cardinality? We get these questions or have this conversation all the time with utilities and partners.
Below, you will find information and some guidance to help you answer these questions. Also, we always recommend that you read through ESRI’s webhelp as a starting point on geometric on networks:
First, you will need to create your network. When creating your network, you have a few options. The most important are choosing which layers participate in the geometric network and what layers, if any, are sources or sinks.
So what layers should you include in your geometric network? Keep in mind that the geometric network should encapsulate how your distribution or collection systems actually operate. So include only layers that participate in the logic network – or to think of it another way the layers that include assets that determine how your collection or distribution system function.
These typically are mains, valves, fittings, hydrants, laterals, virtual lines, manholes, catch basins, etc. Since the geometric network should only contain layers that affect the network, a change in geometry or information can affect analysis on that network. Data like Leaks, or SCADA sensor locations, are operational data sets. It is showing some incident on the network or some value or reading of the network. So these operational layers should be included in the operations dataset and not in the geometic network.
In order to create a geometric network, you’ll need to have a feature dataset which contains all of the feature classes for that network. You’ll also need at least the ArcEditor level of ArcGIS Desktop. When prompted to build the geometric network from existing features or an empty network or to create an empty network, you’ll typically choose the first option.
After you determine the layers that should be part of your geometric network, you need to think about how you are going to model flow. When setting sources or sinks, make sure to only set one of these. This is critically important, and a mistake that we see all too often. Do not set one feature as source and another as sinks. You only need one and having both in a geometric network for water, wastewater or stormwater will lead to odd network behaviors. Typically the NetworkStructure feature class or similar feature class containing a relatively small number of points is used. While creating the network you don’t specify whether it is a source or sink just that it could be one or the other. This adds a field called AncillaryRole to any feature classes you specify. Later, in ArcMap you can set the value of this field for individual features to source, sink or none. These values can be used to establish flow direction for the non-looped portion of your network. You could instead choose to use the digitized direction of your lines to establish flow direction, in this case sources and sinks are not used.
After the geometric network has been created, you need to set up the core properties of the network.
Let’s first think about complex edges versus simple edges. This is an easy one to make a decision on. In a geometric network, a simple edge must be split at every junction, so every valve, manhole, fitting, etc, on an edge, splits that edge. Complex edges allow one segment to have many junctions on top of it and it does not require that segment to be split. And by split, I mean separate records in the geodatabase. Usually, complex edges only are used on your mains and laterals for water, sewer, and storm. This allows you to model your segments by the method defined by your utility – meaning that there is no standard industry definition for what a “pipe segment” is and we often see utilities making a conscious decision for how they want to define a pipe segment and use that definition across all of their operation and business systems. We have seen people model laterals as simple edges. Typically, simple edges are used here because the lateral and the connection point (meter, service connection) is a representation of the actual meter and lateral. The representation allows you to perform tracing through the network to the meter and connect the meter id to a billing or customer system.
Next we need to determine whether to set connectivity rules. Keep this in mind – if you set just one connectivity rule in your geometric network and wish to use the validation tools, then you need to set up all the rules. This can be a complex process to figure out how all your assets connect in every situation-. Despite being complex to implement and maintain, there are certainly large benefits to using connectivity rules.
Connectivity rules allow you to model the logic connection of your network. To support all connection types, you need to make sure your datamodel will support this. Connectivity rules can leverage a geodatabase design element called subtypes. Subtype allow more complex modeling of your data so that within a feature class, features are assigned to a subtype which may have different default values, different domains, and different connectivity rules than the other subtypes within that feature class. The example template geodatabase is simplified and doesn’t include any subtypes. This means that for connectivity purposes a fitting is just a fitting not a tee, bend, or cap. Likewise for connectivity purposes a lateral line is just lateral line not a hydrant lateral or service lateral. With a more detailed design which includes subtypes you can make more extensive use of connectivity rules. That is you could have a rule that says a hydrant must connect to a hydrant lateral line and that a hydrant lateral line must connect to one hydrant. You could also specify that a hydrant feature be added by default at the free end of a hydrant lateral and that a tap fitting be placed automatically where the lateral line connects to the main. You could set up a similar set of rules for service lines and meters.
Within connectivity rules, there is an option to set cardinality. So you can go beyond just how your assets can connect and you can define how many assets can connect to each other. Let’s think about fittings again, with subtypes for fittings, you could specify that a tee fitting must connect to 3 pipes, an end cap to 1 pipe, etc. So you can see that to model proper cardinality, you need to model your data in a way to properly define the number each asset can connect to.
With a simple data model, like the data model that is included with the Water Utility Resource Center templates, you can still set a connectivity rule if desired. For instance, you can specify that wLateralLine should connect to a wMain and by default a fitting must be added.
Some of the Edit Tools in the Network Editing Template are designed to assist with automation and basic connectivity testing without the use of geodatabase connectivity rules. For example, the connectivity checker tool merely looks at feature types and makes sure they logically connect to each other. So if you want to use connectivity to enhance your editing experience, you can do so, without modeling connectivity to represent every asset’s connectivity restrictions in the geodatabase. For instance, you can model a hydrant to a lateral to a main and not worry about modeling everything, you will just have some connectivity errors when you validate, which you can choose to ignore.
Next, you will see an option for setting weights. Geometric network weights can be used in two ways. Weights can be a filter, tracing only features with matched values. This is somewhat advanced and is used primarily in telecom and electric networks. Weights can also be used to aggregate flow. This is second usage is helpful for wastewater and storm water networks where flow direction is known – that is the non-looped portion of the network. Using trace weights, we can accumulate flow upstream from a specified location. You might add a trace weight on the length of your gravity mains and laterals in order to later obtain the total length of pipes upstream from a given location. You might also add a field to your wastewater lateral points representing estimated gallons entering the system. By creating a trace weight on this field, you can summarize gallons at any point in your network using the Find Upstream Accumulation trace task on the Utility Network Analyst toolbar. If desired, the system could then store these values of accumulated flow along your network in the manholes and gravity mains. For an example of this, see the Calculate Accumulation script:
In short, weights are typically not used for most utilities. You can see that they do provide some advance functions but are not required to model and work with a geometric network.
Lastly, we like to recommend that if you are tackling some of these issues, that you take ESRI training so you can understand all of the implications of what we’ve discussed in this blog. The proper training or consulting help with creating your datamodel or implementing the geometric network will undoubtedly save you a lot of time and money when your data model is in production.
As you may have seen, we released the Water Distribution Capital Improvement Planning (CIP) Template a last week. First, we wanted to say a big thank you to all of our users and business partners who helped us to refine the initial geoprocessing models and the toolset also shared their workflows for capital planning.
We’ve already had a few questions about why we chose the term Capital Improvement Planning (CIP) to describe this template, since not all utilities use that term. So when we use the term CIP, what we mean is the long term plans of a utility to manage their assets and/or to expand their system, what you may also call a “Capital Plan”, “Long Term Plan” or “5 year plan”.
Personally, I think the CIP Template is great example of how ESRI listens to our water utility customers and responds to their needs. We’ve had numerous customers over the past few years tell us that they want to be able to leverage their asset data in GIS as well as their operational data (workorders, CIS, water quality) better to support their long term plans. Of course, we thought that giving our customers a geographic view of all that asset and operational data was the best place for them to start. We also heard from many of our water and wastewater customers that their long term planning has evolved from an occasional event to a continual process; because of funding issues, grant availability, coincidence with other projects that a utility could share costs with and the desire to be quick and proactive to eliminate the risk of future critical asset failures.
Also, we are excited, because the CIP template is great example of GeoDesign. We’ll be doing a blog shortly that explores the principals of GeoDesign and relates them back to the CIP template.
2 Parts of the CIP Workflow
As we dug into the CIP process, we observed 2 distinct, but related workflows happening. The first part of the workflow was to assemble data from many sources and analyze that data to look for where projects are needed. This part of the process is tailor made for the benefits of GIS – to use GIS as the place where different types of data are assembled together into a common view and also to use the analytical capabilities of GIS to gain better insight into the aggregated data. Because this analysis needs to be iterative (looking at multiple data layers with different weighted criteria), an auditable process (you have to be able to defend your findings to a PUC and your ratepayers) and an automated workflow (to save time, money and resources) this is a perfect match for Geoprocessing Models in ArcGIS.
GIS Analysis for CIP Decision Making
At first we took the approach that ESRI should try and build a few geoprocessing models that all water and wastewater utilities could use to score and rate their assets by estimated remaining asset life, condition or criticality. We figured that we could do some research, interview some of our users and figure out these geoprocessing models (our inner geography geek begged us to take this approach first). What we quickly realized was that there isn’t a silver bullet set of geoprocessing models we could build because every utility system has their own approach to long term asset management and their own priorities (KPIs, level of service they want to provide, hot button issues, fiscal condition, etc) that drive their long term planning.
This was also a great reminder that even though we have the ability to use technology to automate a process, the human element is still critical, meaning that the more we talked with the engineers who are creating these CIP plans, the more we realized they need a better way to manipulate and process data so they could apply their engineering expertise to make decisions about capital projects. We also noticed when talking to engineers doing capital planning, that while they were somewhat aware of the analytical capabilities of GIS, they weren’t aware of the geoprocessing framework core to ArcGIS and how to use ModelBuilder to automate analysis and create a reusable toolset.
So we decided that we need to focus our CIP template on showing the water utility community how they could benefit from automating spatial analysis with the ArcGIS geoprocessing framework by providing some generic models. So, please keep in mind that the intent of the models we’ve provided in the CIP template is to show you how geoprocessing and ModelBuilder work within ArcGIS so you can create geoprocessing models that reflect how your utility wants to manage assets and plan for the long term. Incidentally, if you want to learn more about GIS analysis, Geoprocessing or Model Builder within ArcGIS, ESRI has lots of great resource including on-line training, books and class room instruction.
Estimating Project Costs
The second part of the CIP workflow we observed was estimating CIP project costs. Basically this workflow was estimating the cost of a project based on either replacing existing infrastructure or adding new infrastructure (main extensions, interconnections, extending service to new sub-divisions, etc). It’s important to note that all of the functionality in this part of the CIP process is core to ArcGIS and the geodatabase, all we’ve done is customized the application to automate and simplify this part of the workflow. This is what we decided to call the Costing Estimating Tools.
The first step in estimating project costs is to create projects by grouping assets together into projects. In this part of the process you are visualizing the data you brought into GIS and also the results of your analysis and then determining what assets you want to include in a project, your rehab or replacement strategy for those assets and then saving that information. So you are literally visualizing data in GIS (most likely working with many data layers of data, including the same feature datasets symbolized different ways) and doing some spatial and attribute queries to come up with candidate assets to include in CIP projects.
From there, assets that are in need of replacement or rehabilitation and spatially close to together are grouped in projects. We’ve heard from many water utilities that without a spatial context it was a real challenge for them to group assets together into appropriate projects without and also it was a challenge for them to track and manage information about candidate assets for CIP projects throughout the CIP planning process. Water utilities were struggling with supporting their CIP process with paper maps and tracking assets that were part of a project, including costs to replace those assets, in spreadsheets.
So traditionally, this CIP process took a lot of staff time and also lead to uncertainty about whether utilities were actually spending their money on the most appropriate capital projects. We also heard that utilities were struggling with how to update data when they tried to refine a large candidate list of CIP projects down to just a few to carry forward into design and that it was next to impossible to look at multiple scenarios for the same project area (assets grouping and rehab or replacement approach) because so much of this process was manual or spreadsheet driven.
We took the approach that if a utility has their assets (water distribution, wastewater collection or stormwater) in GIS, they should use their GIS asset data to group into CIP projects and then to store information about the CIP projects (like the extent and also all of the assets that are part of the project) as new data layers in GIS. This enables a utility to create an authoritative source of data about their proposed capital projects in GIS. So this drove us to create the Cost Estimating Tools.
As we began to demonstrate early versions of the Cost Estimating Tools to our utility users, we got a lot of great feedback that helped us to refine the tools. We were told that to be really useful, the tools should include the ability to either rehab or replace existing assets and to extend mains, so we programmed that functionality into the tools. We also were told by our users that they needed to be able to compare the costs of different replacement strategies (open cut, trenchless, etc) for the same set of assets so we designed the tools to make it easy to compare the costs of use using different rehab methodologies. Also we knew that the costing element of the tools needed to be flexible, because individual utilities favor different pipe materials which can be set as defaults and that unit costs are often specific to a utility and those can be easily configured in a simple table.
So what we wanted to do with this blog was to explain how we arrived at version 1 of the Water Distribution CIP Template. We are very interested in your feedback so we can incorporate more useful changes in version 2. Also we’d like to hear about any geoprocessing models that you would like to use for CIP planning. So, please leave us feedback here – http://forums.esri.com/forums.asp?c=55&s=426#426
In the next few weeks we’ll be recording a video of the Water Distributions CIP Template in action and we are also going to do a webcast in December that takes a deep dive into the CIP Template.
If you have been following us on twitter, you already know that we released the ArcGIS Water Distribution Capital Planning template (we are calling this the CIP Template for short) yesterday. The CIP template includes a set of models to help you understand how to you can use GIS to score and rank your infrastructure and a set of tools to provide cost estimates for rehabilitating, replacing or building new infrastructure.
We will film a video at the end of October that shows you in detail how these tools work, and we’ll be doing a live webcast in December to explore the CIP template in depth. So in the mean time, below is a little help with the CIP Template.
We included 6 models that show different ways you can analyze your data. To run these models, you will need to create a temporary file geodatabase and set the environmental variables for each model. The two variables you need to set are Current Workspace (the folder that has the CapitalPlanning.gdb in it) and the Scratch Workspace (the folder that has the temporary File GDB you just created).
The Project Cost Estimating tools use three tables in the CapitalPlanning.gdb for configuration. These tables are shipped to work with the data in the Sample.gdb. If you want to start changing cost or the configuration, you will need to change these tables.
CIPDEFINITON – This tables defines which featureclass’s to cost, the fields to look at(such as Diameter and Material) and a few other parameters.
CIPCOST – This table defines the cost for a particular asset. When costing an asset, you can define a Strategy, like Replacement or Rehabilitate, then an action for that Strategy, like Open cut for a Replacement . For each Strategy and Action, then you define the cost based on the fields you set up in the definition table. So if you are looking at wMains as a layer, you are interested in the Field’s, Diameter and Material, you would select your Strategy, the Action for that Strategy, the Material(say PVC), then Diameter(say 12) and define a cost for what each foot would cost. So by using a Strategy, Action and two filter fields, you can provide very detailed cost estimates.
CIPREPLACEMENT – This table allows you to provide lookups for replacement. If you are going to replace a 6” DI, you may have a rule saying that each 6” DI is going to be replaced with a 8” PVC. This table allows you to define this replacement. So that costing is preformed an 8” PVC, not a 6” DI.
Since this is our initial release of the CIP Template, we want your feedback. So please post any questions or feedback to our forums under the Water Utilities Template section: http://forums.esri.com/forums.asp?c=55
Many of you have had a chance to test out the Mobile template and provided great feedback. One question that keeps coming up is “How do I interface (or integrate) the mobile map with my other utility systems?” Typically, when we get asked this question, people are referring to their workorder system (also called a CMMS or EAM). Occasionally we are asked about interfacing with a LIMS system, mobile leak detection system, customer information system (CIS), billing system, heck we’ve been asked about interfacing with a utility’s time card system. Hopefully you notice the trend here, that water and wastewater utilities can and do want to “spatially enable” their other business systems because most of these systems contain information that has a location to it, but the other business system can’t store spatial information at all or can’t store it well.
Well, there is one simple easy answer because there are some many types of systems, vendors, API’s, gateways, etc… So I wanted to talk about a few general ways to communicate with other systems and some ideas how to work with other systems. First you need to decide what functions the field crew is going to need. For instance, if you are flushing hydrants, do they need to access to when the hydrant was flushed lasted or every time it was flushed? The reason I would ask this question is the answer is going to help us define how to work with other systems.
Lets start by looking at were to record the inspection or flushing report in the above case. If we are storing our field reports/inspections in the geodatabase, then this is fairly easy process. We can create a feature class with all fields for the hydrant flushing. This is exactly what we did with the template. The user can click the hydrant, copy some relevant information to the hydrant inspection record, such as Asset ID and populated the geometry of the report from either the hydrant or the GPS location of the field crew doing the inspection. The crew can fill out the rest of the information, click save, and use ArcGIS Server to post that information directly back to the Geodatabase. If the user wants access to all historical inspection data, then that information can either be in the same feature class as the new inspections or in a separate one. I would suggest that all historical information been in a separate featureclass. The reason is that the historical inspection or field report data can be very large. You want to manage the update of the devices with this information separately then the newly created field inspections. If all the inspections, both new and old, are in one feature class, then the map may have 100’s of inspections at the same location, may be very confusing for the field crews. The historical inspections never really need to be displayed, they really just need a tool to click the asset and pull up all related information. With the new inspection, once they create it, they then can visually see on the map that they created and save the inspection and are done working with that asset. So in summary, if you are using the GDB as you system of record for assets and their related information, inspections, flushing, etc.., then create a data schema for new data and historical data. This will provide ultimate flexibilty and usability. Just one more note on the above. If you are going to load all your historical inspection information into the geodatabase from another system, then use a process to join your historical, non GIS inspection data, to the geometry of your asset and load this to your historical field inspection data.
Now if you are storing some asset information in another system, like extended asset data in your workorder system, then you have a few ways to interface that data with the mobile map. One way is to use the Geodatabase as your connection to other systems. What I mean is build a backend process to pull out new inspections from the Geodatabase and push them to whatever system you have, and vice versa, use the same process to push information from the other system into the geodatabase. This way you can use ArcGIS Server and ArcGIS Mobile to interface with the information in the field. It is much easier to write back end database scripts to move information around then it is to build a process to push out other systems data into a field, in a format that they can access offline, make edits or entries to that information and push it back into the office.
If want to connect to the other system information store directly, without having to move it into the geodatabase, you have two options. You can work with a local copy or cached representation of that other system. This means that a set or all the data from that system will be loaded on the device. The other option is to use a web services approaches or a Enterprise Service Bus(ESB) to directly talk to those other systems.
If you want to work with the other systems directly on the mobile device, then you are going to need to figure out how to get that information on the device you are using and write a module for the mobile app to talk to that data store. ArcGIS Mobile is built with the .Net framework, so it is very easy get your Mobile GIS Information to talk to other data stores. The biggest challenge with this method is figuring out how to get the information on the device, keep it updated and push changes back into the office. Some vendors have ways to do this, some do not. I would suggest talk to your vendor and discuss what options they have. You can also look at using provisioning software that can manage pushing out information to the field and pulling back in. If you have a homegrown system, then you will need to develop a homegrown field version of the data and a synchronization method.
If the above is technical daunting and you want to use web services to have ArcGIS Mobile talk to your other systems, then I would ask yourself one question. Can your field personal do their job if they do not have a connection to that service? If so, this is a great way to interface mobile and office systems. If your answer is no, then proceed down this route with caution. Even with cell coverage getting better and better, there are always dead spots or connection issues. What is one of the first things that happen when there is an incident, the cell networks get overloaded. Also think about bandwidth. This could be a chatty system. According to Gartner, the days of unlimited data on cell networks are coming to an end, btw, unlimited data is 5GB on most carriers. If you are ok with all the above or your field crews do not need access to this information to do their jobs, then web services are great, effective and easy ways to talk to other systems. They are easy to implement and they can support many applications. All you need to do is build a module for ArcGIS Mobile that when I click an asset, it hits the appropriate web service and displays the results. This could be a simple hyperlink in the attributes of a feature.
In closing there are a number of ways to interface an ArcGIS Mobile Applications with other utility systems and we wanted to highlight few of them. The above strategies are not the only strategies; there are many ways to implement communication between different systems. If there methods you would like to discuss further, please contact us and we can help you figure out the best approach for your utility. You also may find that combining some of the approaches best suites you. For example, with new inspections, you may use ArcGIS Mobile to create a new record, store it in the cache, and post it to the geodatabase using ArcGIS Server, then nightly, use a backend script to move it to the proper system. When that field user wants to look at the historical info tied that asset they are inspection, they can hit a web service. If your field crews do not have coverage broadband, well at least they can complete their work.
I started typing this blog entry last week. My original idea was to share 5 simple steps to set up the dashboard or Flex sample viewer. Since then, two great things came up, the creating effective web maps seminar and an unbelievable flex guide from a user. Both are discussed below with my original 5 steps.
I recently had the honor of presenting the creative effective web maps seminar. If you did not get a chance to attend this seminar, I strongly suggest reviewing the seminar materials. The seminar focused on creating web maps, just like the operational dashboard. A lot of concepts and design practices were discussed. If you would like to review the materials, you can download the powerpoint and handout from this website.
At the end of the seminar, we demonstrated configuring the Sample Flex Viewer for a parcel notification application. If you are interested in doing this, take a look at the handout that was provided at the seminar. It provides a walkthrough for you. You can find it on the web site provided above.
I would also like to share a very detailed document that one of your users shared with us. Tapas Das, from the Arizona Land Department, wanted to learn the Flex environment so they could upgrade their old ArcIMS based Parcel Viewer to ArcGIS Server. In this process, he put together a very detailed word documents that goes through setting up Adobe Flex builder, using a flex tutorial, debugging flex, and much much more. Again, big thanks to Tapas for sharing this. This is fantastic. If you find this helpful, please thank Tapas.
So to my original idea, 5 steps to setting up the Dashboard.
1 .Determine your Data and Services
I like to think of the data to support my application in three formats. Dynamic, Cached and Other services. When you construct your application, you are going to use a combination of these services. By separating data into a series of cached and dynamic services, maybe some custom overlays, you will achieve the optimal performance. Think about it, if you had all data, orthos, mains, parcels, etc.. in one dynamic service. Each pan and zoom has to query all these layers, label and draw them and send the image down to the browser, but if you separate the orthos and parcels into one cache service. A pan or zoom will just grab the pre-rendered cache service of orthos, parcels, etc. and only query and draw the layers that need to be overlaid, the mains, valves, etc..
When you are trying to split up your data, here are some helpful ways to think about your different services:
Dynamic Map Services – Optimized or Regular
- Real-time data
- Frequently-changing data
- Reporting Layers
- Widget Results
Cached Map Services
- Background data
- Data that does not change often
- Projected on the fly – what I mean by this is that if this data is not in the proper projection, set the data frame to the proper projection and the cache will be build with the data projected.
- Geoprocessing, Routing, Locators, Custom XML services
- Real-time data from other systems
- Results from tables or other systems
When configuration the Sample Viewers, you have three options to display your data. Each of these options gives you a little different control over the data.
Typical these are cached maps. They are drawn as the bottom layer in the Viewer and only one base map layer can be displayed at a time. The user does not have the option to toggle layers in the map on and off. The service is treated as one layer.
Typical these are dynamic maps. They are drawn on top of the basemap and can be stacked on top of each other. Think of each map service as a group layer. If they are dynamic services, the user can toggle different layers on and off.
We refer to these as client side graphics. The information behind them is coming from a dynamic maps services or what I referred to above as other services. If you are using a dynamic map service, the supporting map document is usually a few layers. Symbology and scale is not determined by the map services, so do not spend anytime of this map doc look and feel. The widget will query service, map service or other, filter the results and display a graphic.
This can include a table of information. At this moment the Rest API’s do not support tables. This should not prevent you from creating a web service that connects to a SDE table or third party system, and send those results to a widget as XML
Widgets are also a great place to show the results from a selection. We typically call these selection sets in desktop GIS. We can display them in the standard grid, but try to think about the results in a different manor. Can we use a chart to display them? Is there any way to rely the same info in a more intuitive way?
2. Set up the Flex Viewer
Set up files in IIS
- Copy the Unzipped flex application to a folder on your Web Server.
- Create a virtual directory for this application
- If you need help with this section, follow the section in the templates help files
- Apply CrossDomain.xml – to access data from a different server than the one hosting your Flex application, the remote server needs to have a cross-domain file in the root directory. For security reasons, the Web browser cannot access data that resides outside the exact Web domain where the SWF file originated. However, Adobe Flash Player can load data across domains if permission is granted from the server. This is accomplished by including a small crossdomain.xml file on the remote server that permits Flash to connect to services on that server. http://resources.esri.com/help/9.3/arcgisserver/apis/flex/help/index.html#references/using_crossdomain_xml.htm
3. Determine Widgets
The Flex viewer template comes with some widgets for typically web mapping functionality. Some of them are listed below.
- There are more widgets posted on the resource center for the Flex API.
4. Update the Configurations File – Main Viewer
Open Config.xml in the Sample Viewers Root & Change the following tags
- <Title> and <Subtitle> – This is the text in the bar at the top
- <Menus> – Menus are the drop downs in the banner bar. You add will add tools, widgets and layers to these.
- <Map initialextent…..> – Adjust these extents in the spatial reference of the viewer
- <Basemaps menu=”menuMap”> – Add the basemap services. These services are display vertical on menu listed
- <LiveMaps – Add the operational layers services.These are displayed in the LiveMapsWidget and are displayed on a floating widget. Make sure to leave the LiveMapsWidget in the widget section
- <Navtools> – By Default, all navigation tools are listed
- <Widgets> – Add all the widgets you need in your application
5. Update the Configurations File – Widgets
Check and see if your widgets have configuration files. If they do, open them up and adjust the configuration for each.
We’ve had a few questions about whether users and ESRI business partners can submit templates to the Water Utilities Resource Center. We wanted to tell everyone, the answer is yes!
We are encouraging our users and business partners to submit templates to the Water Utility Resource Center template gallery.
Keep in mind there are 5 items that a template must have and all of these items must be in your template zipfile:
1. An instruction document – with information on how to install and configure the template. Including what software is necessary.
2. Any MXD or MXDs necessary – the mxds are critically important to show everyone your cartography, geoprocessing tools, etc.
3. Any custom code – Including the source code.
4. Sample Data – a populated geodatabase with any necessary data for your template. This is so everyone can understand how your template works with sample data. If you can’t share your own data than you can use the sample data we’ve provided with the Mobile, Editing or Dashboard templates for your template.
5. A blank geodatabase – this is an empty geodatabase with the same schema as your sample geodatabase.
Here is an example of the folder structure your template should follow:
A few ground rules for submitting templates – we will review each template to ensure that the proper items are in them and they function as advertised. We won’t accept any templates that have trial software applications or extensions in them or don’t have the source code if there was custom programming in your template.
So anyone ready to share your good work?
If you have any questions about how to create your own template and post it email us at ArcGISTeamWater@ESRI.com