Category: Water Utilities
A few days ago I was talking to a manager at a small combined water and wastewater utility who wanted to know with a limited budget, how can they implement GIS and where should they get started?
That’s a question we often get. No doubt, most large and small utilities implement their GIS as a series of ongoing projects. Most of the time utilities set aside a yearly budget for GIS to fund things like data creation, software purchase & maintenance, hardware and training. Often small utilities will focus their data creation efforts on creating a few new layers a year. This holds true whether they do the data creation themselves or hire a consultant to create their data.
The starting place for the majority of water, wastewater and storm water utility GIS projects is asset management. This means both using your GIS as the system of record to store your assets information and then using that asset information to make decisions about when is the optimal time to repair rehabilitate or replace an asset. If you are implementing an Enterprise Asset Management System (EAM) than both your GIS and your work orders are part of a system of record for your assets.
So if you are a small utility, what is an example of a high level process to get started with GIS?
- 1. Download the ESRI Simplified Water Sewer Datamodel http://support.esri.com/index.cfm?fa=downloads.dataModels.filteredGateway&dmid=16 A data model is the structure (GIS people call this a schema) of how to store data in GIS. ESRI with a consortium of utilities and consultants has identified the best practices for storing water, wastewater and stormwater data in GIS and posted them on our website as data models you can download for free. The ESRI W/WW & SW data models were the first freely available data models for these industries and have become the de facto industry standard for how to store these types of utility data in a GIS. By using these data models as a starting place for your GIS you will save money and time and will not be reinventing the wheel.
- 2. Compare the assets you own with the layers (GIS people call these feature classes) in the data model. Also make sure that data model can hold the descriptive information you need about each of your assets (GIS people call this attributes). So in the sewer data model you’d have a feature class for gravity sewer mains that would have attributes such as diameter, material and installation date. The data models have been in use for years and have most of the common assets in use at W/WW utilities, so more than likely you’d identify features in the data model that your utility does not own and remove them and add some attributes that your utility wants to track.
- 3. Modify the data model you download to have the exact set of asset and attributes that you’ve identified your utility needs in your GIS. You may want to have a consultant do steps 1 to 3 for you or with appropriate training before attempting this task you can do this in house. Some utilities choose to have a consultant do a data model workshop where all of asset management stakeholders (the people in the utility that will use the information in the GIS) identify what specific information they need. Engaging all of the asset management stakeholders in modifying the data model is good investment for utilities.
- 4. Get base mapping data – Base mapping data is data that you want to lay your utility asset data on top of in a map. There is a tremendous amount of free GIS data you can get from places like the cities or counties your utility serves. ESRI also provides free base map content through ArcGIS online.
- 5. Identify data sources that you want to use to populate your GIS – such as hard copy maps, construction plans, as-builts, field maps, CAD files, data from spread sheets, etc. If you don’t have any good source data to start with then you may want to consider doing GPS data collection
- 6. Do a pilot – Start to put data into your asset data into the data model. If you are doing wastewater assets, select part of your collector system (such as one sewer shed). At the end of creating pilot data use your GIS to make some maps and do some analysis – ask the questions you need answered to make better asset management decisions of your new data. Doing a pilot makes sure that any changes you made to the data model suit your needs and also helps you become comfortable with GIS. Identify any lessons learned through the pilot and then modify your approach if needed. Also identify some steps to do quality control of your new GIS data.
- 7. After the pilot keep populating your asset data in GIS and maintaining the asset data you’ve already created. Now you have a production GIS for asset management. If you are a small utility with limited budget you may want to break your GIS up into phases – such as doing 1 sewer shed each year or doing your water distribution system in the 1st year and then your wastewater system the next year.
By investing in training or hiring someone with GIS knowledge you can achieve these tasks yourself. Also many small utilities use an ESRI business partner to help them get started with their GIS. Understanding the steps above will help you better engage with a GIS consultant.
While asset management has been the traditional starting place of W/WW & Stormwater GIS, we’ve seen an interesting trend emerge over the past few years where some utilities have made vehicle routing or a management dashboard their first GIS project. By starting with something like vehicle routing, water utilities can have a quick win with GIS (who can argue with the benefit of fuel savings & reducing vehicle miles driven) and become comfortable with GIS as technology before moving forward with getting your assets into GIS.
As always, if you have anything to share on this topic please post a comment or send an email to: ArcGISTeamWater@esri.com
The ESRI Water Utilities Team posted updates to all three templates in the Gallery today. In summary, these updates include:
Updates to the Operations Feature Dataset in the Sample.gdb.
Updates to the Getting Started documents.
Release Notes for each Template.
General improvements to error handling and messaging in the Mobile Map Template.
Performance improvement for several tools in the Network Editing Template.
Bug fixes in the Network Editing and Mobile Map Templates.
New functionality added to the Network Editing Template.
For details on the new functions and the bug fixes, please refer to the release notes in each templates folder.
The Team will post updates from time to time to resolve known issues and add new functionality. If you have specific questions about the updates, please let us know: ArcGISTeamWater@esri.com
The ArcGIS Water Team
Part 1: Explanation of the Sample and Template GDB
ReferenceData – Landbase data typically acquired from other organizations
We would like to once again thank the Fort Pierce Utilities Authority and St Lucie County, Florida for allowing us to include a sample of their data in the templates. You should be aware that we took a copy of the data at a point in time and built additional content on top of those datasets. As a result, the data in the templates has been significantly altered from the original database and also does not reflect real conditions in Fort Pierce Florida. That will be obvious to many readers of this blog, but we want to clarify that for people who are just getting started.
The Sample.gdb is quite similar to the Template.mdb, but it has more content in the ReferenceData feature dataset. The reason for the difference is that most water utilities do not control the landbase data they use, and we expect that (like Fort Pierce) most of the data loading from partners will essentially be a copy and paste into the target geodatabase. As a result, we only included a few ReferenceData feature classes in the Template.mdb, and depending on your system it is ok to remove those feature classes if you have something different to use.
Below is a detailed explanation of each Feature Dataset in the Sample/Template geodatabase.
This feature dataset contains the water network feature classes. ESRI has built template water network data models for about 10 years and most water utility projects have used those designs as a starting point for implementation. This design should look familiar to most longtime users, but there are some significant design changes here that are described briefly later in this document.
To load data into these feature classes, you should first drop the geometric network. Data loading tools will run faster without the network, and you will also be able to use all of the data loading tools available in ArcGIS. Once you have all of the data loaded you can rebuild the network using the wizard in ArcCatalog.
The loading process is a bit different than the ReferenceData because you have a target database design to load into. Of course it is ok to use a different design or to use the design you already have, but that will mean more work if you want to use the map documents provided with the templates. You might also want to use this latest design if you have existing data because it improves on previous examples.
Generally speaking you will need to do more than copy/paste your water network features into a target geodatabase. You will likely need to calculate new values, combine or split datasets into new feature classes, and you should also have a plan for doing QA/QC on the results of the data loading process. This can be a time consuming exercise that might be more practical to contract out to an organization that specializes in this type of work, and even for simple examples many weeks of work can be involved in getting existing data loaded and validated.
This feature dataset contains administrative and planning content for water utilities. The reason for putting these features in their own feature dataset is that they are edited on a different cycle than the asset data and they may have different permissions/editors.
The simple part of this feature dataset is the feature classes for EngineeringGrid, Map Sheet boundaries, and other administrative areas for water utilities. This part of the database also contains planned replacement mains – wReplacementProject (Capital Improvement Projects) and also proposed mains – wProposedMain (New Development).
This feature dataset also contains content that will be new to most users. What we built in the Fort Pierce database was a set of reporting layers that allow us to organize and view asset inventory, consumption, asset condition, and other data in the context of operational management units like map sheets, districts, and political jurisdictions.
These feature classes were created by spatially joining/intersecting the water network features to sets of polygons like the EngineeringGrid example shown above, and then further calculations for leaks/mile, consumption, and inventory information were performed.
There are 2 main benefits we see from this work:
1. The ability to provide more granular reporting. For example, consumption and number of valves by map sheet or smaller area rather than company-wide reports.
2. Applications such as the Operations Dashboard require that we prepare data so that rather than users running general queries on the data they can just click a few times and get the information they are looking for. As a general principle, we want to “cook” the answers to the common questions to improve performance and reduce load on web servers.
Last but not least the FieldOperations feature dataset contains a number of feature classes that are used in the Mobile and Dashboard Water Utility Templates. These include leaks, redlines, and workorders. While the Mobile Map example only uses the redlines capability in the database, the design includes the ability to manage workorders and inspection information to/from the field as well.
The one thing people notice right away is that workorders and inspections in this data model are Point feature classes. Yes, we know that most workorder/CMMS systems do not store point features because they are tabular systems, but part of simplifying the data/application for field users is finding the geographic location for work. In addition, ArcGIS Mobile works with simple feature classes so this approach works well from an implementation standpoint.
The content for this feature dataset will be more dynamic, and will likely only contain assigned/current work that needs to be pushed to/from the field. For example, today’s work can be loaded into the mobile feature classes and updates can be pushed to the field. Similarly, as work happens in the field, data can be pushed back into the office through a mobile data service.
Ultimately this approach requires some back-end work to get the data from one or more enterprise systems to the field application, and then from the field application back to the enterprise systems. This is not particularly difficult work, but the data loading and management activities will be different than the other feature datasets in the template geodatabase.
Data Model Notes
The data model available with this template has some changes from those data models that reflect the evolution of implementation models for geodatabases. These changes include:
- Uppercase, short field names to ensure that field names are not altered/truncated when moved between different database platforms (and yes shapefiles are still being used for data exchange)
- Longer, more meaningful alias names on fields
- Descriptions on fields and feature classes (FGDC stored as FGDC metadata)
Feature class aliases are plural but table names are singular. When you add a feature class to ArcMap you will get a layer with the plural name (the convention GIS users prefer), but it also works for database people that insist on singular names for tables.
- LifecycleStatus field replaced by ACTIVEFLAG. Previous designs had proposed, active and abandoned features in the same network feature classes. Over time most utilities have separated proposed and abandoned features into different feature classes so they are not accidentally included in network traces or asset inventory reports. Once we added those feature classes to this design, Lifecycle status just became a flag to indicate if the features are active or not. For example, there are temporary/seasonal services that are only used for part of the year, and there are other assets in the ground that are not active because they are still under construction or are temporarily inactive.
- OWNEDBY/MANAGEDBY fields indicate the owner of each asset and also the maintenance responsibility. While most domains/list of permitted values in this data model are strings, these domains are integers. The plan here is that “Company owned” assets that should be counted for inventory purposes have a value > 0 and things that should not be counted have a value < 0. In many cases there are multiple companies involved in county/regional scale systems and this strategy will make asset reporting simpler. Of course in those situations people should extend the list/domain of values for their specific situation.
- You may notice that we also dropped AdministrativeArea and OperatingArea fields because these are better managed as part of the reporting layers we built for the PlanningAndOperations data – which is described in the next section of this document.
In each of the Templates, you will see the sample geodatabase and the template geodatabase in the “Maps and GDBs” folder. If you look in the subfolder “Documentation” there are html documents that provide reports for the geodatabase and map documents.
These documents will help you to understand the details of the geodatabases and maps, and also help you to plan out how to load data into your Geodatabase and make changes to map documents to work with your data.
Most projects will start with a source-target matrix spreadsheet that describes the available data and the target datasets for their new Geodatabase. This is a good place to start and it will help you to assess the suitability of the template design as well as the level of effort required to build your Geodatabase.
Again, this can be a large part of your project, so keep in mind that ESRI and Business Partners are here to help you if you need us. Network with your peers to get their recommendations on who to work with, or email us at ArcGISTeamWater@esri.com and we can get you in touch with someone local to help you to get started.
We’ll be focusing our next few blog posts on building and maintaining water utility geodatabases. Since we went live with the Water Utility Resource Center 2 weeks ago, we’ve already had numerous questions from the user community about how to use your utility data with the templates we’ve provided, so our first post will explain the template and sample geodatabases.
New or prospective ESRI customers often ask us how to approach building a geodatabase for their utility and loading data. Many of our long-term water utility customers have questions about strategies to maintain their geodatabase to ensure it continues to meet their business needs and fit their IT landscape as other enterprise systems evolve. So we’ll also share some best practice and tools for building and maintaining water utility geodatabasesIf you have more specific questions about building your water utility geodatabase, data maintenance, data loading or for that matter anything to do with GIS for water utilities please let us know: ArcGISTeamWater@esri.com
The ArcGIS Water Team
To install the template you need 3 things:
1. Microsoft .Net 2.0 Framework – This is a free download from Microsoft. http://msdn.microsoft.com/en-us/netframework/aa731542.aspx
2. ArcGIS Desktop 93 or 93SP1
3. The .NET support feature of ArcGIS Desktop
a. To get the NET support feature of ArcGIS Desktop have this you need to install Desktop after Microsoft .Net 2.0 Framework is installed on your machine
b. Make sure you have your ArcGIS Desktop DVD or CD inserted into your machine. Go to addremove programs dialog >Select ArcGIS Desktop> Select changeremove > select modify (you will see the dialog below) > Click on the .net support feature > select “will be installed on your local drive” > Next > Next then try to install the template.
If this does not resolve your problem, let us know and we can track down the problem.
Welcome to ESRI’s first industry focused resource center. We are very excited about providing our water, wastewater and stormwater utility users with information specifically focused for your industry, such as free downloadable templates built around core ESRI software, a place for users to share their own templates and knowledge and this blog to assist water utilities in deploying our technology.
We would like to thank the Fort Pierce Water Authority and St Lucie County, Florida for allowing us to include a sample of their data in our templates. Allowing us to develop these sample templates with real customer data makes the samples a more powerful example of the benefits water utilities can expect from deploying ESRI technology.
We’ve posted three templates for the launch of the water utility resource center. These templates are for maintaining your water utility assets, sharing information with the field and enabling operational awareness. We are currently working on a fourth template for capital improvement planning, which we will post in the near future.
The goal of our templates is to give our water utility customers real world examples of how to deploy the ArcGIS product suite in your industry. There is a zip file for each of the templates we’ve posted. Each zip files include detailed installation instructions, map documents, an implementation of ESRI technology, including the source code, sample data schema and a sample dataset using this schema. We’ve also recorded videos with detail information on how to install and configure these templates.
We decided to create these templates because we wanted to pass on to our customers best practices, successful deployment patterns and share industry specific knowledge. Remember, the water resource center is for ESRI users – so we encourage you to post what you’ve developed to the template gallery. We also need your feedback on the templates we’ve provided and ask that you post comments for each template. We’ll be using your comments as a guide for changes to our templates and for new templates we’ll develop in the future.
Keep watching this blog; we’ll be sharing a great deal of valuable information here. We plan to post detailed information on our three launch templates, instructions on how to use your utility’s data with the templates and also share best practices we’ve observed from working with our water utility customers.
This is truly an exciting time to be using ArcGIS for water, wastewater and stormwater!