Category: Water Utilities
Based on our recent blog on Generating Unique Asset IDs, we had a few people ask about how to handle generating IDs in the field and how to handle new assets turned over from developers. So we thought it would be good write a follow-on blog about that topic.
First off, for most utilities having a methodology for generating unique asset IDs doesn’t necessarily mean that an asset’s permanent unique ID is actually created in the field. So while the title of the blog is generating unique asset IDs in the field, that’s a bit of misnomer, because not allowing field staff to create permanent asset IDs may be the most efficient process and also be the best solution to safeguard your data quality.
I think the question to answer for a utility then is “what is my process for creating asset IDs for assets we ‘discover’ in the field?” Since we are focused on GIS here, by discover we mean that you capture the asset’s location spatially and this drives the need to give it a unique ID, but what we are discussing is really applicable to any asset that is found in the field that you need to track in a computerized system. Also, we are using the term ‘discovered’ here because these aren’t new assets, just assets that you didn’t know existed or may have known about, but didn’t know where or what exactly they were before.
There are many ways that you may discover new assets – as a water utility you may find the locations of valves that were paved over and recently restored to grade, or as a wastewater utility you may find buried manholes when you CCTV. So ideally when these assets are discovered they are ultimately captured in the GIS with appropriate spatial and descriptive information. Spatially they may be captured in the field as a feature in a feature class, ideally with the appropriate level of precision (such as with sub-foot GPS) or perhaps as a rough location with descriptive information that tells the utility the asset needs to be captured at a later date more accurately. Incidentally, we often see this type of workflow with water utilities that want sub-foot asset locations, but only have a few crews that have the GPS equipment to achieve that level of accuracy. So the GPS crews will go back and capture the asset locations when time permits because the utility has an attribute that allows them to know the spatial quality of each asset location.
You may also have developers build infrastructure for your distribution or collection systems and turn that over to you – these are new assets and different from discovered assets. For assets that are being built and turned over to you by a developer, it’s best to track all of those assets in your GIS from the time you learn they may be built. We see more utilities starting to use a proposed feature dataset in their geodatabase to track potential system expansion. When the asset are built, the data is modified to reflect developer submitted as-builts or GPS data collected, assigned a unique ID and then moved over to your water or sewer distribution feature datset from the proposed feature dataset. We’ll plan to cover using proposed, abandoned and active feature datasets and other tips on managing asset in GIS as the go through their lifecycle in a future blog.
IDs for Discovered Assets
So for discovered assets, there are 3 options for assigning asset IDs in the field – temporary asset IDs, permanent asset IDs assigned by the field crews & permanent asset IDs assigned by your asset “system of record”.
Temporary asset ID assigned in the field
The simplest way to go for field users. Basically let the field user or have your mobile GIS application give a temporary ID to an asset that signifies it was captured in the field and doesn’t have a permanent asset ID that conforms to the asset ID methodology of your utility. Ideally there is some sort of identifier in the asset ID that shows the asset ID is temporary – for example for a manhole ID you could use MHT and for valves you could use VT.
Then your workflow would be for the appropriate person or system in your office to assign the permanent asset IDs. Using our example, a simple query for the letter “T” would reveal any assets that don’t have permanent IDs assigned yet. If you are using a mobile GIS application with data synchronization capabilities, such as ArcGIS Mobile based solution, when the asset is assigned the permanent ID back in the office, the field users will see the new asset IDs in their applications the next time their data is refreshed.
Permanent Asset IDs Assigned by the Field Crews
Field crews could assign permanent IDs to assets at the time of discovery. We’ve seen data collection forms that call for an asset ID to be entered, often these are paper based forms that have a box for an asset ID. We’ve also seen electronic data collection forms for GPS using ArcPad that allow or require field crews to enter asset IDs.
So in this case a field crew has the ability to assign or record an asset ID in the field. Allowing a field crew to record an asset ID is good workflow when there is already and ID on the physical asset, like a unique number on your manhole lid or a barcode on the asset and all the field crew is doing is recording the ID already given to an asset when they capture a more accurate location or do an inspection.
But allowing a field crew to assign asset IDs without any guidance from a mobile application or without any coordination from your office is a workflow that could introduce data integrity problems. For example, a field crew could assign asset IDs that are duplicates of IDs already in your system.
Permanent Asset IDs Assigned by Your Asset System of Record
If your GIS is your asset system of record, than using a mobile GIS application with real time connectivity back to your office can ensure that the appropriate asset ID convention is used and the next sequential asset number is assigned. Or if you are using an Enterprise Asset Management or EAM System, which is typically now a GIS integrated with a workorder system, the GIS application can communicate with the workorder system on the backend in the office to assign the permanent asset ID on the spot in the field.
If you are a smaller utility and you don’t have mobile device connectivity, you can also call into your office and have someone assign an asset ID from your system of record, but this isn’t a very efficient workflow.
It’s seems like right now, the idea of capital project coordination between utilities and other government entities has reached critical mass. Whether faced with budgetary problems due to the economy, the need to do a significant amount of capital projects now (or both) recently we’ve had a lot of ESRI customers asking us how they can use GIS to coordinate their capital projects better and what information about future projects they should share with other utilities and municipalities in their operating areas.
Since water and wastewater utilities are in the same right of way with other public utilities such as electric, gas and telco and also are typically under the pavement owned by other municipal entities, departments of transportation or highway agencies a little coordination yields big cost savings. We see this capital project coordination happing between departments, such as between a city’s wastewater and streets departments and also between utilities and external entities such as between a water utility and gas utility.
For example if you can coordinate your work with other utilities when a street is opened you can split the restorations costs. Jurisdictions your utility operates in may also have street paving moratoriums, for example they may prohibit streets to be opened until 2 years after they have been paved, so in this scenario you need to know what streets are being paved in the next few years so you can complete any project work in those areas ahead of the schedule paving.
Capital Project Coordination with GIS
So how can you use GIS to coordinate capital projects?
Capital Project Aggregation
Even before GIS, many water and wastewater utilities tried to gather the capital project locations of other utilities and municipalities in their service area. This was often a time consuming process, with multiple requests for information and paper document submittals that the water utility needed to aggregate into a big picture view of capital projects in their service area. With GIS, many water utilities started to input capital project locations from other utilities as GIS layers. This still required a great deal of digitizing effort to create project polygons and even if you had the location of future projects in your service area you may not have had good descriptive information. Maybe if you were lucky, the other utilities in your service area also use GIS and are were able to provide shapefiles or geodatabases of project locations.
We’ve seen some utilities work as “consortium” to coordinate their capital projects with web based map viewers. For example some utilities aggregate capital projects from municipalities and other utilities in their operational area and then provide free access to a web based mapping application for all of the entities that provided project information. Other utilities have created web mapping application where local utilities and municipalities can edit their project footprints and enter basic project information. Setting up a website with security where utilities can enter project locations on a basemap is very easy with the web editing capabilities of ArcGIS Server. The drawback to this consortium approach is that it takes someone to have the infrastructure to host and manage a web based application and the other utilities have to submit data or enter it.
An easier way to share capital improvement project locations for utilities with ArcGIS Server is to publish a map service of project locations. This is easier because if a utility keeps all of their project locations in a centralized enterprise geodatabase than a published map service of project locations will always be up to date. The project location map service could be used both internally at the utility and externally and could be consumed in a variety of applications like other ArcGIS Server applications, ArcGIS Desktop, ArcGIS Explorer, in AutoCAD with ArcGIS for AutoCAD or in Google Earth as KML. If you want more information on publishing map services with ArcGIS Server you can find it here: http://webhelp.esri.com/arcgisserver/9.3/dotNet/index.htm#map_service.htm
With the new free content sharing capabilities, ArcGIS online is an excellent option to share project locations. For those that haven’t seen the new content sharing functionality in ArcGIS, with ArcGIS 9.3.1 you can upload your data as layer packages to ArcGIS online and share it with everyone or with a group you designate. So you could create a group of local utilities and municipalities that you want to share project locations with
and you can securely share your project locations with ArcGIS online.
If published a map service of future project locations you could also share that through ArcGIS online with a group, much like a layer package.
We’re very excited about the new content sharing capabilities of ArcGIS online because it’s a great solution for water, wastewater and storm water utilities to securely share their GIS data and maps. It can be used as we outlined above or used to share data with GIS consultants, engineering firms, developers, etc. Best of all, ArcGIS Online content sharing is free!! We plan on doing a more detailed blog in the future that goes into depth on using ArcGIS Online to share your content for project coordination.
In the meantime we encourage you to explore ArcGIS online content sharing for yourself: http://www.arcgisonline.com/
Utility Coordination Data Model
We commonly get asked by water utilities if there is a data model for capital projects. There isn’t an ESRI community data model for capital project coordination, but from discussions with water utilities that are using GIS for internal capital project tracking or for project coordination with other utilities and municipalities, we’ve seen some trends. Most utilities track their projects as a polygon of the project extent and are capturing fairly simple attributes – project name and/or project ID, project start date, project end date, project contact name, project contact information such as phone number or email. We’ve seen utilities begin to track project funding information as an attribute as well, both total project costs, project cost by year for multiyear projects and also funding source information if stimulus money is being used for the project.
For water and sewer main extension or replacement projects most utilities buffer the mains to be replaced by a predetermined distance, like 25 or 50 feet to generate the project footprint.
Some utilities are also using project IDs to link to a project management or tracking system to store extended project information.
We commonly get asked by water, wastewater and stormwater utilities for some tips on how to establish a system for unique asset IDs, so we thought it would helpful to share some insight on this topic. It seems like almost all utilities now recognize the need to have a standard methodology for establishing unique IDs for all of their assets and often use GIS implementation or data model upgrades as an opportunity to introduce a new system for assigning unique asset IDs.
First, and most importantly, it is an absolute must to use unique asset IDs!! In our modern era of computers, databases and GIS, unique assets IDs play a critical role in reducing duplicate or confusing information about your assets and also is the first step to being able to reduce silos of data and integrating your systems.
That being said, it seems like some water, wastewater and stormwater utilities make the process of coming up with a unique asset IDs a more complex endeavor than launching the space shuttle. It doesn’t have to be that difficult, because inherently GIS has the ability to use any unique number or letter combination to store, manage and retrieve information about your assets. Things become complicated when we try and bring back the human element into our asset ID naming system. That is, you try and create asset IDs that have information in them that allows humans to understand what the asset is or where it is located. An example of this would be using an asset naming methodology that produced names like H-MT601 – where H is for hydrant, MT is for the service area in this case Millbrook Township, and 601 is the number of the hydrant.
But if you are using a GIS, you are already accessing asset information by location and can eliminate location information from your asset IDs. We hear 2 common rationales for having a location code in asset IDs. The first is that field crews need a location code to help them know what service area assets are located in or know what map book sheet they will find an asset. A good mobile GIS application eliminates this need.
The second rational we hear is location codes in asset IDs are necessary for reporting. For example we commonly hear water utilities say they need municipal codes in their hydrant IDs to run reports for fire billing. This problem is also solved by GIS because you run fire billing reports, or any asset reporting for that matter, but municipal boundaries or other polygon layers in the GIS.
Asset ID Methodologies
So here are some methodologies that we’ve seen utilities use for their asset IDs:
Incrementing Asset IDs – incrementing asset IDs by one number whenever a new asset is put in service. This is by far the simplest way to assign asset IDs, provided that one person or system is assigning all of the asset IDs. The Dynamic Value Table in the Water Editing Toolbar that is part of the Water Editing Template is simple way to have ArcGIS increment your asset IDs.
Asset ID preceded by asset type - For example putting an identifier in front of asset ID to identify the type of asset. For example MH preceding manhole IDs or FH preceding fire hydrant IDs, M preceding meters numbers, etc. This is fairly common and is useful to help identify which asset ID corresponds to which asset when labeled on a map.
Asset IDs with geographic identifier - discussed above, in a well done GIS implementation, there is typically no business value in this practice.
Asset IDs with a system identifier – This is mainly seen in wastewater systems where it is important to identify and differentiate interceptor lines, typically for reporting purposes. For example, a utility with 3 different interceptors lines – Bushy Creek, Forest Run & Oak Valley might put BC, FR & OV in front of any of the assets that in those drainage basins. Arguable you could also use a polygon to establish what assets are in each interceptor or sewershed for reporting purposes.
Increatementing Manhole IDs from the lowpoint up – Some wastewater utilities choose to increment their manholes IDs from the lowest manhole to the highest manhole along interceptor or trunk sewer lines. This tends to work well in areas where trunk mains or interceptors aren’t going to be extended in the future and the utility is aware of all of the manhole locations before establishing asset IDs. If new manholes are found (such as buried manholes) or manholes added along the interceptor than typically that manhole is given the next downstream manhole ID with a letter designation. So a buried manhole found above MH27 is given the asset ID MH27A. Once all of the manhole IDs have been assigned from the bottom to the top of the interceptor than many wastewater utilities just increment the manholes on each of the collector lines that feed into the interceptor moving from the bottom to the top of the collector systems.
Here is a good website from the EPA on manhole numbering: http://www.epa.gov/NE/sso/manhole-id.html
Sanitary Sewer Pipe IDs - Often done with a combination of downstream manhole, a dash, and then upstream manhole. So sanitary sewer pipe ID 26-27 takes flow from manhole 27 and carries it to manhole 26.
Geopins – Not often seen in utilities for asset IDs, a geopin is calculated using the X & Y coordinates of a spatial feature. Geopins are commonly seen in land records. Geopins are not a good choice for spatial features that are assigned new loction, such as when a utility improves the spatial accuracy of their asset base by doing something like sub-foot GPS field data collection.
This is an example how geopin would work for a valve at coordinates: X = 23658974.365 Y = 6584752584.968
Geopin Formula = X2 Y2 X3 Y3 – X4 Y4 – X5 Y5 X6 Y6
GeoPIN = 3568-54-8795
Asset ID Tips
Don’t use internal database IDs or geodatabase IDs for your asset IDs – This is a cardinal rule! Never use any internal IDs from a geodatabase or any other database for your unique IDs unless you are 100% certain that it will not be automatically recalculated by the application in the future. An example of this is the OBJECTID attribute in a geodatabase. Because the OBJECTID attribute is internally used by the geodatabase, on occasion in may be recalculated by ArcMap.
Keep your legacy asset IDs - It’s perfectly acceptable to store your “legacy” asset IDs when you switch to a new asset ID system. It may be very helpful to know legacy asset IDs if someone needs to refer back to as-builts or old field map books. We saw a great example of this with a municipality that created a table with a one to many relationship between new asset IDs and legacy asset IDs that were used on any old plans or maps that showed the assets.
Make sure only 1 application or person assign new asset IDs – This is critically important. Only 1 application, such as ArcMap, your CMMS or 1 employee is who assigns unique IDs, otherwise you are running the risk of assigning duplicate IDs.
Storm Water IDs - because of the ad-hoc nature of how most water storm networks were built, especially in suburbia and the difficulty in understanding how the water flows without good elevation data, it’s difficult to create an asset ID system for stormwater that describes which assets flow into each other. It’s much easier to just use incrementing IDs.
If you’d like to share other asset ID systems you’ve seen, please feel free to post a comment on this blog.
If you had the chance to attend the user conference, I am sure you are excited about some of the new functions you saw in 9.4. I know I am. I cannot wait for the new editing environment. To be able to set up editing templates that default attributes, symbols, etc. is going to make managing our assets so much easier. There are many nice additions at 9.4, I wish it was ready tomorrow!!
What did you think of the 9.4? Are you excited about it as I am?
I also wanted to thank all the users who were able to attend the sessions dealing with water, wastewater and public works and those who stopped by the industry island. We had a great turnout and fantastic feedback. Some of you had a chance to talk to the water team and share some great information. If you did not get a chance to talk to the team or you wanted to follow up on the conversion, please send a note to ArcGISTeamWater@esri.com. We also started a new thread on the forums so you can post your enhancement request, success stories, bugs or issues you may encounter with the templates. You can also use these forums to talk about any other issues you may need help or advice.
The direct link to the post for the templates is:
Here is the link to the forums:
Thank you and looking forward to hearing from you.
ArcGIS Team Water
If you have played around with the configuration file or in the source code for the mobile map, you may have noticed some functions that are not visible in the application. That is because a typical ArcGIS Mobile application has a lot more functionality then just a paper map replacement. We built mobile map application so it could be expanded beyond an electronic map as your field crews become comfortable with a mobile GIS application. There are functions included in the code of the mobile map to pull a cache from a web server as a zip, perform field inspections, list out workorders from a workorder featureclass, call a network analyst routing service and perform an isolation geometric network trace. In the June update of the Mobile Map Template, you can find these components. Below are details of each of these components.
Cache Update Web Service
- When I was creating this template, I started to think about how to push updates out to a large number of users. Obviously we could have each user ask the server for an update of their data, but this would require a lot of bandwidth and server resources because each user would be rebuilding the same layers. So instead, how about building the cache once and delivered to the field as a zip file. You can always build a mobile cache or a piece of the cache using a Geoprocessing model and manually zip it. But it’s faster to automate this process, and, perhaps, run it nightly. So I downloaded the python zipping tools(link) and build a model to do this.
- So that was very simple and now we need to transfer the data to the device in the field. I looked around to find a good method to deliver a large file through Http. I settled on a MTOM web service. MTOM is part of the WSE 3.0 framework for .Net 2.0. This may not be the best way and is certainly not the only way, but it is one that looked promising. A MTOM web service can handle transferring a large file through http, chunking that file and it also supports starting, stopping and resuming a download. I did not implement all these function, but there is a sample on codeplex that is very detailed. I used the BinaryDataMTOM sample that is included in WSE install. I attached the web service to this post.
- How does the app in the field know if there is an update? In the mobile app, when a cache is downloaded, I get the file date from the zip and log that to the config file. The attached web service has a method to get the date of the zipped cache on the server. If the dates are different, then the mobile app is prompted to pull down the update.
- So you could always use a different workflow or applications for zipping and file delivery, but hopefully this gives you a good conceptual starting point to build your own cache delivery mechanism. If this is too much for you, then I would suggest looking at some of the COTS provisioning software vendors.
- To set up this sample, follow the word docs in the cache tools folder. Once you have them running. Just change some tags in the MobileMapTemplate.exe.config file.
- CacheServiceURL – needs to point to your cache web service
- ShowCacheUpdate – tells the mobile sync control to show the update cache button, set it to true
- So we have provided the field crew with a digital map, but they are still filling out reports and inspection information on a clipboard and paper, in your quest to keep data up to date and increase efficiency, this does not make sense. Why not use the Geodatabase to capture field inspections, and then write a backend process to move it to the proper system? Sounds more efficient to me. This module shows you how you can set up an inspection feature class for a layer and how to pair the two together. So when I click a hydrant in the mobile map, it copies the geometry and opens a new hydrant inspection record.
- This module works by specifying the asset to inspect, where that inspection goes, and a display label. Like this:
- wHydrant_wHydrant Inspections_Hydrant
- To set up this sample, you will need to add the inspection layers to your mobile map document and recreate the cache with this layer in there. I attached a version of the mobile mxd with this layer in there. Then open the MobileMapTemplate.exe.config file and change the following tags.
- Inspect- tells the application to load the Inspection controls, set it to true
- Inspections – the layers to inspection and their inspection layers
- <Layer to Inspection>_<Result of Inspection>_<Label in dropdown>|Repeat
- LogEditsInspect – This is a backup function that logs all edits to XML, it is not required.
- We also wanted a generic way to show a list of workorders. There are many fantastic workorders systems out there and no way we could build a sample with a connection to all of them. But we can show you how you can load your workorders into a feature class that way an ArcGIS Mobile application can include workorders from any system. All you need to do is write the back end processes to move workorders from your system into and out of this feature class.
- The workorder control is fairly simple. If looks at one feature layer in the mobile cache and loads all them into a display. It is filtered by the crew names, which is changeable. You can open and close workorders, which changes their status. When you open a workorder, it trys to load up the inspection module. So if you do not have this loaded, I hope it does not crash.
- You may notice a some functions for routing and yards. This is a current work in progress, but the idea here is that you can get a workorder or a yard and send it to a navigation control or application. Stay tuned for an update here.
- To set up this sample, you will need to add the workorder layer to your mobile map document and recreate the cache with this layer in there. I attached a version of the mobile mxd with this layer in there. Then open the MobileMapTemplate.exe.config file and change the following tags.
- Activities – tells the application to load the Activities controls, set it to true
- ActivityLayer – Tells the activities control to load workorders from this layer. This layer is the wWorkorder layer in the field operations dataset.
- The rest of the tags- If you are using the feature class that is part of the FieldOperations Dataset, then you will not have to update the rest, if so, update them to match your featureclass.
- Finding the best route to a workorder or an incident is crucial in the field. We are currently working on a sample to show how to call ArcLogistic Navigator from the Mobile Map Template, but in the meantime, we can call a service to get a route. This module calls a Network Analyst Service with a list of stops, runs the route and returns the directions to the field application. Again, not a perfect solution, but shows one option you have for routing your field staff.
- To use this sample, you will need to create a routable network analyst feature class on your street data. This needs to support driving directions. I would suggest testing this in ArcGIS Desktop before publishing it to ArcGIS Server. After testing, publish this map to ArcGIS Server and make sure Network Analyst is turned on as a service type. Note the NA service URL.
- To set up this sample, you will need to add the workorder layer to your mobile map document and recreate the cache with this layer in there. I attached a version of the mobile mxd with this layer in there. Then open the MobileMapTemplate.exe.config file and change the following tags.
- NAServiceURL – tells the application the path the Network Service
Geometric Network Isolation Trace
- Running advance tracing that finds the valves that need to be turned to isolate a main in the field can be extremely valuable. The mobile map template does not support this advance analyst in the field, but it can call a web service to do this. This sample shows how to call a web service that does this advance analyst. The set up can be a little involved if you are not users the data model that is part of this template. Please refer to the ArcGIS Mobile Services – GeometricNetworkRouting.doc in the attached zip for setting this up
- Once you have the services up and running, you will need to adjust the service url in the MobileMapTemplate.exe.config
- NetworkTraceURL – tells the application the path the Geometric Network Service
- Rest of the tags - only change these if your data model differs
There is a new template in the template gallery showing how to use ArcGIS for AutoCAD at a water utility to engageAutoCAD users in GIS data updates.
You can download the template here:
And you can download ArcGIS for AutoCAD here:
How can you benefit from ArcGIS for AutoCAD at a water or wastewater utility?
ArcGIS for AutoCAD helps you solve 2 common problems. The first is sharing GIS content with CAD users quickly without data duplication and wasting disk space. The second is engaging CAD users in your data GIS data updates. The template you can download shows you how to solve the 2nd problem.
Sharing GIS Data with CAD users
Because CAD and GIS are different tools intended for different purposes – GIS to manage, analyze and share spatial data and CAD to create designs for construction plans, they store data differently. To maximize their benefit from GIS, most utilities centralize their data into a single multiuser geodatabase that is then shared among different departments. A centralized geodatabase costs less to maintain, breaks down departmental information silos and help everyone collaborate at utilities. The data in the centralized geodatabase can then be published out to users via a web map, a mobile application and also integrated in with other utility systems such as workorder, SCADA, billing or CIS.
For many utilities, design in CAD starts with data from GIS. A typical workflow for starting a design project would be to export planimetric data (edge of pavement, building footprints, etc), cadastral data (parcel boundaries, ownership information), topography (contours, streams, etc) and utility network data (pipes, valves, manholes, hydrants) from GIS to CAD. To export the GIS data to CAD (either DGN or DWG) geoprocessing tools are used to create the new CAD file with the utility’s CAD standards. Some of the data exported from GIS will be used as background layers in the CAD file while some of the data will be used as starting point for the utility design. Also aerial photos, quad sheets, etc from the GIS may be copied over as image files for use as background data for design. If a utility doesn’t do their own design work they could provide the exported GIS files and imagery to their consulting engineers.
So for each design project a utility undertakes, DWGs or DGNs exported from GIS and file based aerial photography may be stored in the directory for the design. Very quickly you may have many data snapshots exported from GIS and the same aerial photos duplicated many times to be used in multiple design projects over time. So this type of workflow will begin to use up a lot of storage space (especially if designers are using aerial photos) and also is creating a file management headache.
ArcGIS for AutoCAD solves this problem by allowing you to serve maps from ArcGIS Server to your CAD users and eliminating much of the file exporting and data duplication; more efficiently providing data for designers and saving disk storage space. In more technical terms, AutoCAD would be consuming map services from ArcGIS. So for example, you could publish a map to ArcGIS Server that has your parcels, road edge of pavement and building footprints symbolized to mimic your CAD standards and every time you need to show parcels, edge of pavement & building footprints in a design just use the map service with ArcGIS for AutoCAD. The same goes for aerial photos (or another good option is to use ArcGIS Image Server to serve aerial photos up to AutoCAD & Microstation).
Another benefit of using ArcGIS for AutoCAD is that ArcGIS Server is doing the work of generating the map services that AutoCAD consumes, labels can be dynamically created by ArcGIS Server using the GIS labeling engine. As some of you may have experienced, exporting dynamically drawn labels from GIS to a DWG is a multistep process. So for example, ArcGIS for AutoCAD can consume a map service that is creating road and parcel labels on the fly.
Engaging AutoCAD users in GIS data updates
The other benefit of ArcGIS for AutoCAD I wanted to highlight is getting the appropriate data from your designs from CAD into ArcGIS. This is what the downloadable template illustrates. We already explored how design begins with GIS, now let’s explore how it ends with GIS, meaning that when the design is finished you want to get data back into GIS because that is the system of record for most utilities to store their asset data.
So if you are using ArcGIS for AutoCAD to organize your CAD data into attributed feature classes and configured for your utility’s CAD standards & GIS data model than bringing data design data in a DWG will be much easier.
For example, after you’ve created a design for a new sewer main extension, you now need to get the new proposed main features and attributes (material, diameter, etc) back into your GIS and in your proposed features dataset. With the criteria for feature classes configured in your AutoCAD files, you can just open the DWG in ArcMap and then us a geoprocessing tool to append the new main into your geodatabase. In ArcMap the new sewer main looks and acts like a feature class in GIS because of the configuration file that bridges your GIS data model and CAD standards. So for the designer the workflow is to draft the new main on the correct level and populate the correct entity information for the new main (diameter, material, etc).
Really, the idea behind ArcGIS for AutoCAD is make it easier for CAD designers to access data from ArcGIS and more easily pass data back to a utility’s enterprise GIS, yielding a much smoother workflow in a mixed GIS & CAD environment
So, give the ArcGIS for AutoCAD template a try and let us know what you think.
We have updated the Water Operations Dashboard. This version is based on the ArcGIS API for Flex 1.2 libraries and the Sample Flex Viewer that was released on May 29th.
The Water Operations Dashboard is based on the Sample Flex Viewer with a few enhancments.
- Identification Widget: This widget allows the user to click on a asset to get a pop up of the information on that asset. To use this widget in another sample flex app, you need to copy the index.swf and the IdentifyWidget widget. This is because we made changes to the core components.
- ChartWidget/ChartWidgetBar: A widget to provide a bar chart or pie chart. This can be used with the sample viewer or the water dashboard
- LiveMapsWidgetWRefresh: Added code so the live maps can be refreshed at a certain interval
- Handle for null attributes: Added error checking to a few widgets so null values are handled, this should fix some of the 1009 errors you may have encountered(SearchWidget, LiveMapWidget were updated)
You can find the source code on the flex code gallery.
I started typing this blog entry last week. My original idea was to share 5 simple steps to set up the dashboard or Flex sample viewer. Since then, two great things came up, the creating effective web maps seminar and an unbelievable flex guide from a user. Both are discussed below with my original 5 steps.
I recently had the honor of presenting the creative effective web maps seminar. If you did not get a chance to attend this seminar, I strongly suggest reviewing the seminar materials. The seminar focused on creating web maps, just like the operational dashboard. A lot of concepts and design practices were discussed. If you would like to review the materials, you can download the powerpoint and handout from this website.
At the end of the seminar, we demonstrated configuring the Sample Flex Viewer for a parcel notification application. If you are interested in doing this, take a look at the handout that was provided at the seminar. It provides a walkthrough for you. You can find it on the web site provided above.
I would also like to share a very detailed document that one of your users shared with us. Tapas Das, from the Arizona Land Department, wanted to learn the Flex environment so they could upgrade their old ArcIMS based Parcel Viewer to ArcGIS Server. In this process, he put together a very detailed word documents that goes through setting up Adobe Flex builder, using a flex tutorial, debugging flex, and much much more. Again, big thanks to Tapas for sharing this. This is fantastic. If you find this helpful, please thank Tapas.
So to my original idea, 5 steps to setting up the Dashboard.
1 .Determine your Data and Services
I like to think of the data to support my application in three formats. Dynamic, Cached and Other services. When you construct your application, you are going to use a combination of these services. By separating data into a series of cached and dynamic services, maybe some custom overlays, you will achieve the optimal performance. Think about it, if you had all data, orthos, mains, parcels, etc.. in one dynamic service. Each pan and zoom has to query all these layers, label and draw them and send the image down to the browser, but if you separate the orthos and parcels into one cache service. A pan or zoom will just grab the pre-rendered cache service of orthos, parcels, etc. and only query and draw the layers that need to be overlaid, the mains, valves, etc..
When you are trying to split up your data, here are some helpful ways to think about your different services:
Dynamic Map Services – Optimized or Regular
- Real-time data
- Frequently-changing data
- Reporting Layers
- Widget Results
Cached Map Services
- Background data
- Data that does not change often
- Projected on the fly – what I mean by this is that if this data is not in the proper projection, set the data frame to the proper projection and the cache will be build with the data projected.
- Geoprocessing, Routing, Locators, Custom XML services
- Real-time data from other systems
- Results from tables or other systems
When configuration the Sample Viewers, you have three options to display your data. Each of these options gives you a little different control over the data.
Typical these are cached maps. They are drawn as the bottom layer in the Viewer and only one base map layer can be displayed at a time. The user does not have the option to toggle layers in the map on and off. The service is treated as one layer.
Typical these are dynamic maps. They are drawn on top of the basemap and can be stacked on top of each other. Think of each map service as a group layer. If they are dynamic services, the user can toggle different layers on and off.
We refer to these as client side graphics. The information behind them is coming from a dynamic maps services or what I referred to above as other services. If you are using a dynamic map service, the supporting map document is usually a few layers. Symbology and scale is not determined by the map services, so do not spend anytime of this map doc look and feel. The widget will query service, map service or other, filter the results and display a graphic.
This can include a table of information. At this moment the Rest API’s do not support tables. This should not prevent you from creating a web service that connects to a SDE table or third party system, and send those results to a widget as XML
Widgets are also a great place to show the results from a selection. We typically call these selection sets in desktop GIS. We can display them in the standard grid, but try to think about the results in a different manor. Can we use a chart to display them? Is there any way to rely the same info in a more intuitive way?
2. Set up the Flex Viewer
Set up files in IIS
- Copy the Unzipped flex application to a folder on your Web Server.
- Create a virtual directory for this application
- If you need help with this section, follow the section in the templates help files
- Apply CrossDomain.xml – to access data from a different server than the one hosting your Flex application, the remote server needs to have a cross-domain file in the root directory. For security reasons, the Web browser cannot access data that resides outside the exact Web domain where the SWF file originated. However, Adobe Flash Player can load data across domains if permission is granted from the server. This is accomplished by including a small crossdomain.xml file on the remote server that permits Flash to connect to services on that server. http://resources.esri.com/help/9.3/arcgisserver/apis/flex/help/index.html#references/using_crossdomain_xml.htm
3. Determine Widgets
The Flex viewer template comes with some widgets for typically web mapping functionality. Some of them are listed below.
- There are more widgets posted on the resource center for the Flex API.
4. Update the Configurations File – Main Viewer
Open Config.xml in the Sample Viewers Root & Change the following tags
- <Title> and <Subtitle> – This is the text in the bar at the top
- <Menus> – Menus are the drop downs in the banner bar. You add will add tools, widgets and layers to these.
- <Map initialextent…..> – Adjust these extents in the spatial reference of the viewer
- <Basemaps menu=”menuMap”> – Add the basemap services. These services are display vertical on menu listed
- <LiveMaps – Add the operational layers services.These are displayed in the LiveMapsWidget and are displayed on a floating widget. Make sure to leave the LiveMapsWidget in the widget section
- <Navtools> – By Default, all navigation tools are listed
- <Widgets> – Add all the widgets you need in your application
5. Update the Configurations File – Widgets
Check and see if your widgets have configuration files. If they do, open them up and adjust the configuration for each.
We’ve had a few questions about whether users and ESRI business partners can submit templates to the Water Utilities Resource Center. We wanted to tell everyone, the answer is yes!
We are encouraging our users and business partners to submit templates to the Water Utility Resource Center template gallery.
Keep in mind there are 5 items that a template must have and all of these items must be in your template zipfile:
1. An instruction document – with information on how to install and configure the template. Including what software is necessary.
2. Any MXD or MXDs necessary – the mxds are critically important to show everyone your cartography, geoprocessing tools, etc.
3. Any custom code – Including the source code.
4. Sample Data – a populated geodatabase with any necessary data for your template. This is so everyone can understand how your template works with sample data. If you can’t share your own data than you can use the sample data we’ve provided with the Mobile, Editing or Dashboard templates for your template.
5. A blank geodatabase – this is an empty geodatabase with the same schema as your sample geodatabase.
Here is an example of the folder structure your template should follow:
A few ground rules for submitting templates – we will review each template to ensure that the proper items are in them and they function as advertised. We won’t accept any templates that have trial software applications or extensions in them or don’t have the source code if there was custom programming in your template.
So anyone ready to share your good work?
If you have any questions about how to create your own template and post it email us at ArcGISTeamWater@ESRI.com
Yesterday I participated in an all day seminar on Asset Management and GIS at the New Jersey Water Environment Association conference. The seminar had some good presentations on deploying Mobile GIS at water and wastewater utilities. The Mobile GIS presentations and the questions from the seminar attendees echoed what we have seen firsthand at many customers. Deploying Mobile GIS to the field is not that hard with the right technology, software and data. The challenge for most utilities is making field crews comfortable with technology and breaking their dependence on paper maps.
In other words, you should use a change management process to make your field crews comfortable with Mobile GIS. We’ve seen great success from utilities that have taken a proactive approach and engaged the hearts and minds of their field crews as part of deploying a mobile GIS solution. We’ve also seen some W/WW utilities that have a business need to deploy mobile GIS but are so nervous about field crew acceptance they are afraid to move forward.
So here are some successful steps we’ve seen W/WW use to get field crew comfortable with mobile GIS:
- 1. Explain the value of mobile GIS to field crews – start with explaining how mobile GIS will make their jobs easier – they’ll have more up to date information in the field, they’ll have better information in the field (beyond the labels on their paper maps), they won’t have to fill out paper forms, etc. Then explain the value of mobile GIS to the entire utility – save costs, increase field crew safety, give us better information in the office, use less paper, etc. Ensure that all of your field crews know how mobile GIS is a solution that benefits them and why it is important for the utility as an organization.
- 2. Show field crews the technology – You can use the Water Utility Mobile Map to demonstrate mobile GIS to your field crews. This should help them understand that mobile GIS isn’t complicated for them to use. Also engage them for their feedback, what do you they like about the mobile application, what don’t they like. You should also let field crews give their preference for the mobile device in the field (if you haven’t standardized on one yet). Let them see a rugged tablet, versus a laptop or a smart phone.
- 3. Keep it simple – We’ve learned that you don’t want to throw the kitchen sink at field crews when they are starting with mobile GIS. You should introduce some basic functionality and then expand that out over time as they become comfortable with the look and feel of mobile GIS. That is why we created the WRC Mobile Map as a simple application to start with.
- 4. Do a pilot – Deploy the mobile GIS application with your data to a few of your field crews and get their feedback. Ideally you’ll do the pilot with a crew that is more comfortable with technology and one that is less comfortable. This will give you a good idea how to train the rest of your field crews.
- 5. Be prepared to support the mobile rollout – have your workflow to support mobile GIS ready to go & define how you will support the mobile field crews. The actual rollout needs to be geared toward making everything easy for the field users.
- 6. Do a hands on training session – based on what you learned in the pilot, do a hands on training session for the rest of the field crews. Give them a 1 page laminated cheat sheet of how to use the application. Have the pilot crews share their experience. Plan on a doing a refresher training session a week or two after deployment.
- 7. Set a “sunset” date for your paper field map books – let the field crews know you will no longer give them new paper map books a few months after the roll out. All updates will be digital and on in their mobile GIS application. It is important to note that you will still be printing out paper maps for the field as needed, but you will no longer be printing out up
- 8. Celebrate your successes – recognize crews that are excelling with the mobile application. Reward crews that are passing a lot of data back into the office and show them how you are using the information they are capturing in the mobile application.