Tag Archives: Geoprocessing
You’re probably familiar with using feature classes, feature layers or feature sets as parameters of geoprocessing tools in your Python code, but you may not be aware that you can also use instances of Geometry, MultiPoint, PointGeometry, Polygon and PolyLine objects in exactly the same way. This can save time creating feature classes that you really don’t want to persist, for those times you need a geometric property or method momentarily in your script. Here is a snippet that illustrates the concept (in real life we would take care to manage the spatial references of geometries too):
#Make a 5000m buffer around a sample point
SamplePoint = arcpy.Point(1765863,5926652)
SamplePointGeometry = arcpy.PointGeometry(SamplePoint)
outGeom = arcpy.Geometry()
outGList = arcpy.Buffer_analysis(SamplePointGeometry,outGeom,”5000 Meters”)
print str(outGList.area) + ” Square Meters”
78539816.3397 Square Meters
Note that when a Geometry object is given as the output “feature class”, tool outputs are Python lists of geometry features. Therefore, as above, you will need to specify an index position to retrieve a feature.
A powerful way to use Geometry objects in this way is within cursors, to operate on a feature class geometry with row granularity.
Content provided by Bruce Harold
One of the changes for Beta 2 is that the Python window automatically imports arcpy, so you no longer need to perform this step. For standalone scripts and for scripts used in script tools (both validation and execution) you still need to import arcpy, because these may execute outside the desktop process environment. Within the Calculate Value model tool and wherever Python expressions are used within Desktop dialogs, expressions do not need to import arcpy, as it inherits from the application.
Tip provided by Bruce Harold
ArcPy is a site-package that builds on (and is a successor to) the successful arcgisscripting module. Its goal is to create the cornerstone for a useful and productive way to perform geographic data analysis, data conversion, data management, and map automation with Python.
This package provides a rich and native Python experience offering code completion (type a keyword and a dot to get a pop-up list of properties and methods supported by that keyword; select one to insert it) and reference documentation for each function, module, and class.
The additional power of using ArcPy within Python is the fact that Python is a general purpose programming language. It is interpreted and dynamically typed and is suited for interactive work and quick prototyping of one-off programs known as scripts while being powerful enough to write large applications in. ArcGIS applications written with ArcPy benefit from the development of additional modules in numerous niches of Python by GIS professionals and programmers from many different disciplines.
Click here for a more information on ArcPy
I get asked a lot about working with KML in Desktop… The Geoprocessing Team just made a post on their blog about a new tool called “KML to Layer“.
So if you work with KML, make sure and give this new tool a try and let the Geoprocessing team know what you think.
We continue to put more videos on the Desktop Resource Center Video room and are open to suggestions so let us know via a comment if you there is a topic or area of the software that needs a video demo.
One of the most popular and widely used extensions to ArcGIS Desktop is Spatial Analyst so here are a few videos that highlight what’s new in Spatial Analyst 10
In case you haven’t noticed, there are some really good video showing some enhancements to the Geoprocessing framework at 9.4 and demos of the new scripting capabilities using Python.
You have found the ArcGIS Desktop 9.4 blog but here are a few other blogs you’ll want to keep an eye on…
Here on the Desktop blog we’ll cover all things Desktop which includes all the Desktop extension, Desktop customizations and building custom desktop application with ArcGIS Engine.
One of the goals of 10 is to help make you more productive with the software. We are all busy and under lots of demands at work but here are some new features at 10 that we think will help. Continue reading
As you may have seen, we released the Water Distribution Capital Improvement Planning (CIP) Template a last week. First, we wanted to say a big thank you to all of our users and business partners who helped us to refine the initial geoprocessing models and the toolset also shared their workflows for capital planning.
We’ve already had a few questions about why we chose the term Capital Improvement Planning (CIP) to describe this template, since not all utilities use that term. So when we use the term CIP, what we mean is the long term plans of a utility to manage their assets and/or to expand their system, what you may also call a “Capital Plan”, “Long Term Plan” or “5 year plan”.
Personally, I think the CIP Template is great example of how ESRI listens to our water utility customers and responds to their needs. We’ve had numerous customers over the past few years tell us that they want to be able to leverage their asset data in GIS as well as their operational data (workorders, CIS, water quality) better to support their long term plans. Of course, we thought that giving our customers a geographic view of all that asset and operational data was the best place for them to start. We also heard from many of our water and wastewater customers that their long term planning has evolved from an occasional event to a continual process; because of funding issues, grant availability, coincidence with other projects that a utility could share costs with and the desire to be quick and proactive to eliminate the risk of future critical asset failures.
Also, we are excited, because the CIP template is great example of GeoDesign. We’ll be doing a blog shortly that explores the principals of GeoDesign and relates them back to the CIP template.
2 Parts of the CIP Workflow
As we dug into the CIP process, we observed 2 distinct, but related workflows happening. The first part of the workflow was to assemble data from many sources and analyze that data to look for where projects are needed. This part of the process is tailor made for the benefits of GIS – to use GIS as the place where different types of data are assembled together into a common view and also to use the analytical capabilities of GIS to gain better insight into the aggregated data. Because this analysis needs to be iterative (looking at multiple data layers with different weighted criteria), an auditable process (you have to be able to defend your findings to a PUC and your ratepayers) and an automated workflow (to save time, money and resources) this is a perfect match for Geoprocessing Models in ArcGIS.
GIS Analysis for CIP Decision Making
At first we took the approach that ESRI should try and build a few geoprocessing models that all water and wastewater utilities could use to score and rate their assets by estimated remaining asset life, condition or criticality. We figured that we could do some research, interview some of our users and figure out these geoprocessing models (our inner geography geek begged us to take this approach first). What we quickly realized was that there isn’t a silver bullet set of geoprocessing models we could build because every utility system has their own approach to long term asset management and their own priorities (KPIs, level of service they want to provide, hot button issues, fiscal condition, etc) that drive their long term planning.
This was also a great reminder that even though we have the ability to use technology to automate a process, the human element is still critical, meaning that the more we talked with the engineers who are creating these CIP plans, the more we realized they need a better way to manipulate and process data so they could apply their engineering expertise to make decisions about capital projects. We also noticed when talking to engineers doing capital planning, that while they were somewhat aware of the analytical capabilities of GIS, they weren’t aware of the geoprocessing framework core to ArcGIS and how to use ModelBuilder to automate analysis and create a reusable toolset.
So we decided that we need to focus our CIP template on showing the water utility community how they could benefit from automating spatial analysis with the ArcGIS geoprocessing framework by providing some generic models. So, please keep in mind that the intent of the models we’ve provided in the CIP template is to show you how geoprocessing and ModelBuilder work within ArcGIS so you can create geoprocessing models that reflect how your utility wants to manage assets and plan for the long term. Incidentally, if you want to learn more about GIS analysis, Geoprocessing or Model Builder within ArcGIS, ESRI has lots of great resource including on-line training, books and class room instruction.
Estimating Project Costs
The second part of the CIP workflow we observed was estimating CIP project costs. Basically this workflow was estimating the cost of a project based on either replacing existing infrastructure or adding new infrastructure (main extensions, interconnections, extending service to new sub-divisions, etc). It’s important to note that all of the functionality in this part of the CIP process is core to ArcGIS and the geodatabase, all we’ve done is customized the application to automate and simplify this part of the workflow. This is what we decided to call the Costing Estimating Tools.
The first step in estimating project costs is to create projects by grouping assets together into projects. In this part of the process you are visualizing the data you brought into GIS and also the results of your analysis and then determining what assets you want to include in a project, your rehab or replacement strategy for those assets and then saving that information. So you are literally visualizing data in GIS (most likely working with many data layers of data, including the same feature datasets symbolized different ways) and doing some spatial and attribute queries to come up with candidate assets to include in CIP projects.
From there, assets that are in need of replacement or rehabilitation and spatially close to together are grouped in projects. We’ve heard from many water utilities that without a spatial context it was a real challenge for them to group assets together into appropriate projects without and also it was a challenge for them to track and manage information about candidate assets for CIP projects throughout the CIP planning process. Water utilities were struggling with supporting their CIP process with paper maps and tracking assets that were part of a project, including costs to replace those assets, in spreadsheets.
So traditionally, this CIP process took a lot of staff time and also lead to uncertainty about whether utilities were actually spending their money on the most appropriate capital projects. We also heard that utilities were struggling with how to update data when they tried to refine a large candidate list of CIP projects down to just a few to carry forward into design and that it was next to impossible to look at multiple scenarios for the same project area (assets grouping and rehab or replacement approach) because so much of this process was manual or spreadsheet driven.
We took the approach that if a utility has their assets (water distribution, wastewater collection or stormwater) in GIS, they should use their GIS asset data to group into CIP projects and then to store information about the CIP projects (like the extent and also all of the assets that are part of the project) as new data layers in GIS. This enables a utility to create an authoritative source of data about their proposed capital projects in GIS. So this drove us to create the Cost Estimating Tools.
As we began to demonstrate early versions of the Cost Estimating Tools to our utility users, we got a lot of great feedback that helped us to refine the tools. We were told that to be really useful, the tools should include the ability to either rehab or replace existing assets and to extend mains, so we programmed that functionality into the tools. We also were told by our users that they needed to be able to compare the costs of different replacement strategies (open cut, trenchless, etc) for the same set of assets so we designed the tools to make it easy to compare the costs of use using different rehab methodologies. Also we knew that the costing element of the tools needed to be flexible, because individual utilities favor different pipe materials which can be set as defaults and that unit costs are often specific to a utility and those can be easily configured in a simple table.
So what we wanted to do with this blog was to explain how we arrived at version 1 of the Water Distribution CIP Template. We are very interested in your feedback so we can incorporate more useful changes in version 2. Also we’d like to hear about any geoprocessing models that you would like to use for CIP planning. So, please leave us feedback here – http://forums.esri.com/forums.asp?c=55&s=426#426
In the next few weeks we’ll be recording a video of the Water Distributions CIP Template in action and we are also going to do a webcast in December that takes a deep dive into the CIP Template.
ArcGIS 10 is a major release of all aspects of ArcGIS and is designed to help you perform your GIS work faster. Some of the highlights include the following:
Perform Your ArcGIS Desktop Work More Efficiently
- Faster, more responsive drawing performance including smooth, continuous panning of your data
- Easier access to most commonly used geoprocessing tools
- New Search window in ArcMap to let you quickly locate maps, data, and tools
- Catalog window built into ArcMap for quick data access
- Easier and faster ways to find and use symbols and tools
- Auto hide and dockable windows (e.g., table of contents) so your focus remains on the map
- Ability to execute geoprocessing in the background, allowing you to continue to interact with your map
- Automation of additional workflows with Python (maps and layers)
- Easy-to-use software developer kits (SDKs)
- Single-line simplified geocoding
But there is more in 10 so stay tuned…