Category: Python

Welcome to the new unified ArcGIS Blog

For the past few years the various ArcGIS product development teams have hosted a few dozen separate blogs covering the width of the ArcGIS system. Now we have pulled those together into a single ArcGIS Blog so that you can more easily browse, subscribe to, learn from, and stay up-to-speed on the latest information from all our engineers and developers. In addition, the single blog reflects ArcGIS as a system and allows us to better tell big picture implementation stories that we couldn’t in the fragmented system.

If you do want to narrow your focus, just click a Category (under Technical Communities and Industry Communities) or Tag to see the posts for that particular area. You should find a category or tag that matches up with the subject area for each of the old separate blogs. A few may have been overlooked or thought to be unnecessary. We appreciate your feedback on specific categories or tags that should be added to help focus in on subject matter that is of interest to you. Continue reading

Posted in 3D GIS, Analysis & Geoprocessing, ArcGIS Online, Defense, Developer, Editing, Hydro, Imagery, Local Government, Mapping, Mobile, National Government, Public Safety, Python, Services, Spatial Statistics, Telecommunications, Water Utilities, Web | Tagged , , , | 5 Comments

The New ArcGIS Discussion Forums MVP Program

Have a question?  Want to talk GIS? Looking for some good ideas?  Jump into the conversation!

A few weeks back we rolled out some new features that we hope will improve the forums’ usefulness.  One of the new features is the ability for those who post questions in the forums to rate the replies.  In addition, any logged in user can also join in and vote on the various replies and conversations in order to help us identify the best information, helping you find good information more quickly.  Another huge benefit to the new voting tools is that it helps us all identify, recognize, and reward those forums users who contribute the most helpful information.

Until June 2010, our old discussion forums (now archived) supported what we called the “Forums MVP Program”.  Once every 6 months we would identify the Top 10 members of the community and make them eligible for MVP recognition and some valuable prizes. Continue reading

Posted in 3D GIS, Analysis & Geoprocessing, Developer, Editing, Geodata, Imagery, Mobile, Python, Services, Spatial Statistics, Web | Tagged , | Leave a comment

New Features for the ArcGIS Discussion Forums

Have a question?  Want to talk GIS? Looking for some good ideas?  Jump into the conversation!

A few weeks back we rolled out some features that we hope will improve the forums’ usefulness.  To be candid, we’re sure many would say that some of these features should have been there since day one, or at least long overdue, but we listened to what you wanted and found a way to get them in there for you to use.

Let’s start with the big one…

Community Voting
If you read something and you like it, give it a thumbs up!

Now any forum user can let the community know where the good information is. How you do it is simple. If you read a post or a reply and you think it contains some really good information, click the “up” arrow on the right. This is similar to the “Like” button in Facebook.

If we all do this, then the best posts will bubble to the top. If you want to read the entire thread you can, but if you need to most quickly find the good stuff, look at the posts that have a high score.

There is also a “down” arrow, but you can only use that to “Unlike” something you previously Liked.

If you find “The Answer”, give it a check!

If you started a thread with a question, then whichever reply you think is the best, give it a check. That will mark your thread as “answered” for all to see, and it will give some MVP points to the person who wrote it.

Two birds with one stone
Clicking the checks and arrows has two benefits:
1.    You are helping everyone find the best information.
2.    You are helping everyone recognize the best contributors.

One of the great things about any community is the trust earned by those folks who share their experience and help others. Some pros out there are always going to stick out, but these new voting tools are going to help find others who are just as helpful and useful who you might not yet know.

Discussions versus Q&A

When you start a new thread, you can let everyone know if you’re starting a discussion or if you’re asking a specific question. Discussions show up in the thread list with a yellow “D” icon and questions with a red “Q”.

When the original poster or a moderator identifies one of the replies as the best answer, the “Q” icon turns into a green “A”.

This helps you find answers more quickly, and if you want to jump in and let everyone know what you think, this helps you more quickly find questions that haven’t been answered yet.

Private Messages
Most of the time an open public discussion is a great way to get the best information, but sometimes you may want to take it off-line.

Up on the menu is a “Private Messages” link. Click that to see your Inbox or to send direct messages to other users. Also, clicking their name anywhere in the forums provides a pop-up you can use to send a message to them if they’ve activated it.

You can use the “Forum Actions > User Control Panel” menu to control who can send you messages.  You can turn it completely on, completely off, or limited to just those users in your Contacts list.

New Badges
Badges are a great way to find those folks who have been around the block a time or two. Anyone with more than 200 posts in the forums becomes a “Senior Member”, and of course anyone on the forums who works for Esri is badged as “esri” with a globe logo.

And when you see someone with a “Forums MVP” badge, you know you’re talking with someone the community has said has the best information and is most dedicated to your success. These are folks who have been voted by the community to be in the Top 10 of all contributors during any previous six-month MVP rating period. And once you’re an MVP, you’re always an MVP.

And more importantly, now that the new community voting tools have been included, it’s time to roll out the new MVP program. Watch this blog post next week for a description of the new rules, new standards, and a list of awards you can earn through all of your good effort. So jump in and help us figure out who the players are; maybe it’s you!

What’s Next?
The Advanced Search page gives you a lot of flexibility to build a complex search.  What we’re working on next is giving you the ability to save that search definition. This will be good for bookmarking and sharing. We’ve also heard that most users participate in some forums a lot, some forums a little and others not at all. We are going to improve your “What’s New” page so that it only includes those forums you want to browse. If there are any other improvements you’d like to see, reply here, or jump into the conversation on the Resource Center Site Feedback forum.

Content for this post provided by Jim Barry

Posted in 3D GIS, Analysis & Geoprocessing, ArcGIS Online, Developer, Editing, Geodata, Imagery, Mobile, Python, Services, Spatial Statistics, Web | Tagged , , , | 1 Comment

Implementing topology through Python scripting

Geoprocessing can be used to automate many aspects of data compilation, including implementing a geodatabase topology. In my previous post, I created a model that imports data into a file geodatabase and performs initial data cleanup processing. Now, I will use geoprocessing with Python scripting to build a topology on the imported data. Although many spatial integrity issues were resolved by running the model, a geodatabase topology can help check for and repair any remaining errors.

The dataset I am using is a feature class of parcel lot line boundaries that began as a CAD file full of topological inconsistencies, such as lines that overlapped or did not connect to other lines. Once I create the topology and am sure the lines are correct, I will build parcel polygons from the lines and introduce all the features into my production enterprise geodatabase.

Creating a topology in a script
To build topology in an automated manner, I am going to write a simple Python script, add it to a toolbox, and run it as a regular geoprocessing tool. Essentially, the script just performs the functions of the New Topology wizard, but without requiring my intervention. In fact, this script could be developed without any programming by creating a model to perform these tasks and exporting it from ModelBuilder as a Python script.

When scripting, the ArcPy site package allows Python to access and run any of the geoprocessing tools in the ArcGIS system toolboxes, including the topology tools. The Topology toolset in the Data Management toolbox contains all the tools I need to add a geodatabase topology to the line feature class. The import arcpy statement adds ArcPy to a script and is at the top of every ArcGIS Python script.

Before I start adding the tools to the script, I define variables for the paths to the feature dataset, feature class, and topology. Because I created folders to store my tools and data following the recommendations in A structure for sharing tools, I can make the script more portable by setting these paths in relation to the folder containing the script. Since the folder locations are not hard-coded to match the C: absolute path of my hard drive, the script should run on someone else’s machine regardless of the machine’s directory structure.

With the paths defined, I first use the Create Topology tool to add a topology to the feature dataset. The syntax for the tool in my script is arcpy.CreateTopology_management(featureDataset, topologyName, “”), where CreateTopology is the name of the Create Topology tool,  _management is the toolbox in which it resides, featureDataset is a variable I defined earlier representing the path to the feature dataset, and topologyName is a variable for the name of the topology. When working with topology, it is recommended to use the default cluster tolerance, which is the distance in which vertices are determined to be coincident. Since I want ArcGIS to calculate the cluster tolerance, I left the value blank as “”instead of supplying one.

Although that function creates a topology, it is currently empty and has no classes or rules in it. I can use the Add Feature Class To Topology tool to add my line feature class with the statement: arcpy.AddFeatureClassToTopology_management(topology, featureClass, “1″, “1″). If the topology contained multiple feature classes, I can set ranks so the feature class with the highest accuracy is not adjusted to match vertices in a feature class which is known to be less accurate. However, since I am using only one feature class, I leave the rank parameters as “1″.

Next, I set which topology rules to include by calling the Add Rule To Topology tool. When choosing which rules to add to a topology, there are a few rules that many editors commonly add to every topology, such as the line rule for Must Not Have Dangles. In the script, this rule is coded as arcpy.AddRuleToTopology_management(topology, “Must Not Have Dangles (Line)”, featureClass, “”, “”, “”). Because the rule only applies to one feature class that does not have subtypes and is not a rule between two feature classes or subtypes, the other parameters are left blank. I also want to make sure that the lines do not overlap or intersect themselves, so I include the Must Not Overlap and Must Not Self-Intersect rules as well. If I need to add more rules later, I can run the Add Rule to Topology tool or use the topology’s Properties dialog box in the Catalog window.

Once all the feature classes and rules have been added, my script validates the topology. The Validate Topology tool identifies features that share geometry, inserts common vertices into features that share geometry, then performs integrity checks to identify any violations of the rules that I defined for the topology. Since the topology has never been validated before, I am going to validate the entire extent of the data with the syntax, arcpy.ValidateTopology_management(topology, “Full_Extent”). However, when working in ArcMap, validating the visible extent of the map instead of the full extent limits the area to be validated and can be useful for very large datasets that take a long time to validate.

Finding and fixing topology errors
After the script runs, I add to ArcMap the resulting topology so I can inspect the results and fix any errors using the ArcMap editing tools. However, with all the previous automated QA work from my Import and Clean Lines model, the remaining manual edits are minimal in comparison to what they could have been without running it first.

The errors identified with topology are indicated by the orange-colored squares. Almost all of these are dangle errors that could not be fixed by the original model since they exceeded the tolerance value for the Extend, Trim, or Snap tools. The topology found only two Must Not Overlap errors, which I can fix by deleting one of the overlapping features. There are no violations of the Must Not Self-Intersect rule, indicating that the lines were split properly in the model. Using the editing tools on the Topology toolbar, such as the Error Inspector and Fix Topology Error tool, I can review each error to determine if the built-in topology fixes can be used or if the lines should be edited manually to resolve the topology error.

In some cases, the topology error may need to be marked as an exception, which is a valid violation of a topology rule. One of the most common examples of exceptions to the Must Not Have Dangles rule is a cul-de-sac road, which are dead ends that do not connect to other roads. However, when working with parcel lot lines, there are fewer scenarios that are valid exceptions. I do have some lines at the edges of the dataset that do not connect to other lines. I can either mark these as exceptions or choose to delete the features, depending on whether these features are supposed to connect to existing features in my enterprise database.

If an edit is made to correct a topology error, I have to validate the topology again to make sure the error no longer exists. After I perform a visual inspection and fix all the remaining topology errors, I can create new polygons representing landownership parcels using the geometry of the lines. If I attempted to create polygons from lines that do not connect to each other properly, either no polygons would be created or one large polygon would result where there should actually be two polygons. After creating the new polygons, I add the polygon feature class to the topology and check for any gaps or overlaps and make sure the parcel polygon boundaries are always coincident with the lot lines.

After implementing the topology and making edits in ArcMap, the lines and the polygons created from them meet standards for our spatial data. I can now introduce the features into the production enterprise geodatabase.

While a script tool or model may take time to set up initially, in the long run, it is quicker to automate data compilation tasks through geoprocessing whenever possible. I can run a tool as needed and re-run it later with different parameters and tolerances or apply it to other datasets. Scripts are particularly useful because they can be run at specified times as a scheduled task in Windows. For example, I could combine the tools presented in these blog entries into a Python script that imports a dataset into a geodatabase, processes it, and implements topology. If I set the script to run automatically in the evening after working hours, I am ready to start editing on clean data when I come into the office the next morning.

For more information:
The sample tools and data can be downloaded from the Editing Labs group on ArcGIS.com. An ArcInfo license in required to run the tools.

Content provided by Rhonda (Editing Team)

Posted in Analysis & Geoprocessing, Editing, Python | Tagged , , , , , , , , , , | Comments Off

ArcGIS 10 (Desktop, Engine, Server) Service Pack 3

ArcGIS
10.0 Service Pack 3 is now available to download via the ArcGIS
Resource Center

Here is the link to SP 3 for ArcGIS 10 (Desktop, Engine, Server)

Posted in Analysis & Geoprocessing, Developer, Editing, Geodata, Imagery, Mapping, Python, Spatial Statistics | Tagged , , , , , , | 9 Comments

New Resources Available for Getting started with Python Map Automation

ArcGIS 10 provides a new Python
mapping module (arcpy.mapping) that allows you to interact with the contents of
map documents and layer files without necessarily needing to interactively open an
ArcMap session.  The methods, properties and functions available in this
new map scripting API enable you, for example, to automate changing data
sources, modify layer properties, export and print maps, as well as automate the
creation of
thematic maps and map series.  Because the new mapping module is part of
the ArcPy geoprocessing framework, scripts can be used within ArcGIS Desktop
but can also be published to ArcGIS Server as geoprocessing services making
it much easier to make mapping and printing capabilities available on the
server.

The following links are resources
that will help you learn more about arcpy.mapping, get access to popular
sample scripts available for download, and
links to new training resources:

Help Resources

  • A new Introduction to
    arcpy.mapping
    help topic is a great starting point. It includes
    links to a new arcpy.mapping tutorial and general guidelines for working with
    arcpy.mapping. ArcGIS Desktop help has a complete section
    dedicated to the ArcPy mapping module. Embedded within the help topics
    are over 100 different, practical help samples that can be copied/pasted into your applications. Be
    sure to review the “Best ways to get started” section.

Sample Applications

  • Approximately 20
    script tools
    that perform routine map and layer management tasks, printing and
    exporting, as well as basic cartographic operations.
    This is an excellent download because it
    includes many practical code samples that perform a variety of tasks and they
    are easy enough to modify for your own purposes.
  • A script
    tool
    that combines Data Driven Pages, arcpy.mapping, and the ReportLab site package to generate a reference map book that includes street index
    pages. It demonstrates how arcpy.mapping can be used to extend Data
    Driven Pages capabilities.
  • A script
    that incorporates Data Driven Pages and arcpy.mapping to build a map series
    that includes dynamic graphic tables. There is a very complete README.doc file
    included with the download that also addresses other useful tips and tricks
    that go along with the application.

 Web Courses

  • Basics of Python (for ArcGIS 10).
    This course teaches fundamental concepts you need to know to create
    Python scripts in ArcGIS. You will learn guidelines for proper Python syntax,
    techniques to troubleshoot common errors, and how to use loops to test for
    conditions and execute different code based on the result.
  • Python Scripting for Map Automation in
    ArcGIS 10
    .
    This course teaches how to automate map production and
    related data management tasks that would be time-consuming and tedious to
    perform manually. You will learn how to work with the mapping module of the
    ArcPy site package to quickly and easily update map layers and map document
    properties, modify map content, and produce individual maps and map books.

 Special thanks to David from the Mapping Team for sharing these links.

Posted in Developer, Python | Tagged , , , , , , , , , , , , , , | Leave a comment

Creating Map Books using Data Driven Pages

A few weeks ago, thousands watched a live training seminar on the new Data Driven Pages in ArcGIS 10.  If you missed this excellent live seminar you can watch it on demand from Esri Training.

Description
Using the ArcGIS 10 Data Driven Pages feature, you can quickly and easily create a professional-quality map book from a single map document. This seminar teaches the workflow for using Data Driven Pages. The presenter also covers how to create an index layer from a feature layer and add dynamic text and locator maps to your map pages.

Who Should Attend
GIS professionals and cartographers working in utilities, transportation, public safety, and government mapping agencies and others who need to produce map books.

Key Points
The presenter discusses

  • Data Driven Pages, map books, index feature extents, and geoprocessing tools.
  • The process for building map books.
  • Updating, printing, and exporting map books.

 http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2152

Posted in Developer, Mapping, Python | Tagged , , , , , , , , , , , | 6 Comments

PyScripter — free Python IDE

There’s a new freeware Python IDE that’s grabbing our attention. It’s called PyScripter and you can download it here:

 http://code.google.com/p/pyscripter/

For more information on Python IDEs, see the blog post Review of IDEs for Python

Posted in Analysis & Geoprocessing, Python | Tagged | 8 Comments

Review of IDEs for Python

A common question we get from our Python user community is “which Python IDE is best?” or “which IDE do you guys use?” The answer we’ve given over the years has varied depending on whom you asked and the evolving Python IDE landscape.

We at Esri haven’t had the time to do a review of what’s out there but since we know a lot of you are interested in this topic, you may want to read the infoworld article: “Nine fine Python development tools” http://www.infoworld.com/d/developer-world/infoworld-review-nine-fine-python-development-tools-374

Kudos to Jason Scheirer for finding this article, Ghislain Prince for writing the post, David Wynne for pointing out that I had misspelled Ghislain’s name in the first post, and Ghislain for pointing out that he didn’t actually find the article, Jason did, and he (Ghislain) only forwarded me the email from Jason after adding the blog post details, but David didn’t know that when he caught the original misspelling.  My goal this week is to make this note longer than the actual post…

Posted in Analysis & Geoprocessing, Python | Tagged , | 9 Comments