By Kenneth Field (Research Cartographer), Damien Demaj (ArcGIS Online Cartography) and Linda Beale (Geoprocessing)
At the 2012 Esri Education GIS Conference and the 2012 Esri International User Conference, we demonstrated how you can build informative thematic maps using the ArcGIS System. The purpose of the sessions was to take relatively simple datasets and create a range of alternative thematic map types that told a story in different ways. This demonstrated the techniques for creating the maps using ArcGIS for Desktop as our authoring environment and ArcGIS Online as our publishing mechanism. As the XXX Olympiad is currently taking place in London, UK we illustrated how alternative maps can be made to tell different stories of the relative success of nations over the period since the first Olympic games in 1896. Continue reading
The Well Count Aggregation map template for ArcGIS 10.1 is available for download. This template was designed to allow you to create your own version of the Well Count web map using the provided sample data as well as your own data. The ZIP file contains map documents, sample data, an ArcGIS style file, and documentation. The Well Count map shows how a complex data source, such as gas wells in a producing field, can be aggregated and portrayed in an easy-to-understand manner. Continue reading
By Jaynya Richards, Esri Research Cartographer
We just added to Esri’s Mapping Center ArcGIS Resources – Styles page a style that contains newly updated cartographic symbols for the National Park System (NPS). Tom Patterson, NPS Harper’s Ferry Center (HFC) cartographer, shared the symbol files with us, and we used them to create this ArcGIS 10 style, its five associated Esri fonts, and three associated graphic files. With this style, you can use the most current NPS symbology on your maps.
ArcGIS 10 introduced the concept of editing with feature templates, which define a new feature’s symbology and default attribute values, among other properties. Anytime I want to add a feature, I use the Create Features window, which displays a list of available feature templates and tools for creating new features.
Sometimes, though, I do not see the template I want to use in the Create Features window. This could be because there are no templates for the layer, but it could also be that the template exists but is being filtered out of the Create Features window. The underlying philosophy for determining whether ArcMap shows a feature template is that new features created with the template must be visible after creation. Therefore, templates are hidden whenever new features would immediately disappear and not be displayed on the map.
While a layer being turned off is one of the more obvious reasons why feature templates are not shown on the Create Features window, layer definition queries can be subtle causes. A definition query displays only the subset of features that match an attribute query defined on the Layer Properties dialog box; the remaining features are not drawn on the map or shown in the attributes table.
This post provides an overview of and best practices for the use of definition queries while creating features.
The National Geographic basemap has now been added to the File > Add Data > Add Basemap dialog in ArcGIS Desktop 10. It has also been added into the list of basemaps that appears if you choose the File > Add Data From ArcGIS Online command in ArcGIS Desktop 9.3.1.
In the Add Basemap dialog, the Shaded Relief basemap entry has been removed in order to make space for the National Geographic basemap and keep the number of basemaps to 12. However, we have updated the Terrain basemap so that it includes the Shaded Relief service. So if you want to add shaded relief to your map, choose the Terrain basemap, and then in the Table Of Contents you can choose between the terrain service and the shaded relief service, whichever looks best for your map. The layer also includes reference overlays that you can turn on.
In ArcGIS 10, feature templates define all the information required to create a feature, such as the target layer and the attributes the new feature will receive. Most of the time, you interact with feature templates on the Create Features window. However, some editing operations present a dialog box containing templates that you can choose for the output for that particular feature creation workflow. For example, the Select Feature Template dialog box used with the Editor menu > Buffer command allows you to set the feature template that will be used to create the new buffer feature. The list of templates is filtered to show only editable and visible polygon or line layers in the map; feature templates for other templates are hidden and cannot be used for the buffer output.
If you are developing a custom command or tool that requires the user to specify a feature template for the output of your workflow, you have two options for working with or presenting feature templates in the map. You can require the user to select a template from the Create Features window or you can implement a feature template dialog box that works similar to the Buffer command. Often, implementing your own feature template dialog box is the better option, as you can ensure the user has specified a template that meets the criteria of your customization. For example, you may only want the user to choose from templates from certain layers or those that are of a particular geometry type.
If you decide to create your own template dialog box for a customization, you can start by downloading a sample that Esri has created and shared in the Editing Labs group on ArcGIS.com. This sample is a Visual Studio 2010 project written in C# that creates an add-in containing simple dialog boxes to display templates. The sample was modeled after the template dialog box from the Buffer command and contains the same controls and considerations for displaying feature templates. You can use some or all of the functionality from this sample for your own customization. It is important to note that the dialog box can only display existing templates; you must create, delete, and manage templates for layers as usual through the Organize Feature Templates window.
Using the Template Dialog Sample command
The sample creates an add-in command that provides a similar chain of dialog boxes as Buffer: clicking the command opens a dialog box containing a button that opens another dialog box with a list of feature templates. The command is called Template Dialog Sample, which is TemplateAppButtton.cs in the Visual Studio project. To explore the sample’s functionality in ArcMap, compile the project or install the add-in that accompanies the download. Then, drag the Template Dialog Sample command from the Customize dialog box > Commands tab > Editing Labs category and place the button onto any toolbar. The command is only enabled when you are in an edit session.
Click the Template Dialog Sample button on the toolbar to which you added it to display the template application form (TemplateAppForm.cs). Click the Template button on the Template App Form dialog box to open the Template Picker Form (TemplatePickerForm.cs). Similar to the Select Feature Template dialog box opened from the Buffer command’s dialog box, the Template Picker Form allows you to choose a template and search for templates by name, tag, and layer.
In this sample, the Template Picker Form displays feature templates from all visible point, line, and polygon layers in the current editing workspace. If you want to enable filtering by geometry type, for example, you can modify the sample code through an enumeration in TemplateAppForm.cs when calling the TemplatePickerForm.
Most of the internal work in the sample occurs in the TemplatePickerForm class. This class contains a Visual Studio ListView control to display the templates as ListViewItems arranged into ListViewGroups that represent the layers. Selecting a template sets the _currentTemplate variable back on the add-in button, which can be used to set the current Editor template target on IEditor3.CurrentTemplate. The template can then be used to help you create new features as described in the Working with templates section in the Using feature templates help topic.
Content provided by Sean (Editing Team)
Geoprocessing can be used to automate many aspects of data compilation, including implementing a geodatabase topology. In my previous post, I created a model that imports data into a file geodatabase and performs initial data cleanup processing. Now, I will use geoprocessing with Python scripting to build a topology on the imported data. Although many spatial integrity issues were resolved by running the model, a geodatabase topology can help check for and repair any remaining errors.
The dataset I am using is a feature class of parcel lot line boundaries that began as a CAD file full of topological inconsistencies, such as lines that overlapped or did not connect to other lines. Once I create the topology and am sure the lines are correct, I will build parcel polygons from the lines and introduce all the features into my production enterprise geodatabase.
Creating a topology in a script
To build topology in an automated manner, I am going to write a simple Python script, add it to a toolbox, and run it as a regular geoprocessing tool. Essentially, the script just performs the functions of the New Topology wizard, but without requiring my intervention. In fact, this script could be developed without any programming by creating a model to perform these tasks and exporting it from ModelBuilder as a Python script.
When scripting, the ArcPy site package allows Python to access and run any of the geoprocessing tools in the ArcGIS system toolboxes, including the topology tools. The Topology toolset in the Data Management toolbox contains all the tools I need to add a geodatabase topology to the line feature class. The import arcpy statement adds ArcPy to a script and is at the top of every ArcGIS Python script.
Before I start adding the tools to the script, I define variables for the paths to the feature dataset, feature class, and topology. Because I created folders to store my tools and data following the recommendations in A structure for sharing tools, I can make the script more portable by setting these paths in relation to the folder containing the script. Since the folder locations are not hard-coded to match the C: absolute path of my hard drive, the script should run on someone else’s machine regardless of the machine’s directory structure.
With the paths defined, I first use the Create Topology tool to add a topology to the feature dataset. The syntax for the tool in my script is arcpy.CreateTopology_management(featureDataset, topologyName, “”), where CreateTopology is the name of the Create Topology tool, _management is the toolbox in which it resides, featureDataset is a variable I defined earlier representing the path to the feature dataset, and topologyName is a variable for the name of the topology. When working with topology, it is recommended to use the default cluster tolerance, which is the distance in which vertices are determined to be coincident. Since I want ArcGIS to calculate the cluster tolerance, I left the value blank as “”instead of supplying one.
Although that function creates a topology, it is currently empty and has no classes or rules in it. I can use the Add Feature Class To Topology tool to add my line feature class with the statement: arcpy.AddFeatureClassToTopology_management(topology, featureClass, “1″, “1″). If the topology contained multiple feature classes, I can set ranks so the feature class with the highest accuracy is not adjusted to match vertices in a feature class which is known to be less accurate. However, since I am using only one feature class, I leave the rank parameters as “1″.
Next, I set which topology rules to include by calling the Add Rule To Topology tool. When choosing which rules to add to a topology, there are a few rules that many editors commonly add to every topology, such as the line rule for Must Not Have Dangles. In the script, this rule is coded as arcpy.AddRuleToTopology_management(topology, “Must Not Have Dangles (Line)”, featureClass, “”, “”, “”). Because the rule only applies to one feature class that does not have subtypes and is not a rule between two feature classes or subtypes, the other parameters are left blank. I also want to make sure that the lines do not overlap or intersect themselves, so I include the Must Not Overlap and Must Not Self-Intersect rules as well. If I need to add more rules later, I can run the Add Rule to Topology tool or use the topology’s Properties dialog box in the Catalog window.
Once all the feature classes and rules have been added, my script validates the topology. The Validate Topology tool identifies features that share geometry, inserts common vertices into features that share geometry, then performs integrity checks to identify any violations of the rules that I defined for the topology. Since the topology has never been validated before, I am going to validate the entire extent of the data with the syntax, arcpy.ValidateTopology_management(topology, “Full_Extent”). However, when working in ArcMap, validating the visible extent of the map instead of the full extent limits the area to be validated and can be useful for very large datasets that take a long time to validate.
Finding and fixing topology errors
After the script runs, I add to ArcMap the resulting topology so I can inspect the results and fix any errors using the ArcMap editing tools. However, with all the previous automated QA work from my Import and Clean Lines model, the remaining manual edits are minimal in comparison to what they could have been without running it first.
The errors identified with topology are indicated by the orange-colored squares. Almost all of these are dangle errors that could not be fixed by the original model since they exceeded the tolerance value for the Extend, Trim, or Snap tools. The topology found only two Must Not Overlap errors, which I can fix by deleting one of the overlapping features. There are no violations of the Must Not Self-Intersect rule, indicating that the lines were split properly in the model. Using the editing tools on the Topology toolbar, such as the Error Inspector and Fix Topology Error tool, I can review each error to determine if the built-in topology fixes can be used or if the lines should be edited manually to resolve the topology error.
In some cases, the topology error may need to be marked as an exception, which is a valid violation of a topology rule. One of the most common examples of exceptions to the Must Not Have Dangles rule is a cul-de-sac road, which are dead ends that do not connect to other roads. However, when working with parcel lot lines, there are fewer scenarios that are valid exceptions. I do have some lines at the edges of the dataset that do not connect to other lines. I can either mark these as exceptions or choose to delete the features, depending on whether these features are supposed to connect to existing features in my enterprise database.
If an edit is made to correct a topology error, I have to validate the topology again to make sure the error no longer exists. After I perform a visual inspection and fix all the remaining topology errors, I can create new polygons representing landownership parcels using the geometry of the lines. If I attempted to create polygons from lines that do not connect to each other properly, either no polygons would be created or one large polygon would result where there should actually be two polygons. After creating the new polygons, I add the polygon feature class to the topology and check for any gaps or overlaps and make sure the parcel polygon boundaries are always coincident with the lot lines.
After implementing the topology and making edits in ArcMap, the lines and the polygons created from them meet standards for our spatial data. I can now introduce the features into the production enterprise geodatabase.
While a script tool or model may take time to set up initially, in the long run, it is quicker to automate data compilation tasks through geoprocessing whenever possible. I can run a tool as needed and re-run it later with different parameters and tolerances or apply it to other datasets. Scripts are particularly useful because they can be run at specified times as a scheduled task in Windows. For example, I could combine the tools presented in these blog entries into a Python script that imports a dataset into a geodatabase, processes it, and implements topology. If I set the script to run automatically in the evening after working hours, I am ready to start editing on clean data when I come into the office the next morning.
For more information:
The sample tools and data can be downloaded from the Editing Labs group on ArcGIS.com. An ArcInfo license in required to run the tools.
Content provided by Rhonda (Editing Team)
Editing and data compilation are less commonly thought of as operations that can be automated through geoprocessing. However, ArcGIS 10 introduced the Editing toolbox, which contains a set of geoprocessing tools to perform bulk edits. These tools combined with others in the geoprocessing environment can automate data import and maintenance work. Automated data compilation tools are especially useful for importing data into a geodatabase but can also be employed on a regular schedule to perform routine quality assurance (QA) checks. In this entry, I will discuss the use of geoprocessing to clean CAD line data as part of the import process.
Importing data with geoprocessing
Lines that are created without the use of spatial integrity measures, such as snapping or topology, almost always contain some inconsistencies. These errors are likely in data that originated in formats such as CAD, shapefile, or KML. Fortunately, many common topological issues can be resolved in an automated manner by using ModelBuilder to link together tools that will import data into a geodatabase and perform standard data cleanup techniques.
I have a CAD file for a new subdivision that needs to be integrated with my existing GIS parcel data. The GIS data must be kept to stringent accuracy standards, so I need to fix any issues where lines do not connect to each other, overlap, or are duplicated. Rather than risk reducing the quality of the main parcels geodatabase, I can create a local temporary geodatabase where I can preprocess the CAD lines before introducing the features into the production geodatabase. Although the CAD file contains buildings, roads, text, registration tic marks, and other features, I plan to use only the parcel lot lines.
I have built a model that imports the CAD lines into a temporary scratch workspace, cleans and processes the lines, and then copies the corrected lines into an output file geodatabase. When importing CAD data into a geodatabase, I can choose from several available tools, including CAD to Geodatabase or Feature Class to Feature Class. The CAD to Geodatabase tool converts all the geometries in a drawing to individual feature classes, such as a line feature class for the parcel lines, annotation feature class for CAD text, and so on. In my case, I am using Feature Class to Feature Class tool because I need only the lot line geometry from the CAD file. This tool makes the model reusable because it can import many different formats and not simply CAD. In addition, the Feature Class to Feature Class tool allows for an SQL expression so I can further refine the import to include only the CAD features that satisfy an attribute query for lot lines (in this case, “Layer” = ‘LOT-L’).
Performing automated quality assurance on lines
Once the CAD parcel lot lines are imported into a geodatabase feature class, I can begin running tools to perform automated QA processes. Many tools are found in the Editing toolbox, although other toolboxes can be purposed for data compilation QA tasks. For example, I can start by using the Integrate tool in the Data Management toolbox to address minor inconsistencies in the line work. Integrate makes features coincident if they fall within the specified x,y tolerance. By using a small tolerance on Integrate (and other similar tools), I can avoid editing the data beyond the level of cleanup I intended. In addition, since I am running the tools on a copy of the data outside my production database, I can run the tools repeatedly to refine tolerance values to fix more issues in an automated manner. The intermediate data created as the model runs is maintained and can be reviewed in the scratch geodatabase.
After the dataset is integrated, I check for duplicated lines with the Delete Identical tool (Data Management toolbox). The dashed lines connecting to this tool represent preconditions, which are used to control the order of operations in a model. For example, the Integrated Lines output is a precondition to the Delete Identical tool. This way, the Delete Identical tool will not execute until the lines have been integrated.
The next part of the model identifies lines that are dangles. With the Feature Vertices to Points tool in the Data Management toolbox, I create a new point feature class containing the line endpoints that are not connected to any other lines. I can then use Select Layer By Location to identify the lines that intersect these dangling endpoints. The resulting selection represents lines with dangles.
Many of these dangle errors can be fixed by running the Editing toolbox’s Trim Line, Extend Line, and Snap tools. Effective use of the Editing toolbox geoprocessing tools can improve productivity because the tools apply edits in bulk, such as to all features or all selected features. In most cases, the similar editing function applies to only one feature at a time. Because I exposed the tolerances as variables and model parameters, I can easily run the model with different values because the tolerance settings appear as input boxes on the tool’s dialog box. For example, I am willing to extend or trim the lines from this CAD dataset initially up to a maximum length of five feet. After that procedure, I want to inspect the lines visually to see how many issues remain to ensure that I will not be making incorrect edits if I increase the tolerance value. I can change the tolerance as needed depending on the accuracy of the lines I am importing.
In addition, since my organization’s spatial integrity rules indicate the parcel lines should be split and not intersect themselves, I can use a sequence of spatial and attribute queries to find the locations where lines have intersecting endpoints. Lines are often split so that each length can be attributed separately.
Once these processes have run, the lines are output into a feature dataset in a geodatabase and are much cleaner topologically. After the model completes, I can run the Feature Vertices to Points tool again on the cleaned output to see the remaining dangles and compare the current number of endpoints that are dangles (the yellow circles in the graphic) to the number in the original CAD lines (the red circles). While there may be a few remaining issues, there are less than before running the model. At this point, I can build a geodatabase topology to check for and repair any other errors. When I am satisfied that the lines meet the standards for our spatial data, I can import it into the production database.
For more information:
The sample tools and data can be downloaded from the Editing Labs group on ArcGIS.com. An ArcInfo license in required to run the tools.
The notion of “custom steps” in ArcGIS Workflow Manager can be confusing. Many people hear “custom” and think immediately of software development. With Workflow Manager, the reality is that a workflow designer will create and configure many different step types by using the Workflow Manager Administrator application. Even steps available out-of-the box – including special purpose steps called “Custom Step Objects” – will likely be customized in Workflow Manager Administrator to your organizations specific environment.
This blog post will discuss step customization (somewhat), Custom Step Objects (mostly), and how they can help in your workflows. If you remember nothing else, just remember that using custom steps in Workflow Manager doesn’t necessarily equate to writing custom code!