The United States Geological Survey (USGS)’s Protected Areas Database of the United States (PAD-US) has been updated.
A new version of the ArcGIS Metadata Toolkit is now available for download. This version of the Toolkit documents how to create and share custom metadata styles that use the validation capabilities provided in the ArcGIS metadata editor with ArcGIS 10.1 SP1 for Desktop.
Customizations that change how you view and export metadata, validate metadata using an XML schema, and reduce the number of pages available in the ArcGIS metadata editor require no programming. However, the Toolkit also provides the complete source code for the metadata editor’s pages if you want to change how they work. The documentation walks you through the process of creating a custom page and adding it into the ArcGIS metadata editor, modifying the contents of a drop-down list, and changing the validation rules associated with pages in the ArcGIS metadata editor.
In my previous blog, I discussed how Data Reviewer’s Metadata check can be configured to validate object metadata based on a formally published metadata standard such as the Federal Geographic Data Committee’s (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) or the ISO-19115/19139 international metadata standard. In the second part of the metadata series, I’d like to discuss how the Metadata check can be configured to validate metadata content based on an organization’s unique content guidelines.
The need for validating metadata element content
For many organizations, simply validating their metadata against a standard is not always enough to ensure that data publishing requirements are met. These data publishing requirements will vary based on the organization, intended use and audience of the data being documented, and may change through time as new requirements are identified. Validating metadata is particularly important for content that are defined as “free text” by a given metadata standard. Free text-based elements cannot be validated in the same way as numbers, dates or other elements which have a defined list of valid values. Free-text elements, see screenshot below, are common throughout each standard and include such things as a dataset’s title, abstract, point of contact name, contact address, etc.
For this blog topic, I’d like to talk about how ArcGIS Data Reviewer can be used to validate item metadata created to serve as documentation for an organization’s spatial data holdings. For purposes of this discussion, metadata is defined as information that describes the content, quality, condition, origin, and other characteristics of data or other pieces of information. For ArcGIS for Desktop users, this information has traditionally been authored using ArcGIS metadata editor and is typically based on either national or international metadata standards.
With the release of Data Reviewer 10, a new check for automating quality control review of item metadata was introduced. The Metadata check enables validation of metadata associated with spatial data stored in multiple formats and based on formal metadata standards. This includes validating metadata created to document objects stored in your Geodatabase (feature classes, tables, feature datasets, etc.), features stored in a shapefile, or metadata stored in stand-alone XML documents.
During the summer, a metadata patch was promised for ArcGIS Desktop 10. The fixes that were to be provided with this patch were incorporated into Service Pack 3. They address many important issues relating to metadata. While the focus centers on improving support for creating FGDC CSDGM metadata, the fixes cover all aspects of metadata in ArcGIS and apply to all metadata styles. Anyone who works with metadata should download and install this service pack at their earliest convenience.
In the ArcGIS metadata editor, support for handling dates was greatly improved. Support was also added for editing temporal and vertical extents, references to related resources, times associated with dates, and several other metadata concepts. Metadata content is now organized in the same manner when it is displayed and edited—headings in the display correspond to pages in the editor. Issues were fixed in how metadata is updated automatically, and how metadata geoprocessing tools handle geodatabase items. Improvements were made in how metadata is upgraded from, imported from, and exported to the FGDC CSDGM XML format. Also, a new XSLT-based translator is provided that exports ArcGIS metadata to the ISO 19139 XML format, ARCGIS2ISO19139.xml; it corrects some issues that can’t be fixed in the existing translator. Scripts and models should transition to using the new ArcGIS to ISO 19139 translator. In all, 73 metadata fixes were included in Service Pack 3.
Several metadata Help topics have now been updated in the ArcGIS Desktop 10 Resource Center to reflect the improvements provided with Service Pack 3. These Help updates are not included in the Service Pack 3 installation. Some new topics have also been added.
The following topics apply to everyone who works with metadata:
Key information about working with FGDC CSDGM metadata in ArcGIS 10 has been centralized in the topics The ArcGIS metadata format and Creating and managing FGDC metadata. And, a new help topic will guide you through the process of creating complete FGDC CSDGM metadata content in ArcGIS 10:
Also, topics that compare the metadata geoprocessing tools and explain when to use one tool rather than another for exporting, importing, and validating are now included in the table of contents with the rest of the metadata topics. These topics were previously only available as concepts associated with the Metadata toolset in the Conversion toolbox.
The ArcGIS Metadata Toolkit is now publicly available. The documentation provided with this toolkit explains how to create and share custom metadata styles. Customizations that change how you view, validate, and export metadata, and reduce the number of pages available in the ArcGIS metadata editor require no programming. The documentation walks you through the process of creating a custom page and adding it into the ArcGIS metadata editor. Also, complete source code for the pages provided with the ArcGIS metadata editor are available, if you want to modify the contents of dropdown lists or the elements available on one of the pages provided with ArcGIS Desktop.
Content provide by Aleta (Metadata Team)
Here’s another good blog post on the topic.
A new step-by-step workflow topic has been added to the ArcGIS Desktop
10 Web-based Help system that walks you through the process of creating
and managing FGDC metadata in the Description tab. It builds on the
previous metadata articles published to the ArcGIS Desktop Blog and adds
- Upgrading content created with ArcGIS Desktop 9.3.1 FGDC metadata editor or the FGDC Metadata Editor add-in
- Editing the minimum mandatory set of FGDC metadata content in the Description tab
- Publishing FGDC metadata to geodata.gov
- Importing FGDC-formatted metadata in XML and text files
video walkthrough for some of these workflows will also be available
soon. Other topics describing metadata concepts have also been added or
updated in the Help system, including:
A future update will include a complete, illustrated guide to producing FGDC metadata in the Description tab with ArcGIS 10.
Content provided by Aleta (Metadata Team)
Exporting FGDC metadata always involves exporting a correctly-formatted XML file. However, you may have to create additional FGDC-specific files that present the same information in different ways such as structured text and HTML files.
The old way…
In ArcGIS 9.3.1, you would repeat the process you used to export an item’s FGDC metadata to an XML file as many times as necessary, choosing the appropriate export format each time. If you used the Export Metadata button on the Metadata toolbar, you would pick different types from the Format drop-down list. If you used the USGS MP Metadata Translator tool in the Conversion toolbox you would run the tool again using other conversion types.
Both methods first created an FGDC-compliant XML file then used the USGS metadata parser utility, known as mp, to transform that XML file to the other formats. A version of mp is provided with ArcGIS to support this task.
The new way…
In ArcGIS 10, if you are managing your FGDC metadata content in the Description tab, you would take the FGDC-compliant XML file that you already exported from the item’s metadata and use the USGS MP Metadata Translator tool to produce the other file formats from it. A model can be created to export all needed FGDC files in one step, as illustrated below.
If you are editing an item’s metadata using the 9.3.1 FGDC metadata editor add-in instead of the Description tab, you can export the other FGDC file formats the same way that you export an XML file—using the USGS MP Metadata Translator tool. Use the appropriate conversion types to generate the file formats you need. A model can be built to export all needed FGDC files in one step, as illustrated below. This same model can be built and used in ArcGIS 9.3.1.
Continuing the discussion about how to perform familiar metadata tasks in ArcGIS 10, this post looks at exporting ArcGIS metadata to standards-compliant metadata formats.
ArcGIS 10 creates and manages ArcGIS metadata as part of the item. When you share items that have ArcGIS metadata with other ArcGIS users, those items come with information that is relevant within ArcGIS such as thumbnails. However, sometimes you might work with an item’s metadata apart from the item outside of ArcGIS; typically, that metadata must be in a standard-compliant format.
You export ArcGIS metadata to a standard-compliant XML format using the Export button in the Description tab, which opens the Export Metadata geoprocessing tool. ArcGIS has translators that know how to take ArcGIS metadata content and produce XML files that are formatted correctly for different metadata standards. Each metadata style knows which translator to use to create an XML file that is appropriate for the standard it supports. Therefore, the first step in exporting standards-compliant metadata is to make sure ArcGIS 10 is configured correctly for that metadata standard.
For example, metadata catalogs typically require published metadata documents to be provided in a standard-compliant format. To publish an item’s ArcGIS metadata you would export it to a standard-compliant XML file, then publish the exported file.
Exporting an ISO 19139-compliant XML file
The old way…
In ArcGIS 9.3.1, you would edit an item’s metadata using the ISO metadata editor, then export it to an ISO 19139-compliant XML file using the ESRI Metadata Translator tool in the Conversion toolbox. You would manually open the tool and select the ESRI_ISO2ISO19139.xml translator, which extracted information from the item’s ESRI-ISO metadata elements and produced a correctly-formatted XML file.
The new way…
In ArcGIS 10, three metadata styles are associated with ISO 19139-format XML files: ISO 19139 Metadata Implementation Specification, INSPIRE Metadata Directive, and North American Profile of ISO19115 2003. After editing an item’s metadata in the Description tab, click the Export button in that tab. All three styles instruct the Export Metadata tool to use the ESRI_ISO2ISO19139.xml translator. This translator was updated for ArcGIS 10 to export information from the item’s ArcGIS metadata elements. It also checks the metadata style, and will handle some content differently for the INSPIRE and North American profiles.
Don’t use the ESRI Metadata Translator tool directly; extra processing steps are needed to handle ArcGIS metadata correctly. Use the Export Metadata tool instead; it performs the other steps, then uses the ESRI Metadata Translator tool to complete the transformation.
If an item has ESRI-ISO metadata that hasn’t been upgraded to ArcGIS metadata yet, the same translator will export information from the appropriate ESRI-ISO metadata elements instead.
Exporting an FGDC-compliant XML file
The old way…
In ArcGIS 9.3.1, you would edit an item’s metadata using the FGDC metadata editor. Then, you would export it to an FGDC-compliant XML file by clicking the Export Metadata button on the Metadata toolbar in ArcCatalog and choosing FGDC CSDGM (XML) from the Format drop-down list. This same task could also be performed using the USGS MP Metadata Translator tool in the Conversion toolbox with the XML conversion type. Both methods extracted the FGDC-formatted XML elements from an item’s metadata and arranged them in the correct order.
The new way…
In ArcGIS 10, first ensure the FGDC CSDGM Metadata style is selected in the Options dialog box.
Edit an item’s metadata in the Description tab, then export it to an FGDC-compliant XML file by clicking the Export button. The Export Metadata tool opens, and is automatically set to use the ARCGIS2FGDC.xml translator. It knows how to extract information from the appropriate ArcGIS metadata elements and organize it to create an FGDC CSDGM-compliant XML file.
If you aren’t managing your metadata using the ArcGIS metadata editor in the Description tab, you can’t export the information you’ve provided using the Export button on that tab. If you are using the 9.3.1 FGDC metadata editor add-in instead, the information you’ve provided is stored in FGDC XML elements in the item’s metadata. You can export this information to an FGDC CSDGM-compliant XML file using the USGS MP Metadata Translator tool in the Conversion toolbox with the XML conversion type.
The next post will discuss how to export metadata to other file formats that may be used to share FGDC-style metadata.
- Aleta (Metadata Team)
The first series of posts about metadata discussed the goals for the changes made in ArcGIS 10. The next series of posts will focus on the transition from ArcGIS 9.3.1 and how to perform your familiar tasks in the new ArcGIS 10 metadata environment. It is meant to help those of you who were using metadata in a previous ArcGIS release to transition to the ArcGIS 10 release. If you never created metadata before ArcGIS 10, you probably don’t need the information in this particular post.
This first installment in the second series of posts builds on an earlier post, and looks at how to configure ArcGIS to manage metadata correctly for a specific standard. This is always the first metadata task you need to perform in ArcGIS.
The old way…
To configure ArcGIS 9.3.1 for a specific standard you had to individually select all of the correct utilities for that standard by performing the following tasks.
- Select the metadata editor in the ArcCatalog Options dialog box.
- Select a stylesheet for viewing metadata from the dropdown list on the Metadata toolbar.
- Set your favorite stylesheet as the default in the ArcCatalog Options dialog box.
- Enable or disable the correct synchronizers using the Set Working Synchronizers dialog box to automatically update metadata appropriately.
- Select the correct exporter for your metadata standard each time this task is performed, either using a button on the Metadata toolbar or by using the correct geoprocessing tool.
If you accidentally selected the wrong utility for a task you would end up working with the wrong set of metadata content and you would get incorrect results. There was no way to simply tell ArcGIS to create metadata for a specific standard.
If you wanted to create FGDC CSDGM-compliant metadata, you got a bye for some of these tasks because the default metadata editor was the FGDC editor, the default stylesheet displayed some FGDC metadata content, and the FGDC metadata synchronizer was enabled by default.
The new way…
To configure ArcGIS 10 for a specific standard, choose the appropriate metadata style from the Options dialog box in any ArcGIS Desktop application. This one choice sets:
- The stylesheet used to display metadata
- The pages included in the ArcGIS metadata editor
- The exporter used to generate a standard-compliant XML document from ArcGIS metadata
- The XML schema used to validate an exported, standard-compliant XML document
The default settings support creating simple descriptions to accompany items that can be published to ArcGIS Online. That means, people who want to create FGDC CSDGM-compliant metadata no longer get a bye when it comes to configuring ArcGIS 10 for that style of metadata; to view and edit complete metadata content and export standard-compliant XML documents you must choose a different style.
ArcGIS 10 is designed to use and maintain ArcGIS metadata. You create it using the ArcGIS metadata editor, which is available within the Description tab, and it is updated automatically to include an item’s properties by the ArcGIS metadata synchronizer. Turning synchronizers on and off is not a relevant task for ArcGIS 10. Existing metadata created with a previous version of ArcGIS must be upgraded to ArcGIS metadata to manage that content in the Description tab.
Configuring ArcGIS 10 for ISO metadata standards and profiles
Choose the appropriate style in the Options dialog box for the ISO metadata standard or profile you need to follow. To create ISO 19115- or ISO 19119-compliant metadata content, choose the ISO 19139 style; ISO 19139 is the implementation specification for those metadata content standards.
Since the supported ISO metadata profiles subtly modify the base standards, all of these styles are very similar. An extra page might be available in the editor, and some shared pages and the exporter might work a bit differently for each style. However, because the profiles don’t provide custom versions of the ISO 19139 XML Schemas, all styles are validated the same way.
Any existing content provided with the ArcGIS 9.3.1 ISO metadata editor that is read-only is displayed with gray text under the ArcGIS Metadata heading alongside your ArcGIS 10 content. Any existing 9.3.1 FGDC metadata content that is present will also be displayed under the FGDC metadata heading.
If an item has existing 9.3.1 ESRI-ISO metadata content and no 9.3.1 FGDC metadata content, ArcGIS 10 will automatically upgrade the existing content to the ArcGIS metadata format when you look at its metadata. Otherwise, upgrade to ArcGIS metadata using the Upgrade Metadata tool.
Configuring ArcGIS 10 to create FGDC metadata
First, install Service Pack 1, which provides the FGDC CSDGM Metadata style. Then, choose this style in the Options dialog box.
Metadata content created and managed using the Description tab in ArcGIS 10 is displayed under the ArcGIS Metadata heading, as illustrated below. Any content entered with the ArcGIS 9.3.1 FGDC metadata editor is displayed under the FGDC Metadata heading.
- The item properties added automatically by ArcGIS
- Any FGDC metadata content that corresponds to the general item description
- Any attribute descriptions provided using the ArcGIS 9.3.1 FGDC metadata editor; these are shared with ArcGIS metadata
To manage your FGDC metadata content in ArcGIS 10, upgrade it using the Upgrade Metadata geoprocessing tool. The upgrade process copies all existing FGDC content to the appropriate ArcGIS metadata elements. After upgrading, the old 9.3.1 FGDC metadata content isn’t removed, so you’ll see the same information under both headings. If you edit your metadata in the Description tab, your changes will be saved in the ArcGIS metadata format (the big blue oval in this diagram) and will only appear under the ArcGIS Metadata heading.
Using the FGDC metadata editor add-in with ArcGIS 10
The FGDC metadata editor add-in gives you access to the ArcGIS 9.3.1 FGDC metadata editor in ArcGIS 10. This editor is provided as a temporary measure while you become comfortable with the new metadata environment. It lets you edit existing information stored in all FGDC CSDGM XML elements (the yellow oval in this diagram) in an item’s metadata.
Choose any metadata style other than Item Description in the Options dialog box to see this 9.3.1 FGDC metadata content under the FGDC metadata heading. Any edits you make using the add-in will appear under the FGDC Metadata heading only. This content will not be updated automatically by ArcGIS 10, for example, when the number and extent of an item’s features change.
Content provided by Aleta (Metadata Team)