Tag Archives: Data & Maps
Is your data “CRAAP”?
Is your data any good or is it “CRAAP”? Assessing spatial data quality grows in importance as it grows in volume and diversity and as it becomes easier to access. Research and development on metrics and standards to measure data quality took off during the mid-1990s, and thus there is no shortage of evaluation instruments to choose from. Even so, it often is difficult to evaluate the quality of a data set you are considering using.
How good is your data?
People in library science really understand data and their implications, and some of the metrics I find most useful come from the library and information science community. My colleague Linda Zellmer, Government Information and Data Services Librarian at Western Illinois University, uses a “CRAAP” test, originally from CSU Chico and based on the CRAP test from LOEX. This is a schema to evaluate information: Currency, Relevance, Authority, Accuracy, and Purpose. When I teach with or about GIS, to get across the point that assessing data quality ultimately depends on metadata, I frequently refer to the FGDC’s “top 10 metadata errors” document. The document’s number one identified error is “not doing it!”: ”If you think the cost of metadata production is too high, you haven’t compiled the costs of not creating metadata, including loss of information with staff changes, data redundancy, conflicts, liability, misapplications, and decisions based upon poorly documented data.” Ouch!
One of my favorite papers examining measurement standards comes from Dr Jingfeng Xia of Indiana University, who, in his research published in Issues in Science and Technology Librarianship, proposed a set of dimensions for data quality measurement. He discusses measures such as accuracy, consistency, completeness, and integrity, but also accessibility, validity, timeliness, currency, conformance, uniqueness, and others. One of his main points is that both quantitative and qualitative metrics are essential for determining the quality of geospatial data.
As my co-author Jill Clark and I point out in the book The GIS Guide to Public Domain Data, it is more important than ever before to document your data, and understand what you are using, because with each passing day it becomes easier to combine data from an amazing array of sources. With opportunity comes responsibility!
Using Topographic Maps to Study Landforms in ArcGIS Online
Using topographic maps to study landforms has been a key part of Geography and Earth Science instruction for over a century. It has never been easier to do, thanks to the seamless USGS topographic maps for the USA and the base topographic map for the world available in ArcGIS Online. I have created a set of 12 questions and a map containing 20 landforms as a starting point for these investigations.
Topographic Map Activity in ArcGIS Online.
Remember the old days when the landform you were seeking to analyze seemed to inevitably fall across corners of 4 topographic paper map sheets? The USGS maps, originally published at 1:24,000, 1:100,000, and 1:250,000 scales, display seamlessly in ArcGIS Online – no more corners! The map above opens to the Ennis, Montana area, on the classic alluvial fan that has long been a staple with these sorts of investigations.
Questions include the following, which can be used as is or as a springboard for your own questions.
Use the bookmarks to zoom to the 1:24,000-scale map. Measure the distance between each contour line. Determine the contour interval by reading the numbers on the contours. Calculate the slope in percentage and in degrees. Calculate the slope of the fan again using the 1:100,000 scale map. Is this measurement different than the measurement you calculated using the 1:24,000 scale map? Explain a few reasons for possible differences. Calculate the slope in another location on the fan. Is the slope similar to your other reading? Why are slopes so constant on an alluvial fan?
Calculate the area of the alluvial fan using the square mile grid shown on the topographic map as a guide, and the scale bar in the lower left of your ArcGIS Online map window. Then compare this measurement against what you get by using the measure tool above the map. Be sure to indicate the units you are using. Name 3 differences in the type and number of features shown on USGS maps at the 3 different scales. Why do these differences exist?
Examine the following features, each of which is accessible through the Bookmarks above the map. For each landform, indicate: What is the name of the landform? What is the location of the landform? How did the landform form? What did the landform and area look like 100 years ago? 1000 years ago? Why? What will the landfrom and area look like 100 years from now? 1000 years from now? Why? Would you classify the landform as rapidly changing or slowly changing? Why? How has the landform influenced human activity and settlement in this area? How have humans modified the landform, if at all, in this area? What is the climate and vegetation like in this area? Can you find the same landforms in other areas? If so, where are they?
The 20 landforms included in the map and lesson are a tombolo, a col, a salt dome, lava beds, marine terraces, the Llano Estacado, sand hills, drumlins, moraines, a caldera, an estuary, karst, a water gap, a tarn, an arete, a structural dome, a slow moving landslide, trellis drainage, an oxbow lake, and an inselberg. The lesson also includes comparison of landscapes shaped by the public land survey system, long lots, and metes and bounds.
You can use ArcGIS Online to draw your own points, lines, and areas on the topographic map using “Add” and “Create Editable Layer.” Link your features to text, photos, and videos. Save your map (requires either a personal or an organizational ArcGIS Online account). You can also add USGS topographic maps to any ArcGIS Online map through the “Add” function by searching for “USA Topo.” You can also use the Add tool to add climate, weather, ecoregions, and other layers to help you understand the interaction between climate and landforms.
Working outside the USA? Then make sure your base map is set to “Topographic” and you can explore landforms using a topographic map base all around the world!
Walking On Water? Reflections on Resolution and Scale
I recently gave presentations at the University of Wisconsin Milwaukee for GIS Day, and took the opportunity, as most geographers would, to get out onto the landscape. I walked on the Lake Michigan pier at Manitowoc, enjoying a stroll in the brisk wind to and from the lighthouse there, recording my track on my smartphone in an application called Runkeeper. When my track had finished and been mapped, it appeared as though I had been walking on the water!
Map of my walk from Runkeeper.com.
Photograph of my destination: The lighthouse at the North Pier, Manitowoc, Wisconsin.
According to my map, I walked on water. Funny, but I don’t recall even getting wet! It all comes down to paying close attention to your data, and knowing its sources. Showing these images provides a teachable moment in a larger discussion on the importance of scale and resolution in any project involving maps or GIS. In my case, even if I scrolled in to a larger scale, the pier did not appear on the Runkeeper’s application’s base map. It does, however, appear in the base map in ArcGIS Online. In the book that Jill Clark and I wrote entitled The GIS Guide to Public Domain Data, we discuss how scale and resolution can be conceptualized and put into practice in both the raster and vector worlds. We cite examples where neglecting these important concepts have led not only to bad decisions, but have cost people their property and even their lives. Today, while GIS tools allow us to instantly zoom to a large scale, the data being examined might have been collected at a much smaller scale. Much caution therefore needs to be used when making decisions when the analysis scale is larger than the collection scale.
What example have you used in class that well illustrates the importance of scale and resolution?
- Joseph Kerski, Esri Education Manager
Reflections on Terraserver and the Evolution of Data for GIS
Back in 1998, I and my colleagues were thrilled with the arrival of Terraserver. While maps and images for use in GIS on the web today are commonplace, back then it was revolutionary. Suddenly, thanks to an agreement between the USGS and Microsoft, the GIS community had access to USGS topographic maps and aerial photographs down to 1 meter spatial resolution for the entire USA. Two additional features made this service extra special. First, these images were georeferenced, meaning that they could be easily used within a GIS environment. Second, these images were online: No CD-ROMs or other physical media were required! After downloading the maps and aerials for our area of interest, we could read these maps and images into our ArcInfo or ArcView GIS software. True, the header files often needed to be edited first, but this resource gave us a huge leap forward because we had terabytes of data at our fingertips via http://www.terraserver-usa.com, later becoming http://msrmaps.com. Even better was when some enterprising folks at Esri wrote programs to automatically stream these images to ArcGIS.
Now, 14 years later, Terraserver was recently retired. As the National Atlas recently wrote, “We note its passing and salute all those who developed the service. Many people were involved in this groundbreaking effort. Still, there were three individuals who largely provided the vision and hard work that resulted in this remarkable service: Tom Barclay (Microsoft), Beth Duff (USGS, deceased), and Hedy Rossmeissl (USGS, retired). The National Atlas switched over to services provided by Esri so that Atlas users can continue to link from our maps to large-scale topo maps and aerial views. This takes us full circle. The National Atlas Map Maker was the first on-line, interactive mapper offered by the Federal government. It was partially developed under a joint research effort by the USGS and ESRI
in 1997.”
A plethora of base maps, topographic maps, satellite images, and aerial photographs are now available to the GIS user and the general public such as via ArcGIS Online. Times have changed but the need for good base data lives on. While I don’t long for those days of tinkering with header files, I salute the early pioneers who made it all happen, and look forward to the future. The evolution of GIS data, and discussion about data sources, quality, and related issues are detailed and blogged weekly about in the book that Jill Clark and I wrote, entitled The GIS Guide to Public Domain Data.
I and my colleagues frequently need old aerials for land use change studies, however, and therefore, I wish Terraserver had remained online. Why couldn’t it have done so? What are now the best sources for old aerial photographs?
- Joseph Kerski, Education Manager, Esri
New Book: The GIS Guide to Public Domain Data
Central to the interest of the GIS community is spatial data: Where to find it, how to use it, how to gauge its quality, its scale, format, and resolution, privacy issues, copyright and licensing, the policies that govern the use of data, the role of data in the evolution of spatial data infrastructures, fee vs. free issues, cloud vs. desktop, downloading vs. streaming, crowdsourcing and citizen science, and a host of related issues. I am pleased to report that a book that Jill Clark and I co-authored on this subject has been published by Esri Press, entitled The GIS Guide to Public Domain Data.
This book [website] is accompanied by a blog that we update weekly with data sources and news on the issues explored in the book. The book is also accompanied by 10 activities free to use that involve the access and use of public domain data to solve problems. These problems range from selecting the most suitable locations for tea cultivation in Kenya, investigating the Gulf Oil Spill, siting a café in a metropolitan area, assessing citizen science portals, creating an ecotourism map in New Zealand, analyzing sustainable land use in Brazil, analyzing floodplains in Colorado, and much more. These activities are linked to the concepts presented in each chapter, and are accompanied by quizzes and answer keys, designed for easy use by an instructor, students, or the individual GIS practitioner. All of these resources are linked to the Spatial Reserves site and reside on ArcGIS Online. Our goal for the text and the activities is to provide GIS practitioners and instructors with the essential skills to find, acquire, format, and analyze public domain spatial data.
“This book fills a very big gap in the literature of GIS and brings together for the first time discussions of issues users of public domain data are likely to confront,” says Michael F. Goodchild, professor of geography at the University of California, Santa Barbara (UCSB), and director of UCSB’s Center for Spatial Studies. “It will prove useful to GIS practitioners in any area of GIS application, including students anxious to learn the skills needed to become GIS practitioners and data producers who want their data to be as useful as possible.”
How might you use this book and its associated resources in your own GIS journey or instruction?
- Joseph Kerski, Esri Education Manager
Reflections on Data Interpretation and Resolution
One of the recurring themes in GIS education blogs, forums, listservs, conferences, and in our recent book The GIS Guide to Public Domain Data is the importance of paying attention to the characteristics of data so that correct interpretations of that data can be made when mapping that data. During some recent work with one of our favorite resources mentioned in the book, that of the Center for International Earth Science Information Network, (CIESIN), I examined a map showing the population density for central Europe (shown in image).
As I examined the map, population density for Germany seemed unnaturally higher than that of France. True, according to a variety of outside sources, Germany does have roughly twice the population density as does France (235 people per square kilometer for Germany versus 108 for France). But the map seemed to indicate that broad areas of Germany are even higher in density, approaching that of The Netherlands. Is this accurate?
Upon further investigation into the metadata, indeed, the resolution of the data set was to blame. CIESIN uses the highest resolution data available for generating maps like this, and the data they were able to obtain for France had a much higher resolution. The resolution is calculated as the square root of the land area divided by the number of administrative units. For Germany, it was 28, and for France, 4. The resulting population per administrative unit in 2000 was 184,000 for Germany versus only 2,000 for France. A-ha!
Close examination of the map prompted my initial concern: The areas mapped in Germany are much larger than in France. But the differences in data collection could not be confirmed until the metadata was examined. Fortunately, CIESIN does an extremely good job documenting their sources and methods. The user still needs to make it a point to read that documentation. But what should we do when working with sites that do not document their data well? In today’s world of a myriad of data, maps, and tools, it is more important than ever to have a good grounding in map interpretation and spatial analysis, but also to ask questions of the data you are using. How can we as the GIS education community foster this kind of questioning of data by our students?
- Joseph Kerski, Esri Education Manager
Fostering Spatial Thinking Through Paper Maps and Aerials
Those of us on the Esri Education Team have dedicated our careers to promoting and supporting the use of GIS and spatial thinking in education to enhance teaching and learning and to have a positive impact on society. We believe that Esri GIS tools are some of the best means available to fostering spatial thinking and prepare students for 21st Century decision making. Yet we are frequently asked what non-software activities are effective in fostering spatial thinking. Thousands indoor and outdoor activities, games, and lessons serve as excellent resources to prepare students to use GIS and also are excellent where no access to the Internet or software exists. Indeed, one of my favorite stories from the recent International Perspectives on Teaching and Learning with GIS in Secondary Schools book I co-edited came from South Africa, where students studied issues in their community and country using paper maps. I really like what my colleague David DiBiase said to the United Nations in a recent address: “The digital divide is no excuse to ignore geography.” Space doesn’t permit me to expand on the many activities based on paper maps and aerials that are suitable that I and others in the community have used for years, so let me describe just one in this essay. I have used this activity many times from age 6 to university level.
- Before you teach the lesson, access ArcGIS Online and change the base map to imagery or Bing aerial (whichever is higher resolution) and zoom the map to focus on school where you will teach this lesson. Make sure the school and the school grounds take up most of the image, but include some of the surrounding neighborhood as well.
- Print one of the aerials for each student, leaving space on the right and top for the title, legend, and other information.
- Obtain one piece of translucent paper for each aerial, and some clear tape.
- Go to the class where you are teaching with your papers and printed aerials.
- Ask the students what they think the school looks like from above. Have the students sketch the school on the white board or on paper, noting which way is north. Can they identify the cardinal directions by standing up in the classroom and pointing?
- Hand out the aerials and translucent paper. Discuss how well the aerial matches the students’ drawings on the board. What matches, and what doesn’t match, and why?
- Have the students tape the paper to the aerial along the top edge only.
- Get out colored pencils or markers. Discuss elements that are important for a good map, such as TODALSIGS – Title, orientation, date, author, legend, scale, index, grid, and source. Add name, orientation, title, scale, and source to start with.
- Discuss map themes (trees, school building, street, playground, soils, lakes and streams, and so on). Have the students choose one color for each theme for the translucent paper. Trace each theme from the aerial photo onto the translucent paper, lifting up the paper when necessary to have a clearer view of the aerial.
- Add each theme to the legend using the same color used for the theme.
- Ask students to remove the translucent paper from the aerial photo: Now they have a map! How is the map the same as the aerial image? How is it different?
What paper-based activities have you used to foster spatial thinking?
- Joseph Kerski, Esri Education Manager
Civil War Sesquicentennial: Fort Sumter
On April 12, 1861 the range of issues that the country had been contending with for decades erupted into armed conflict between the North and the South. It began at Fort Sumter, an island Federal fortification in Charleston Harbor (South Carolina), as Confederate artillery opened fire. Literally, surrounded by Confederate installations, in little over a day, the Union garrison there surrendered. The Civil War had begun.
There is much historical narrative content available to help you and your students better understand this event and the future ones that unfold across the ensuing years of the war. Here are some geographically-focused resources that may provide you a different/broader perspective.
Esri colleague, Allen Carroll (former chief cartographer at National Geographic) has brought 1861 and 2011 together in an online map story, http://mapstories.esri.com/sumter/. In addition to leveraging ArcGIS Online and creating a custom app, the past comes alive through the assistance of David Rumsey and primary documents from his historical map collection. These resources provide clear context to what, from a geographical standpoint, had to seem like a inevitable outcome to both sides in the fight.
The combination of the historical map content and present-day imagery also provides a 150-year look at persistence and change in the landscape. For instance, note the battery placements to the southwest of Fort Sumter and the current shoreline.
Other commanding geographic resources to explore include maps and an animation from the Civil War Trust. They provide a look at the fall/capture of the fort in April 1861 and offer maps depicting defenses in the successive years. Likewise, the National Park Service has a range of resources about Fort Sumter and associated areas.
Peer into the Civil War in greater depth (geographically and temporally) by making the most of materials in the Civil War map collection at the Library of Congress. Also ready for use is the rich array of geographic resources in the David Rumsey Map Collection, in particular nearly 300 Civil War maps and documents he has organized by publication date for easy viewing and use, such as this panoramic map of the seat of war in the Delmarva region
Stay tuned: In the coming months, we will post other resources and ideas for exploring this pivotal period in American history.
- George Dailey, ESRI Education Program Manager
Investigating the Gulf of Mexico Oil Spill Using GIS
Like other disasters, human and natural, the current horrendous situation with the oil spill in the Gulf of Mexico is a dynamic event, changing rapidly over space and time. Because of this, it can be effectively investigated within a GIS. GIS provides context and content, helping students understand the location and extent of the disaster through taking measurements, overlaying other map layers, and examining scale. Moreover, GIS allows students to understand how physical and human systems are interrelated, such as winds and cleanup efforts, oil and fisheries, and much more. Far from static tools, desktop and web-based GIS tools can be used together with imagery and maps that are updated daily—even by ordinary citizens.
First, download the oil and gas drilling platforms in the Gulf of Mexico offshore from Texas from the state’s General Land Office and bring it into ArcGIS. The density of these platforms is truly astounding:
http://wwwdb.glo.state.tx.us/gisdata/disclaimer.cfm?dmap=shapefiles/subleases.zip
A set of data available on ArcGIS Online includes forecasted movement of the plume, a 3-D ArcGIS set of data, ASTER and Landsat imagery, environmental impacts, critical habitat, and more, on:
http://www.arcgisonline.com/home/search.html?t=content&q=gulf%20oil%20spill
ESRI created and posted an interactive map that allows the viewing of location-based feeds, including news and videos, and also allows users to add their own content, on:
http://www.esri.com/services/disaster-response/gulf-oil-spill-2010/index.html%20%20
The ability to add content is provided by ArcGIS Server 10 beta hosted in the Amazon Web Service infrastructure.
Dig deeper and use some remotely sensed MODIS imagery:
http://rapidfire.sci.gsfc.nasa.gov/
For daily images see:
http://rapidfire.sci.gsfc.nasa.gov/subsets/?subset=USA7
These pages provide GIS compatible imagery. Select “more info,” select the granule of interest from the list at the bottom of the page. Click on the link to get an uncorrected 5-minute swath image, down to 250 meter resolution. Make sure you also download the world file so the image will be georegistered within your GIS. You can also download a KML and use it in ArcGIS Explorer.
For example, a 250m MODIS image is available on:
http://rapidfire.sci.gsfc.nasa.gov/gallery/?2010119-0429/Louisiana.A2010119.1648.250m.jpg
I encourage you to use GIS in your analysis of the Gulf oil spill to try to make sense of this disaster.
–Joseph Kerski, ESRI Education Manager
2010 Census: Take a Sneak Peek with ArcGIS Online and ArcGIS Explorer
April 1, 2010 is Census Day here in the U.S. This decennial event marks only the 23rd time in our history that we have taken an enumerated population snapshot of ourselves. Unlike years past, the form we complete contains only 10 questions about basic household characteristics. The larger body of questions aimed at socioeconomic, housing, and other data are now part of the ongoing American Community Survey. The first 2010 Census data reported will be total population for the nation and states and used for the reapportionment of Congress, but not available until December 31, 2010.
What if we wanted to get a taste of what will be coming from the census? Well, we can. Using ESRI data services found in ArcGIS Online (AGOL) and ArcGIS Explorer (AGX) we can examine likely patterns. ESRI has a team of demographers, economists, and statisticians that are focused on creating a range of data for a range of geographies. They make keen use of data from the Census Bureau and elsewhere to construct various estimates and projections. (Interested? Examine ESRI’s methodology).
So, let’s map some recent demographic information. Launching AGX, I decide to keep my map in 2D mode and focus on the contiguous 48 states. I also change my Basemap to “World Streets.”
From the “Add Content” menu I select “ArcGIS Online” which opens my Web browser. In the “Search” window, I enter “USA population ESRI” which returns 20ish items and for now I focus on 2000-2009 population change.
Clicking on the item’s name gives me metadata about it.
From the “Use this item” window, I select “Open in ArcGIS” and when prompted, I “save” the LYR to my hard drive for this and future projects.(NOTE: I could also simply use this layer in my Web browser by clicking “Add to WebMap” but I’m going to build on this AGX project in the future.)
Back in AGX, I add the layer to my map (Add Content > ArcGIS Layers > ___.lyr). I open the layer’s “Legend” (Tools tab > Show Legend) and expand its window to discover that there is actually a legend for each of 4 geographic levels that are revealed when I zoom closer.
Currently my map shows an estimated state population change with the highest growth in the south and the west. Assuming these patterns hold what are the implications for the reapportionment of Congress?
Zooming in on Colorado and adjusting the transparency of the layer to see the underlying Basemap (Tools tab > Transparency), I quickly see that change is varied across counties with population loss evident in the east and possibly related to the pattern further east and north. What impacts could these varied changes indicate? What does this look like in your state?
Closer to home I see a patchwork pattern of population gain and loss in the center of Dallas near where I live. What local factors could be at play to spawn rapid gain and rapid loss in adjacent neighborhoods? How about where you are?
The questions I’ve posed can be explored more fully with other demographic and socioeconomic layers using the AGOL search presented above. Consider repeating my steps and expanding the exploration. And remember to have the data that you eagerly use requires responding to Census Bureau and other key data collection efforts. It’s in your hands.
-George Dailey, ESRI Education Program
…and Census Bureau employee 1985-1993
