ArcGIS Resource Center

On April 20, 2010 an explosion disabled the offshore drilling rig Deepwater Horizon and touched off a prolonged regional disaster. Several crew members were killed, and a series of equipment failures left a well on the floor of the Gulf of Mexico freely releasing oil into the ocean for several months. The oil posed a very serious threat to the economy and wildlife in the region. The response to this event was massive, and Esri contributed to the containment and recovery with both manpower and software.

One of the ways Esri was able to contribute was with a Web application that brought together some of the new features in ArcGIS 10 with data services for the affected area. The application was unique in that it built on some recent trends in the geospatial community. Specifically, it was focused around authoritative content for projected spill locations (from NOAA), sensitive natural resource areas (from the Louisiana, Mississippi, and Alabama departments of wildlife), a disaster response feed from Ushahidi, and live data feeds from social media outlets such as YouTube, Twitter, and Flickr. Additionally, users of the application could post their own content. This has been commonly referred to as volunteered geographic information, or VGI. The end result was the Esri Social Media/VGI application.

While the Gulf Oil Spill was the first such incident for which the application was deployed, it has subsequently been used to allow the concerned community to share information for the recent flooding in Pakistan and wildfires in Colorado.

The development and release of ArcGIS 10 has really made this possible. Some of the new features like direct support of time-aware data in the ArcGIS Web APIs and the new ArcGIS Server feature service have made it much easier to envision and build such an application.

Throughout the implementations of these viewers, we’ve learned a lot about using social media and user-generated content to gain a clear visual picture and deeper understanding of specific events. Some of the important lessons we’ve learned are highlighted below.

The engaged community can really be a valuable source of information

From the earliest stages of envisioning, these applications were intended as a prototype to try to apply some of the new functionality in ArcGIS 10 in a way that tested the real-world validity of VGI. Past success stories like the boom of OpenStreetMap in Haiti (video) had already validated the argument that the engaged community can step in and generate great data on very short notice. This is especially true in a place like Haiti, where before the earthquake there were no economic factors driving the geospatial community to do so. There was very little geospatial investment in the country, and no market for commercial data. We wanted to test this concept in a new way and see if it could be useful.

What we found is that the community at large is very willing to contribute, especially for a cause they feel passionate about. Despite the prototypical nature of this app, we wound up with dozens of useful, unique user-generated links to other content on the Web about the Deepwater Horizon spill. This is in addition to the thousands of videos, photos, and tweets from social media outlets.

The primary thing we take away from this story is that VGI could be helpful to those responding to disasters or even just looking for a cheaper way to gather accurate data. In an economic climate where budgets are shrinking and organizations are looking for more cost-effective ways to generate accurate data, involving the engaged community to generate or police geospatial data can be very attractive. This can be especially useful if you can get the community involved in something they are passionate about, and leverage the increasing computing power they have in their homes, offices, or pockets.

It can be tricky to filter social media to display only relevant content

While working on this project, we frequently used the term “firehose” to describe the flood of data coming from social media outlets like YouTube and Twitter. One of the biggest challenges was using the filtering mechanisms within the various APIs for these media sources to limit the amount of data to only that which is significant for the event. This included the obvious keyword or hashtag (Twitter) searches, but also included filtering by geographic location + radius, and by time.

It can be a challenge to navigate the different APIs and find a combination that shows the most relevant social media for an event. Trial and error is the name of the game. What we generally settled on was a very generic search phrase like “oil spill”, coupled with a spatial search radius (where possible) and limiting results to those with a geospatial component (x,y) only. This allowed us to get only spatial media and easily add that to the map.

It is extremely important (almost mandatory) to set up filters and community policing to keep social media and user-generated content relevant

We learned two very important lessons when the application went live:

1. You never know what you’re going to get from social media outlets


2. If you allow people to add data, they certainly will!

Unfortunately, quite a few inappropriate tweets and videos were shown before we came up with a system for filtering out vulgar or offensive content. Much of this can be done by analyzing the text of the content in the client code. You can see our work in this area when you review the code for adding social media to the map.

In addition, a lot of the early user-generated data that was added was of limited value. Most users were just testing out the tools and seeing what they could add. Most of the submissions did not contain valid links to information or anything else that constituted a “contribution”.

We approached this problem by adding attribute validation both in the client code and in the geodatabase that stored the submitted data points. We checked to make sure that all the entry fields were filled out and that the URL was valid before the feature was checked into the database. This eliminated most of the unwanted data.

Another strategy we quickly adopted was allowing the community to police themselves. This is one of the key principles for crowdsourced data, and can be commonly seen in large sharing sites like Craigslist. The app includes a “report as inappropriate content” link at the bottom of the shared content pop-ups. If someone sees a submission that they feel is inappropriate, they can click this link to make it known.

When the link is clicked, the feature is flagged in the database for review, and a notification e-mail is sent out to the administrative team for the applications. Administrators can use ArcGIS Desktop or other Web apps to connect to the feature service or database layer, review the entry, and hide it from view if necessary. Future plans actually call for a strategy to use this on social media layers as well. Stay tuned to the application download page for more information.

Time is extremely important to incident data

As mentioned above, the ability to include a temporal component in the application data proved to be just as valuable as real-time editing. Time is an important aspect of the data for this application, and future plans call for the use of time sliders and other more advanced visualization tools. The current applications and the code download allow the user to filter the data for the past 24 hours or to show “all data.” This allows users to see the most recent data for the event at the click of a button and easily compare that to the data that has come in over the period since the incident occurred. Once the temporal information is committed to the geodatabase layer, it is an easy jump to using the great time-specific tools in ArcGIS 10 and the ArcGIS Web APIs.

Other people have a real need for applications like this

Since these social media/VGI applications have been live on Esri.com, we have received a lot of positive feedback from our user community. A common thread has been an interest from a diverse group of organizations on how to make similar applications available to their own user communities.

A great example came from a provincial government in Canada that needed to obtain place names from the native First Nations populations. The need was especially high in areas governed by the First Nations themselves. In the past, the province hired contractors to go out and interview tribal elders. With smaller budgets, this is no longer an option. An application that applies VGI could allow the elders to input the names themselves with nothing more than access to a Web browser.

Another use case involves conservation societies that send engaged citizens hiking through protected areas. These societies could provide basic Web apps that run on smart phones or tablets. Citizens could then report animal or plant sightings with a fairly accurate GPS coordinate of where the sighting occurred.

Both of these examples show how VGI can engage the concerned community to help generate useful data at minimal cost.

As a result of the great feedback we’ve received about this application, we have made the code available through ArcGIS.com. The application is provided with all code in HTML and JavaScript files, and contains a readme file to aid in setting up and modifying the app. Sample services have been set up to make sure the application runs right away when unzipped and deployed on a Web server. In addition, a sample ArcGIS 10 geodatabase and map document have been provided in the download bundle as an example of how you might design your own ArcGIS 10 feature service for use in VGI applications.

We encourage you to start using these resources right away to get your community’s help collecting useful data.

Contributed by Jeff Archer of the Esri Technical Marketing team