Thank You – Latest Innovations from APL session @EsriDevsummit

A huge thank you to all of you who attended our sessions yesterday at Esri Developer Summit in Palm Springs. There were many topics and technologies covered during the sessions and we hope that you found our work interesting, informative and worthwhile. For those who didn’t get a chance to attend our sessions, looks like one of the session got recorded so you might get a chance to catch up soon via online videos soon.

We were enthused and thrilled to have such huge attendance for both the sessions, you were a great group and your enthusiasm and comments helped make our time together both productive and fun.

Please join us at the Esri Labs area in the upcoming Esri User Conference this year at San Diego for more awesome stuff we are working on. Follow us on @esrilabs twitter account for more updates and details closer to the event.

Here are a few pictures and tweets about the sessions yesterday, Thanks and see you all soon!


Click here to view tweets on Storify

Posted in Uncategorized | Tagged , | Leave a comment

Proximity Map for Windows Store

YouTube Preview Image

This video demonstrates peer-to-peer collaboration in a mapping application developed using the ArcGIS Runtime SDK for Windows Store apps.  The app uses Windows 8 native proximity capability to establish an ad hoc Wi-Fi Direct connection that is fast, low latency, secure and doesn’t require any existing network.

The first part of the video shows extent synchronization or “shared navigation”.  As a map on one device is panned or zoomed, its extent is immediately broadcast to the other device.

The second part is somewhat more advanced.  We developed a map-friendly overlay, similar to a graphics layer, that could broadcast graphics as they are being digitized.  Graphics, or partial graphics, are serialized to JSON and transmitted to the other device.  Note: the ability to digitize to a graphics layers will be part of the next release of the ArcGIS Runtime (see Map.Editor.RequestShapeAsync()).

The last part of the video shows the ability to offset one screen from the other.  Any extents that are transmitted or received are automatically shifted to give the illusion that the screens are connected.  This would be useful if someone wanted to create a single map display from multiple devices.

We have shown that map collaboration with Windows 8 is viable even in the absence of a local network.  The proximity capability at Windows 8.0 is still rather limited, for example, only two devices can participate in a Wi-Fi Direct connection.  Later this year when Windows 8.1 is released, proximity will be expanded to support ad hoc networks with two or more nodes using either Wi-Fi or Bluetooth.

Cinematography by Mark D.

Posted in Uncategorized | Tagged , | Leave a comment

QR Map for Windows Store

YouTube Preview Image

This video demonstrations a proof-of-concept that uses a dynamic QR code to communicate map information from a presenter’s screen to mobile devices.  Whenever the presenter’s map changes, the map’s coordinate extent is compressed and encoded into a QR code using the ZXing.Net.

The same app used by presenter is also installed on the mobile device, in this instance, a Microsoft Surface.  Rather than a “presenter mode”, the app on the mobile device is placed in a “following” mode, this turns on the devices back facing camera and uses ZXing.net to scan for, and decode, coordinate extents.

We agree that by itself it may seem somewhat unremarkable.  We found it intriguing because the communication, while one way, was established without traditional connectivity such as Bluetooth or Wi-Fi.  This technique illustrated here may be useful in classroom settings, seminars and briefings.

References:

Video filmed by Mark D.

Posted in Uncategorized | Tagged , , , | Leave a comment

Geometric Network Configuration Manager

This post discusses the release of a sample utility called Geometric Network Configuration Manager.  In some instances when it is necessary to temporarily remove a geometric network, this tool can recreate the geometric network from a backed definition file.

The add-in can be downloaded from here.

The source code is available here.

Configuration Manager has a very long lineage.  More than a decade ago the Prototype Lab published Geodatabase Designer for documenting geodatabases and exchanging schema.  Designer is now obsolete but fortunately much of its capabilities are now incorporated into ArcMap or other tools such as ArcGIS Diagrammer and XRay for ArcCatalog.

However the one feature from Designer that has yet to be replicated is the ability to save and restore geometric networks.  This useful if loading large amounts data or performing a schema change like switching a feature class from a simple edge to a complex edge.

Prerequisites

Libraries Used

The following section will walkthrough the steps required to backup geometric network, remove it and then restore it.  The use case for this workflow could be for bulk data loading or transferring a geometric network from a test server to a production server.

Walkthrough

Following the successful installation of the add-in.  Display the Geometric Network Tools toolbar and click the first button to launch the main dialog.  Drag and drop a geometric network into the configuration manager window.

The complete definition of the geometric network will be loaded into the dialog.  The four tabs below the ribbon make it is possible to review and, in some cases, modify classes, weights and connectivity rules.

The geometric network definition is current stored in memory and should really be saved to a file so that it can backed-up or restored at a later time.  Click the save or Save As button to export the definition to a file with a esriGeoNet extension.

Recreating a geometric network is just a matter of loading an esriGeoNet file and clicking Export.  The application will prompt the user for the name and location of the exported geometric network.  If the geometric network already exists, it will be overwritten.

Known Issues:

  • The dialog that appears when the Export button is click may be hidden by the Configuration Manager window.  Either minimize or move the window to the side to continue with the export operation.
  • Add and removing of network classes and weights is currently not supported.
Posted in Uncategorized | Tagged , , | Leave a comment

Augmented Map for Windows Store

YouTube Preview Image

The Applications Prototype Lab is proud to present a preview of “augmented map”, a windows store based augmented reality app.  The purpose of this app is to provide an intuitive alternative to image discovery and interrogation.

The app is running on a Windows Surface and the only other peripheral is a block marker printed on a white piece of paper.  This marker orients the map displayed in the video feed of the augmented reality app.

In the demonstration video, the user starts by zooming to, and requesting imagery for, New Zealand’s capital.  LandSat preview images are downloaded and placed on the map.  Because there are potentially hundreds or thousands of satellite images, the user will use augmented reality to sort through the collection.  The demonstrator first chooses to sorted and stack the imagery based on the date of capture with the newest images located at the top.  Detailed information such as the sun’s position and percentage cloud cover is available by tapping on an image.  To analyze older images, the user applies a filter to progressively hide images from top to bottom.  Lastly, the user sorts images based on percentage cloud so that the clearest images are at the top of the stack.

This project, to a large extent, is derived from the Silverlight-based app completed almost three years ago.

Data used:

Technology used:

Posted in Uncategorized | Tagged , | Leave a comment

Distributive Flow Maps – More Raster, More Faster

NOTE:  The Distributive Flow Lines tool was updated  Jan 10, 2014 .  If you are using ArcGIS 10.1sp1 or 10.2 please download the latest version.

Based on comments and feedback from the first version of the Flow Map tool and blog post we gave it a second look and have released an update.  As you can see in the screen capture above, the output is similar to the original tool but we have switched to an all raster approach which provides more control over the lines and reduces processing time.

A distributive flow map made by Charles Joseph Minard showing coal exports from England in 1864.

For convenience, I have placed the introductory text from Brad Simantel’s original post below to provide a concise introduction to the history and use of Flow Maps.

“Flow maps are used to show the movement of goods or people from one place to another. These maps use lines to symbolize the movement, often varied in width to represent the quantity of the flow, and fall into one of three categories: radial, network, and distributive. Radial flow maps are used to show relationships between one source and many destinations. If there are more than a handful of destinations and you still want to show the quantity of flow, however, the lines overlap too much to discern individual values. Network flow maps are used to show the quantity of flow over some existing network — transportation and communication networks being the most common. Distributive flow maps are similar to radial flow maps, but rather than having individual lines from the source to each destination, lines are joined together, only forking once they get close to their destinations”.

If you would like to try it out as you read this post, you can access the new  Distributive Flow Lines tool (DFLT) on ArcGIS Online. If you do not have data handy, the British Coal export data, used to create the first map in this post, is also available for download on ArcGIS Online.

Tip: The DFLT requires the Distributed Quantity Field to be of type Integer.  If you use the British Coal sample data a new integer field will need to be added and calculated based on the existing CoalTonnage field.  The current field represents thousands of tons so multiplying by 10 and rounding to the nearest integer would represent hundreds of tons rather than thousands.

Like the first Flow Map tool, the Spatial Analyst extension is required to use the new DFLT.   In fact, the new tool is completely raster based until the end when the flow line feature class is created as output.

The DFLT will accept polygon or point features as input for the Source and Destinations, however, point features are recommended.   If polygons feature classes are used as input for either Source or Destinations they will be converted to points for internal use but these intermediate point datasets are not saved as part of the tool output.  After the Destination feature class is selected the user will need to select an integer field from this dataset representing the quantity that will flow to the destinations from the source.

The DFLT provides an option for the user to specify a feature class to use as impassable features.  These features will be buffered by 1.4 times the Cell Size parameter.  The area inside this buffer will be NoData and will be used as a processing mask within the tool.  A second, slightly larger buffer, 3 times the Cell Size parameter, is also created. The area between the inside buffer and this larger buffer is given a very high cost so Flow Direction will not calculate routes that go into the NoData area (most of the time).

The second optional parameter is a Impedance feature class.  These polygons represent  features in your map that you would like flow lines to avoid as much as possible.  How much these features are avoided is controlled by the Impedance Weight slider.  When impedance features are specified, they are given a high cost; but may still be crossed in extreme cases where there is no alternative or where they can be crossed in a narrow location as shown in the example below.

Tip: One common effect of the DFLT is that it will often skirt Impedance features too close.  This can be avoided by buffering the Foreground features and then using this new feature class as the Foreground features in the tool.

Since the primary use case for the DFLT is to create pleasing flow lines, the user will most likely run it several times using different values for the weight sliders to achieve a good starting point for the effect they are looking for.  Using an all raster approach makes this process a bit easier and faster.  In raster processing, cell size has a major effect on processing time. The DFLT calculates a default cell that size works well for the final product.  In most cases it will not be necessary to use a smaller cell size than the default. We recommend using a much larger cell size during initial iterations to reduce processing time until you start getting results close to what you want.

Tip: It is a good idea to make a note of the default cell size for final processing.  For initial runs of the tool,  multiplying the default cell size by 5 or 10  works well to get a feel for how the various weights and optional parameters effect the output flow lines.  More drastic changes in cell size may have a significant effect on how the lines are routed around Impassable and Foreground features.

Tip: If you use the post 10.1 sp1 version of the tool you will notice the tool dialog has been simplified.  Source Weight and Destination Weight sliders have been replaced by a single slider to control how close to the Source or Destinations the flow lines branch.  Also processing extent options have been reduced to “Same as Display” and “Same as Destinations plus Source.”  This was changed to prevent most of the cases where no solution is possible.

 

Cell size is so important in the DFLT because the tool calculates Euclidean distances from source and from all of the destination features. These two distance rasters are then Sliced into an equal number of discrete cost classes.  The number of classes is determined by dividing the maximum processing extent dimension by the cell size.  This acts to normalize the effect of the Source and Destination cost rasters and provide a basis for the cost of the Impedance raster.  By normalizing these costs the tool provides the user more control over the shape of the final flow lines through the Source and Destination Weight sliders.

To understand how cell size affects the total cost surface it helps to see a few examples.  In the following ArcScene screenshots we are using a Source location in the center of the US and the destination features are the centroids of a several countries in Africa. The CostDistance surface is shown floating over the “flat Earth” to provide a means of comparing the effects of the weight parameters and cell size. In each example the perspective and Z exaggeration are the same.

Source Attraction = 10 Destination Attraction = 1 Foreground Weight =8 Cell Size = 100,000 meters

Source Attraction = 10 Destination Attraction = 1 Foreground Weight =1 Cell Size = 100,000 meters

Source Attraction = 1 Destination Attraction = 1 Foreground Weight =10 Cell Size = 100,000 meters

Source Attraction = 1 Destination Attraction = 10 Foreground Weight =8 Cell Size = 100,000 meters

Source Attraction = 1 Destination Attraction = 10 Foreground Weight =8 Cell Size = 30,000 meters

In this last example, it was necessary to reduce the elevation exaggeration and to zoom out so the cost surface would display in a reasonable way.  The only difference between the two results is that the cell size has been reduced from 100,000 meters to just 30,000 meters.  The effect is that the number of “cost slices” is larger and so the CostDistance “elevation” is increased.

As previously stated, the new DFLT processes the Impassable features separate from Impedance features.  Impedance features are treated much the same as Impassable features in the first tool.  They are features the user would like the flow lines to avoid if possible but the flow lines will cross them if necessary to reach a destination location.  Impassable features represent a hard barrier in the cost surface and can be used to control the shape of the flow lines as in the example below.

Flow lines representing distribution of Coal by ship are crossing Central America regardless of Impedance weight

With the red barrier added flow lines now go around the tip of South America

As with the previous Flow Map tool, the output of this new tool will usually be the starting point to produce a final flow map with smoother more organic looking flow lines.  Once the lines are roughly where you want them the next step is to use Graduated Symbols to display the Distributed quantity.  The “GRID_CODE” field will contain values based on the input Distributed quantity field.  The values in this field will be equal to the Distributed quantity field for the smallest lines nearest to the destinations.  Each time a tributary flow line meets another line the combined line closer to the Source feature will have a GRID_CODE value equal to the sum of previous two tributaries. Temporarily placing labels on the lines and destination points makes this clear. In the examples above the lines are represented using 10 classes of line weights ranging from 1 – 10. Using rounded joints and ends on the lines also improves the output. After setting up graduated symbols many users may also prefer to run the Smooth Line tool on the flow lines. This will also add some curve to the flow lines which is often desirable. A good starting point is to use a tolerance value approximately four times the Cell Size used to create the flow lines. It is also recommended to select the PEAK algorithm and FIXED_CLOSED_ENDPOINT options. Even after these steps it may be necessary to make manual adjustments to the shape of the line.

Installation instructions:

After downloading and unzipping the Distributive Flow Lines tool you should see the following in the ArcMap catalog window.

Expanding python toolbox will reveal one script tool as shown below:

By default, python toolboxes do not display the “pyt” file extension. To display this and other known extensions in ArcCatalog (or the catalog window in ArcMap) check the following:

This dialog is accessible in ArcCatalog by clicking Customize > ArcCatalog Options.  Unchecking the highlighted option will show the following:

We hope you find the new tool useful and the tips above help get you started experimenting with the tool.  We look forward to you comments.

NOTE:  The Distributive Flow Lines tool was updated  Oct 22, 2013 to address a backwards compatibility issue.  If you are using ArcGIS 10.1 please download the latest version  here.

NOTE:  The Distributive Flow Lines tool was updated  Jan 10, 2014.  If you are using ArcGIS 10.1sp1 or 10.2 please download the latest version.

Contributed by Bob Gerlt.

Posted in Uncategorized | Tagged , , | 22 Comments

iPad display wall showcases Urban Observatory project

YouTube Preview Image

After helping with the Urban Observatory exhibit at the User Conference, 2013 we were inspired to bring the same concept to mobile devices. This would provide for the same visualization and comparison of different themes across different cities around the world.

The Urban Observatory works in a grid format. All the cities to be compared would be in the bottom most row. Each had the same theme such as housing density, traffic etc,. and the map scale in each device was synchronized. As devices are stacked horizontally they would load a user defined city with the same theme and the same map scale as adjoining iPads.  Likewise, vertically stacked devices would display the same city but with a different theme as defined in that row.

We replicated this concept by using multiple iPads. As soon as we put down a new iPad they would realize their position with respect to the previous iPad and would pull in the correct data. All the devices synchronized the map scale and only the same city iPads synchronized the map extent. You can change the city on one device and all the other ones would update accordingly to show a different city. We also brought in the concept of using the iPhone as a remote control to manage the themes and the cities on all the iPads. The top row of iPads could also serve additional information about the cities. We had them show the fly by video of each city shown in the iPad below.

Technical Details :

This application was written using ArcGIS Runtime SDK for iOS.

All the transactions happened using the Gamekit framework for iOS. This helped us automatically setup a sessionID to which other iPads would connect to. Using CoreLocation, we also knew their relative positions and we could connect a lot of these devices together and they automatically knew what data to pull from ArcGIS Online. They established a peer to peer networking using the bluetooth or the internet when available.

The original content for the Urban Observatory, its goals, objectives, sponsors and contributors, could be accessed at http://www.urbanobservatory.org.

Posted in Uncategorized | Tagged , , , , , , , | Leave a comment

Where Did Water Flow on Mars? Displaying 2D Maps on a 3D Planet

In the 2008 article ‘Where Did Water Flow on Mars? Modeling Mars’ surface in search of ancient rivers and oceans’ Witold Fraczek demonstrated how GIS can furnish support for the theory that at some time in the past, water did flow on the Martian surface.  By utilizing NASA’s available Martian DEM and other supporting data layers, a hydrologic network was created by running a series of hydro functions. For this analysis, a selected section of the Martian DEM was treated in exactly the same way that a DEM from Earth would have been handled. A series of cylindrical projections were then exported from ArcMap and wrapped around 3D spheres to represent Mars. These 3D planet models were then imported into CityEngine as Collada where small selectable domes were added to represent the many probes that have successfully landed on Mars. Finally this model was exported as a 3D Web Scene and uploaded to ArcGIS online to easily share with the public. Since 3D Web Scenes are based on WebGL technology, no plug-in is required for most browsers.

Read more about how GIS helped to derive the Martian Ocean: http://www.esri.com/news/arcuser/0408/mars.html

Exporting to a 3D Web Scene is currently available for CityEngine, ArcGlobe and ArcScene. 3D scenes and the ability to publish directly on the web is revolutionizing the way we share, collaborate, and communicate analysis results or design proposals with decision makers or the public. After all, our world is in 3D.

ArcMap is used to analyze the digital terrain model for Mars’ hydrological network.

The cylindrical projection is then wrapped around a 3D sphere and imported into CityEngine as Collada.

CityEngine was then used to export the layers to a 3D Web Scene which is now hosted on ArcGIS Online.

You can view the Web Application here.

Credits: Dr. Witold Fraczek (APL), Brooks Patrick (City Engine)

Posted in Uncategorized | Tagged , , , | Leave a comment

Watch our Developer Summit Presentation!

At this year’s developer summit, the Lab presented a session entitled “Latest Innovations from Esri’s Prototype Lab”.  This sessions consisted of a brief introduction to 23 of our most recent apps/demos/prototypes in less that 60 minutes.  As of today, this session is  available for download or as a streaming video.  Click here for details.

Posted in Uncategorized | Leave a comment

Raster Shader Addin for ArcMap 10.1

YouTube Preview Image

The Lab is pleased to announce the release of Raster Shader, a new addin for ArcMap 10.1!  Raster Shader is a tool for creating full-color, relief-shaded visualizations of single-band raster datasets.  Similar to the Image Analysis Window, Raster Shader leverages the power of raster functions to process the data dynamically as it is displayed in the map.  Since the processing occurs entirely in memory, it is extremely fast and does not affect the source datasets.

The idea for this tool came from projects we’ve done over the last couple of years involving large collections of single-band raster datasets containing climate data such as temperature averages and precipitation totals over time.  Our standard approach for visualizing these datasets is to use chained raster functions to apply a combination stretch/colormap renderer.  However, building and configuring raster function chains which produce an attractive and meaningful visual result can be a challenge if you are not familiar with the concepts and techniques involved.  This is where Raster Shader can help.  Not only does it build the raster function chain for you.  It also provides a simple interactive UI for adjusting key rendering parameters so you can see the results immediately in the map layer.  An interesting feature of Raster Shader is a histogram which shows how the colors in the colormap relate to the distribution of values in the data.  This relationship is based on stretch parameters which you can adjust with slider bars and/or text boxes.  Optionally, if the map contains a layer based on a DEM, you can add topographic context to the data with a hillshade effect and easily adjust parameters such as sun azimuth and sun angle.

Click here for the addin.

Posted in Uncategorized | Leave a comment