Tag: Feature layers

What’s new with hosted feature layers: Getting to know Append Part 1 (December 2017)

2017-12-08 09_10_02-Append_2.gif ‎- Photos

Managing your hosted feature layers is an important aspect of managing data in ArcGIS Online.  For many, when it comes to adding new data in bulk to existing hosted feature layers, the typical workflow is: download a copy of the … Continue reading

Posted in ArcGIS Online | Tagged , | Leave a comment

What’s new with hosted feature layer views (December 2017)

b09-thumb

Hosted feature layer views give you great flexibility, allowing you to tailor the data to meet specific user requirements.  But now hosted feature layer views are even better. We’ve just added a new capability that many of you have been … Continue reading

Posted in ArcGIS Online | Tagged , , , | 4 Comments

Taking control of transformations with hosted feature layers (September 2017)

2017-09-25 08_59_48-B08 - Taking Control of Transformations.docx - Word

Imagine how simple the world would be if there was only one projection.  In our quest to flatten the earth onto paper and device screens, we’ve created numerous projections to suit our purposes. Each projection has a set of goals, … Continue reading

Posted in ArcGIS Online | Tagged , , , | 2 Comments

ArcGIS Online and ArcGIS Enterprise Early Adopters Community Available Now

OAE

With the June 2017 Release of ArcGIS Online and the 10.5.1 Release of ArcGIS Enterprise, new functionality to collaborate between ArcGIS Online Organizations and ArcGIS Enterprise Deployments is available. This new capability provides the functionality to copy Feature Layers and … Continue reading

Posted in ArcGIS Enterprise, ArcGIS Online | Tagged , , , , , , , | 4 Comments

Who would win? Matching layer types to common workflows

thumb

One of the great things about ArcGIS Online is having the multiple ways of delivering your data, it ensures that you can tailor it different ways and deliver the best experience for your many different audiences.  In a sea of … Continue reading

Posted in ArcGIS Online | Tagged , , , , | Leave a comment

Tips for creating hosted feature layers

Layers5

Feature layers are often the best type of layer to use for visualizing your vector data in web maps. They are easily placed on top of tiled basemaps for geographic context and support interactive queries and selection, on-the-fly styling, and … Continue reading

Posted in ArcGIS Online | Tagged , , , , , | Leave a comment

Esri picture marker symbol generator for JavaScript developers

The ArcGIS.com map viewer provides a nice collection of picture marker symbols that can be used to symbolize features on your map. If you’re a developer, you might wonder how you can use these symbols in your applications. You could potentially use tools like Firebug or Chrome Developer tools to inspect ArcGIS.com and discover the image URL, but to make things easier, we’ve created a sample that allows you to select a symbol and generate the JavaScript Object Notation (JSON) needed to draw the symbol:

When using this sample, just click a symbol and Continue reading

Posted in Web | Tagged , , , | 3 Comments

Visualizing time-aware data in a web map

You may have heard that ArcGIS 10 is “time-aware”. This simply means that you can designate one or more fields in your data as containing date/time values, and then ArcGIS can play back the temporal progression of that data using a “time slider” bar. There are time sliders built into ArcMap and ArcGIS Online, but you can also add them to web apps that you build with JavaScript, Flex, or Silverlight. Showing the temporal progression of data like this was technically possible before ArcGIS 10, but it required a lot more coding.

To demonstrate one way that time-aware GIS data can be put in a web app, I created this JavaScript map of fish migration up the Columbia and Snake Rivers earlier this year. The dams along the river have fish ladders, where each fish is counted as it swims up the river. Visualizing the daily fish counts over time is an excellent way to see migration patterns. Take a few seconds to give the app a try.

Daily Fish Migration at Northwest US Dams

In this post I’ll explain a little about how I built this app and where you can get the code.

Getting the data and making it time-aware

The raw fish count data came from the Fish Passage Center website, www.fpc.org. Here I was able to download comma-separated value (CSV) files showing fish counts for each dam in 2011. These CSV files are similar to what you might be able to export from Microsoft Excel. To make this data work on a map, I also had to have the dam locations, so I created a point feature class and heads-up digitized each dam.

Digitized dams

Sometimes time-aware datasets are structured differently from traditional datasets you might use for a paper map. What I really needed for this web app was not one record for each dam, but rather, one record for each dam for each day. Thus, what looks like a simple dataset with only a handful of points, really contains over 800 records.

With a little bit of Python, I was able to read the fish counts out of the CSV files and write them as attributes for each dam for each day. Here’s a portion of the final table:

Attribute table of dams dataset

In order to make this type of data time-aware in ArcGIS, you have to add it to ArcMap. Then open the Layer Properties and click the Time tab. Here is where you tell ArcGIS about your dataset and how its time fields should be interpreted.

Time tab of Layer Properties dialog box

One of the first settings you need to choose is whether the layer has a single time field, or both start and end time fields. Some phenomena, like fish counts, are taken at one point in time and just have one field. Other things might have a field recording both the start and end time of an event, such as a wildfire that begins on June 9 and ends on June 13.

You can also indicate which field holds the time values, as well as the time extent and intervals that should be displayed in the time slider.

In my particular map, I wanted the same service to expose data about four fish types. So I made four copies of the layer, symbolized them with proportional symbols, and made them all time aware. Below is what the final map looked like. (It’s okay that all layers are turned on and overlap, because my app only requests one layer at a time from the map service.)

Map of dams in ArcMap before publishing

Exposing time-aware layers through a web service

At this point I was ready to publish the map as a service. I saved a map service definition (MSD) file (yes, these support time-aware layers) and published it to ArcGIS Server in the usual way. All of the time information saved in the map is accessible through the map service. You can verify this by looking at a layer in the Services Directory:

Time-aware properties viewed in the Services Directory

One easy way to view a time-aware map service is to add it to the ArcGIS.com map viewer (go to www.ArcGIS.com and click Map to get to this viewer). The viewer recognizes any time-aware services you add and automatically gives you a time slider. In my case, I wanted to go a little further and make my own web app with a time slider. I chose the ArcGIS API for JavaScript because I had a bit of previous experience with it, and my adjacent co-workers happen to know this API very well. (I can’t deny asking for the occasional hint!)

Building a time-aware web map with JavaScript

When you’re starting out with the ArcGIS web APIs, a good strategy is to find some of the online samples that do the things you want to do, combine them into an app, and then modify the result to fit your requirements. I decided I wanted to use elements from these two samples:

  • Overview map sample from the Layouts book. You’ll see this is where I got the overview map, color scheme, styling, and partitioning of the window.
  • Layer Definition sample from the Time book. This is where I got the time slider and the legend.

The fish counts update very quickly because the app uses client-side graphics to display them. In other words, the browser makes a request to the map service saying essentially: “Give me all the point vertices for the dams from April 17 – June 30, and their attributes. I’ll do the work of drawing them.” After the server sends the response, the browser can draw the points and their symbols without ever having to go back to the server for more information. Because I’m dealing with fewer than 1000 records, I don’t have to worry about hitting the map service’s default limit for how many features it will return, but at the same time I am not requesting an unreasonable number of points for the browser to display.

The key to displaying the graphics this easily is to use the FeatureLayer class. Here’s where I create it in my code:

var fishLayer = new esri.layers.FeatureLayer(initialFishLayerUrl, {
  mode: esri.layers.FeatureLayer.MODE_SNAPSHOT,
  outFields: ["*"]
});

When you connect a FeatureLayer to a TimeSlider, the time slider does all the work for you of requesting and displaying the appropriate features for the currently displayed time slice.

The time slider is a widget included with the ArcGIS API for JavaScript. To use it, I had to put the following at the top of my code:

dojo.require("esri.dijit.TimeSlider");

Then, I had to create the time slider and set some properties on it, such as the thumb moving rate (how fast the slider plays) and whether the time slider should replay itself when it gets to the end (accomplished with the setLoop method).

// Create and configure the time slider
var timeSlider = new esri.dijit.TimeSlider({
}, dojo.byId("timeSliderDiv"));
map.setTimeSlider(timeSlider);
timeSlider.setThumbCount(1);
var layerTimeExtent = results[0].layer.timeInfo.timeExtent;
timeSlider.createTimeStopsByTimeInterval(layerTimeExtent, 1, 'esriTimeUnitsDays');
timeSlider.setThumbMovingRate(350);
timeSlider.singleThumbAsTimeInstant(true);
timeSlider.setLoop(false);
timeSlider.startup();

If you use the time slider in the ArcGIS API s for Flex or Silverlight, you’ll see similar properties.

You can handle events on the time slider to control what happens when the “thumb” on the slider moves. The drawing of the fish counts is handled automatically, but I did add some code to change the date label that appears directly below the slider.

// Report the date whenever the time slider moves
dojo.connect(timeSlider, "onTimeExtentChange", function(timeExtent){
  map.infoWindow.hide();
  dojo.byId("timeSliderLabel").innerHTML = dojo.string.substitute("<b>${startTime}, 2011</b>", timeExtent, function(val) {

      return dojo.date.locale.format(dojo.date.add(val, "hour", hourOffset), {
      selector: 'date',
      datePattern: 'MMM dd' // http://www.unicode.org/reports/tr35/tr35-4.html#Date_Format_Patterns
    });
  });
});

The above code results in a label that changes as the slider moves:

Label that changes with the time slider

Conclusion and download link

This post has shown how you can start with raw tabular data and ultimately display it in a time-aware web application. The coding part is optional and can be eliminated if you want to use the time slider in ArcMap or in ArcGIS Online. However, because many of you may have to write similar types of web apps, I’ve made the complete application and a readme file available for download at http://esriurl.com/fish.

Contributed by Sterling Quinn of the ArcGIS Server software development team

Posted in Services | Tagged , , , | 5 Comments

High performance web apps with big datasets as feature layers

High performance feature layer applicationDealing with big datasets is a frequent topic of discussion among web mapping app developers. The prevalence of huge, crowdsourced datasets ensures that big data will continue to be a common point of discussion when you get more than two web mapping geeks in the same room. The traditional approaches of generating a raster tile cache or using a dynamic map service work, but if you want any interactivity with the underlying data, it requires a server round trip. This post will discuss some of the key concepts behind building a web app that provides an extensive interactive experience while maintaining responsiveness and performance.

The main factor causing big data to be a tricky topic is that browsers cannot elegantly handle upwards of a few thousand features. Send more than a few thousand points, or a few hundred polygons to a browser and you can watch it grind to a halt, especially if you’re running a legacy version of Internet Explorer. This is a problem because slow performance is a death knell for web apps. If someone using your web app has to wait ten seconds, twenty seconds, or longer for five megs of JSON data to be retrieved and displayed, there’s a good chance they’ll just give up and move on. What good is a web app that no one uses?

At some point, you cross a threshold where it’s not practical to send all of your data across the wire and let the client sort it out. That threshold is an ambiguous one (and was explored in a previous post), but you know when you hit it. If you’re a JavaScript developer, how should you deal with big data? If you’ve read my recent posts, this answer will not surprise you: Feature Layers.

Building on the concepts discussed in the previous feature layer posts (generalizing features and vector tiling) and client side graphics performance post, I’ve put together an application that demonstrates how to use Feature Layers to set up custom scale dependencies and follow best practices to ensure performance does not suffer when querying, retrieving and displaying large datasets in an ArcGIS API for JavaScript application: High Performance Maps with Feature Layers.

The goal was to display appropriate US boundaries (states, counties or census block groups) for the current map scale. For example, trying to display census block groups for the whole US would overwhelm the browser with features. This app makes sure that only the states appear at the small scales, counties at the medium scales, and census block groups at the large scales. This ensures that a manageable number of features are always being transferred and displayed.

The app is simple, but the concepts can be applied to just about any web mapping app that has to deal with big data. The total size of the data used by the app is around a couple of hundred megabytes. This is accomplished through custom scale dependencies. This is as simple as creating a few feature layers, listening for their onLoad event and setting their minScale and maxScale properties. Here’s a simple code sample showing how this is done:

var fl = new esri.layers.FeatureLayer(url, options);
dojo.connect(fl, ‘onLoad’, function() {
fl.minScale = minScale;
fl.maxScale = maxScale;
});

The app includes a couple of other bells and whistles that demonstrate some of tools that are available when you have your features client side. The first is a rich Popup that displays feature attributes, as well as an option to zoom to the feature when you touch a graphic. The second is the ability to filter features based on population using a slider. Neither of these would perform nearly as well if features weren’t available client side.

The final point to drive home is that the app’s performance is kept snappy because only the required data is being sent to the client. For geometries, that means using maxAllowableOffset to generalize geometries on the server before they’re sent to the client (see my previous post on maxAllowableOffset and generalizing features on the fly for more info). For attributes, only the fields required by the Popup are sent across the wire.

Contributed by Derek Swingley of the ArcGIS API for JavaScript development team

Posted in Services | Tagged , , , , , | 2 Comments

Determining limits for map graphics

We’ve recently been talking about some of the benefits of Feature Layers. Since Feature Layers are a subclass of GraphicsLayer, we thought it was appropriate to spend a little time talking more about map graphics, limits on how many can be added to a map, and how to find the appropriate limit for your app/environment.

“How many graphics can I put on my map?” or “How many points can I put on a map?” are questions that continue to be asked. Probably because we keep saying “it depends.” While technically correct, “it depends” is not helpful on its own. So, with the lofty goal of providing some actionable guidelines, I’d like to discuss two scenarios: point graphics and polygon graphics. To aid in this discussion, I’ve put together a simple app for each scenario: Display an Arbitrary Number of Points and Incrementally Add U.S. County Polygons To a Map.

The objective of the point graphics app is to see how many graphics you can add to a map before performance starts to suffer. Since this is dependent on many factors (your browser, your hardware, your definition of “suffer”), it’s probably best to let you see for yourself how a map performs with 100, 1,000 and even 10,000 graphics. I wouldn’t try 10k in IE, but Chrome and even Firefox hold up well (at least on my machine, YMMV).

There is also an option to select symbol type. Not all point symbols are equal in terms of performance so it’s important to realize that 500 picture marker symbols might result in a more sluggish app than one with 1,000 simple marker symbols.

But wait, just because you can put 3,000 points on your map and not see a performance penalty, does that mean you should? My answer, nine times out of ten, is no. Performance is one thing but user experience is another. Does putting a map with thousands of point graphics on a 15” screen provide a meaningful user experience? Again, I would say no. Outside of looking for a general spatial distribution pattern, I don’t think you’re helping a user discover much or answer any particular question by showing thousands of point graphics at one time. If you find yourself in this situation, it’s a good time to consider using a raster surface (like a heatmap), filtering your data through queries, or maybe clustering.

The story for polygon graphics is a bit more complex. Not only do you need to worry about the number of polygons, but the number of polygon vertices is also an important consideration. More detailed geometries take longer to transfer to the client, and once they make it to the client, they require more resources to display.

Like the point graphic app, the polygon app linked above lets you add any number of polygons to a map (but you probably should keep it under 1000), and it also allows you to specify whether or not maxAllowableOffset should be used. If maxAllowableOffset is a new term for you, please read my previous post to catch up.

Let’s look at a couple of reasons to always use maxAllowableOffset. The image below shows geometries for some states in the Northeastern United States:

Northeast states graphics comparison

The geometries on the left were fetched using a query that set maxAllowableOffset appropriately. The geometries on the right are not generalized. The generalized geometries look nearly identical to the non-generalized geometries (maybe even better according to some) but they are less than a quarter of the size. That’s the key takeaway and why we use maxAllowableOffset: transfer the minimum level of detail required for effective display.

One more example:

Mississippi graphics comparison

I think this example really drives the point home. Using maxAllowableOffset, you’re transferring ~44KB to show the county boundaries for Mississippi. Without maxAllowableOffset, you push over 3MB across the wire. If you’re setting maxAllowableOffset correctly, you’re not likely to be sending more than 100KB (and usually a lot less) per query to the client regardless of the level detail present in your geometries.

The total number of polygon graphics on a map is something that you should experiment with but I would recommend to keep it below a few hundred.

We still don’t have a hard and fast rule for the number of graphics on a map (and probably never will), but hopefully this post along with the accompanying apps will make it easier for you to find your own answer.

Contributed by Derek Swingley of the ArcGIS API for JavaScript development team

Posted in Services | Tagged , , , , | 6 Comments