What’s new with hosted feature layers: Getting to know Append Part 1 (December 2017)

Managing your hosted feature layers is an important aspect of managing data in ArcGIS Online.  For many, when it comes to adding new data in bulk to existing hosted feature layers, the typical workflow is: download a copy of the data, reconcile the new additions (and any additional edits), and overwrite the layer in ArcGIS Online. In many cases that workflow works; for others, it creates extra steps for an otherwise simple task.

One of the best examples is when working with data that has periodic updates. That’s where the new Append Data to Layer tool (Append) really shines.  With Append, you can simply insert the new data into the layer instead of exporting the layer, merging it with the new data, then overwriting.

Append allows you to upload and append new data to an existing layer from a file.  That file can be a feature class in a geodatabase, a shapefile, or in the case of point layers; CSV(be sure to include x,y fields in lat/long format; don’t worry, we’ll be working toward other coordinate formats in a future release).   Another scenario where it would make sense to append vs overwrite is when your feature layer is large.  Overwriting the entire layer when it’s very large (say millions of features) can take time.  If the number of features you want to append into the layer is proportionally smaller, it is much faster to use Append instead of overwriting.

To get started, go to the item page for your hosted feature layer.  From the Overview tab, you’ll notice a new menu button on the right called Update Data. Click it and you’ll find.  Append is also accessible from the Data tab.

After clicking Append Data to Layer, you’ll be prompted to upload the file that contains the data you want to append.  Supported formats include:

  • File Geodatabase (.zip)
  • Shapefile (.zip)
  • CSV (.csv)

One quick note at this time you can’t append data to a hosted feature layer that has synchronization or change tracking (Keep track of created and updated features) enabled.

To save time and to reduce errors, it is strongly recommended that the schema of the uploaded file matches the schema of the hosted feature service.  Let’s look at an example.  NASA publishes a daily active fires dataset, which can be crucial in mapping fire perimeters and visualizing the day-over-day change.   By using Append instead of publishing new layers, you can store all that information in one layer, with the bonus of being able to use the time slider to visualize daily changes.  You can just download the file and append the new data daily (in the case of the active fires dataset, make sure you take a few extra seconds to define the projection before uploading). By default, ArcGIS Online will attempt to match fields to your feature layer. If you want to change that, just click Show Field Mapping and change the defaults, or unmatch fields you don’t want used.  When you’re happy with the selected options, click Apply to append the data.

You might have noticed a little check box that was skipped that said, “Update existing features”.  While Append is primarily about adding new features into a layer, it also can append updates into existing features.  I’ll cover that option in more detail in a separate blog article (Part 2). So, stay tuned for more details and examples.  In the meantime if you’re looking to read more on managing hosted feature layers you can find additional reading here.

This entry was posted in ArcGIS Online and tagged , . Bookmark the permalink.

Leave a Reply


  1. tig.asset.management says:

    The Update Data is not showing up on my Hosted layers on AGOL. Is there something I need to enable to make that appear? The layer is already set as editable.

    • Paul Barker says:


      The most common reason for the menu to be hidden is if the hosted feature layer has “sync” or “keep track of created and updated features” turned on. We’re working to remove that restriction in a future release. The other scenarios are: if the hosted feature layer is a view or if you do not own the service. Check those, and if none of those apply please reach out to support and we can investigate further.

      • wiscrapids says:

        Can these types of details that restrict applicability be included in blog posts by default? A guess at expansion of a new feature (weeks vs. years) would also be useful. As an example, GIS administrators may want to delay staff training on the current process because a new method available in Hosted Layers would simplify the process considerably and would be easier for end users.

        • wiscrapids says:

          Noting that the restriction is included, could those restrictions be bolded or otherwise emphasized (preferably in a consistent manner across blog posts advertising new functionality)?

      • tig.asset.management says:

        Gotcha. Working now. Thanks

  2. mstarry_mnpower says:

    Can this be scripted?

  3. kulasdn says:

    Thanks for this post (and for Part 2 as well), and for adding this functionality! Just so I’m clear, “One quick note at this time you can’t append data to a hosted feature layer that has synchronization or change tracking (Keep track of created and updated features) enabled.”

    To reiterate your answer to a previous poster, ESRI is planning to remove the sync restriction in a future release… can you give any indication as to how soon/not soon this will be? It would be nice to have an idea for workflow planning purposes. And once this sync restriction is lifted for appending data, would that also apply to the overwrite capability as well? I’d love to one day be able to tell my staff they don’t have to delete and re-download maps in collector every time I update reference data quarterly. Any insights would be greatly appreciated!

    • Paul Barker says:

      That’s certainly where we want to get to. I can’t give you a timeline at the moment unfortunately but removing restriction is something we’re actively working on.

      • kulasdn says:

        Great, thanks! One other question… for the overwriting portion, I see in the help that the ‘Overwrite Entire Layer’ option is not available if the service was published by ArcMap originally. Is this going to change at any point? I published most of my layers via ArcMap in the early days because I felt I had greater control of the data, but it would be nice to overwrite them directly through ArcGIS Online now that they’re hosted and configured properly. Will more flexibility be coming with the overwrite option as well as the append function?

        • Paul Barker says:


          That’s something that has been discussed some but no firm plans at this time to change. I would be interested in learning more about your ideal workflow for this though. If you have time can you write your workflow up and share it with support as an enhancement request.

          • kulasdn says:

            Thank you for the clarification. My ideal workflow is to be able to overwrite an entire sync enabled layer, and/or append via update existing features regardless of whether it was created via ArcMap or via ArcGIS Online. For appending being able to choose our own unique ID for matching is ideal, which I believe you stated is an eventual goal.

            I work for a state agency that tracks rare and endangered species, so our field staff absolutely need to have their maps downloaded in Collector to capture data because they are often in remote, rugged areas with no service. I have set up a map in Collector with some editable layers for them to capture their location tracks as well as species information, along with 7 or 8 reference layers. These reference layers come from our database and are meant to help folks orient themselves in relation to our existing rare species info, and while the live database is constantly updated, I update these reference layers quarterly in ArcGIS Online.

            We don’t have server so I’m not aware of a way to automate this… my current workflow for updating the reference layers is to warn folks a week ahead of time that I’m going to break their map connections and they need to have their collections synced by X day. Then on said day I remove the ‘sync’ function for all the reference layers to delete the replicas and overwrite with the updated database layers. I like overwriting rather than publishing new layers because webmap settings are retained. When I’m done and everything looks good I email folks to say ‘all clear’ and they can re-download new maps. This is not a big deal for our field biologists who are constantly going to new places and downloading new maps all the time, but for our Natural Area Preserve stewards who always visit the same preserves in their region, re-downloading the same maps over and over is a pain I’d imagine.

            I got excited when the append function came out because I thought this would solve some of my workflow issues, but it doesn’t since 1) replicas exist and 2) I published these reference layers with ArcMap in the first place.

            Further, I noticed when I was testing the ‘update existing features’ function (which we would need if appending due to revisits to existing locations), it didn’t seem to take into account that there were deleted features. So, in the case where a plant was misidentified say, or a species de-listed, and that feature was removed from our database, it didn’t seem to be reflected via the update existing features process but I could have missed something. Or maybe that’s where overwrite is more appropriate.

            Thanks for your time! If anyone reading sees potential for efficiencies in my workflow I would love to hear your ideas!

            How can I share this with support as an enhancement request?

  4. heisenbarth says:

    Is replacement of features no longer an option?

  5. derek.wicks_onegis says:

    Are there any plans to add support for appending related attachment tables?