Collector – Disconnected editing and sync

With the 10.2.2 release of Collector for ArcGIS in March you will be able to use maps on your smartphone or tablet when disconnected from the network. We started blogging about this last month by discussing how you will be able to download maps to your device. In this article we want to discuss feature services, the flow of data, and synchronizing edits.

Feature Services and Disconnected Use

In the last article we briefly touched on the composition of a map and the fact that the GIS features you collect and update with Collector are managed using a Feature Service. Collector supports use of both Hosted Feature Services that are stored within a managed database inside of your ArcGIS organization as well as on-premises Feature Services that are stored locally within your enterprise geodatabase and published via ArcGIS Server.

Feature Services provide the capability to disconnect feature data and synchronize changes between the field and office as well. There are a number of things you need to do to prepare your data for disconnected use with feature services.

The workflow from preparing your data to synchronizing changes is best explained using the graphic below. It is important to note that you DO NOT need to publish your data to the cloud. As mentioned above you can use your own on-premises feature services and manage all of your content behind your firewall. Publishing directly against your enterprise geodatabase you will need to consider using either a VPN tunnel to access those services for download/sync or you can use the web adapter to provide access to remote clients.

Feature Services provide a new operation called Sync. If you want to work disconnected with Collector, you will need to enable the Sync operation on your feature services. Sync provides the ability to download a copy of data to your device as well as synchronize changes when connected. With ArcGIS Server, this is accomplished using a new SyncTools Geoprocessing Service. This service is installed with ArcGIS Server and must be up and running at all times. If you are using Hosted Feature Services within ArcGIS Online, this is managed for you.

Downloading a map to your device creates a copy of data for the area you have chosen to take offline. The copy itself is a replica that is created and managed by the Collector app. It is important that you do not erroneously delete replicas from your feature service – let the Collector handle them for you. Within the context of Collector, the replica is unique to the device you download from and all synchronization will occur from the device that created the replica. In a future release, Collector will support pre-caching and provisioning of replicas.

Map Schemas and Managing Change

If you have used the Collector in the past, or any Esri mobile product for that matter, you know that the user experience for collection and editing is bound to the map and it’s schema. Making maps for field collection, you also know that you will never get the map schema right the first time. Knowing this and knowing that you will have disconnected field workers – understanding/managing the impact of an evolving schema is important.

Let’s consider hosted feature services first. Hosted feature services are a physical copy of your data and solely managed through the service/service item itself. If you need to make a schema change (add a field or a domain value for example) then you will need to re-publish services again (overwrite your existing service) as you cannot edit the schema of a hosted feature service. If you have disconnected field clients and/or edits that are in that service already, then you will need to plan for this change as you will be deleting the service they created replicas from and you do NOT want to lose edits – so sync up and export changes first! Also if you have additional reference feature layers that you will be taking to the field, it is recommended that you publish them as a separate feature service – no point in having to republish them each time – the map you take to the field can support layers from multiple feature services.

Now let’s consider feature services published using ArcGIS Server. These services reference feature layers within your enterprise geodatabase. To change the schema, you can simply stop the service, modify fields/domains, etc and start the service again. But what happens to all the disconnected field clients? Their schema is now obsolete! Actually not to worry. The sync process will still synchronize with the service (so long as you did not delete the feature layer itself!). You can add a field, remove a field, add/remove a domain value if you like. The replica is founded on an old schema but both the replica and the service can manage the change. Remember that the disconnected client is ALWAYS working with the schema version from the point in time of “downloading the map”. To take advantage of the changes you make they will need to go into the manage experience on their device and remove features from the device. This will delete the replica and they can download a fresh schema for the area they are working in.

Regardless of Hosted or On-premises feature services, you will want to take extra care in managing changes across an evolving schema.

Local Edits and Synchronizing Changes

Once downloaded, maps on your device can be opened and you can edit existing GIS features or collect new features just as you can with maps you open from the Cloud. What is different is that a local map works in a fully disconnected mode – meaning that it will only draw content stored on the device, it will only show you popups from local feature data, and when you collect new features or edit existing ones that all of the edits are stored locally on the device itself.

Synchronization itself is a manual process. You will need to exit the map and tap the Sync button on the map card to sync changes when you have connectivity. It is important to note that the synchronization is bi-directional. You will receive all changes that others have made inside of the area you downloaded to your device when you sync.

Manage Maps On Device

You can manage the maps that are downloaded to your device within the Collector app. Local content is organized into both Maps and Basemaps from a management context and you can see this by tapping Manage from the Action button. From the Manage page you can view when maps were downloaded/sync’d, how much size they are taking on the device and remove them. Removing feature content from the device will delete local replicas and un-register any replicas it created on the feature service as well.

Basemaps are tiled map service layers, tend to be rather large, and take time to extract/download to your device. Collector lets you re-use basemaps that you download to your device and so you can manage them separate of the map itself.

As mentioned in the previous post, you can either extract basemaps referenced in the map you authored within your ArcGIS organization or you can side-load basemaps directly to the device. We will discuss this in another blog article.

Summary

A lot of content was covered in this article and there are some key takeaways that we hope you think about/plan for as disconnected editing comes to smartphone and tablet devices:

  • You can use on-premises or hosted feature services to manage the data that you collect/update in the field. Each have their benefits/challenges. If using on-premises feature services, make sure to prepare your data and services.
  • Consider the implications of disconnected clients when changing map schema’s so you keep everyone up to date.
  • You can manage maps on devices. Consider the use, re-use, and deployment of basemaps. They often constitute the largest volume of data you work with.
  • Plan your field deployments and updates carefully. Think about download once and sync many times as a key workflow.

The Collector team is working hard to finish these capabilities now and release in March. We look forward to your feedback!!

This entry was posted in Apps, ArcGIS Online, Mobile and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

15 Comments

  1. billfox says:

    Does this work with editing curves so they are not broken into tiny segments?

    • Jeff Shaner says:

      Unfortunately no. All of our runtime clients do not handle true curves. I should mention that when you update an existing feature we do ship back the entire feature. At this point in time we cannot distinguish between an attribute update and a shape update. So updating an attribute value for a feature with a true curve will densify the curve into straight line segments.

  2. timmerspk says:

    I hope it stays a fairly simple app to use. That has been it’s greatest feature for us so far. Our non-gis users have appreciated the simple and easy to learn interface.

  3. u0281276 says:

    Where’s the docs on the new sync operation on feature services?

  4. jacksoe says:

    Will it be possible to collect photos offline and sync those back too?

    • terrainpro says:

      +1 on this question and the need for collecting photos offline.

    • Jeff Shaner says:

      Absolutely. My apologies if that was not clear. Disconnected editing and the sync process will fully support features with photos/video. The way that Collector manages media is via attachments. If you have been using ArcPad in the past, this might be new to you. Attachments are a data type within the geodatabase that uses a 1:M relationship class to store and manage media. Using this approach all media you collect are synchronized using the feature service and you do not need to manage a separate folder and set of links for photos.

      Here is a link to the 10.2 Desktop doc for attachments:
      http://resources.arcgis.com/en/help/main/10.2/index.html#//01m90000000r000000

      Jeff

  5. ludo2020 says:

    Jeff, I know you already commented on relationship classes being on the roadmap for Collector and Explorer for ArcGIS. Will this relationship class functionality be available with the upcoming 10.2.2 release and will the functionality carry over to AGOL as well? Also, will the data be accessible in tabular view instead of just a map/geometry view in 10.2.2 in collector? Is the 10.2.2 release date March 4th along with AGOL?

  6. sercmeg says:

    good work with the offline editing capabilities!
    1) table editing – it would be extremely field-realistic/field-helpful to be able to view & edit a subgroup of features (e.g., all points within a block) via attribute table – it’s immensely useful to create new/edit features based on a map display but checking data entry in table format in the field after completing a particular survey area is ideal.
    2) offline editing feature limits – for our field projects, we need to have ~10,000 points accessible for reference and update. will there be editable feature limits in the March release?
    many thanks!

  7. gwmwatergis says:

    We have a big project coming up very soon that needs the offline capability. 80% of the project will happen in offline mode and it’s critical for me to supply contractors with devices to capture assets in the field. Can anybody please give me a release date in March ?
    The 4th has come and gone. I just want to know if I should start to look at other products in the market with this capability. I have been holding back for more than a month since the original release date was set for end of Feb, I am running out of time and need to make a decision.

    Kind Regards

    • tjohnston87 says:

      From here: http://blogs.esri.com/esri/arcgis/2014/03/05/whats-new-in-arcgis-online-march-2014/
      “Collector for ArcGIS now supports offline maps.”

      I have not seen where Server/Desktop/Mobile 10.2.2 versions have been released yet. I get the feeling that folks internal to ESRI don’t always know when new releases are released (i.e. ESRI staff may be using a final release version of something for a bit before it is public)? At any rate, I am excited to try out this new technology! We have been having a lot of trouble with ArcGIS for Windows Mobile and it would be refreshing to have an easily and reliable app like Collector to run on common, lower-cost devices. At UC2012, it was said that the offline functionality would be available in the November 2012 update…

  8. amarsden says:

    Am I missing something here – I’ve updated my Android version and cannot “Manage from the Action button” – as I have no Action button, or anything like it.

  9. rvburton says:

    I have just started working with the disconnected editing feature of Collector and have found that while I can edit existing data, I cannot add new items. There are a couple of threads in the Forum section on this issue. I also noticed that when I check the edit tracking fields for those items updated after a disconnected editing session, the Creator and Editor fields are null, but the CreationDate and EditDate fields contain the updated time. For online collection, all fields are completed as expected. I am using the Android 10.2.2 version of Collector.