Meltdown and Spectre Processor Vulnerabilities

Updated 1/19/18: While this vulnerability is independent of our application code, some of the patches for the vulnerability on Windows 2008R2 and Windows 7 systems result in the ArcGIS Server Geoprocessing service crashing (Error code #8273 will be generated in the log).

We strongly recommend customers use OS versions covered under mainstream support – This type of support from Microsoft ended a little over three years ago for both Windows 2008R2 and Windows 7.  The overall security risks of using these older OS versions are significantly higher than the moderate risk security issue the patch for these particular vulnerabilities addresses.  While these vulnerabilities have received significant media attention it is important to realize that the severity rating of them is only medium, with a CVSS score of 4.4 out of 10.

Therefore, if your organization is unable to move to an OS version under mainstream support, we recommend waiting on applying MS patches KB4056894, KB4056568, and KB4056897 for only the above specific OS versions until the issue is validated further and patch updates released (moving to a mainstream supported OS version is best as noted above).

If you have already installed one of the OS patches, you can uninstall it and ArcGIS Server will continue working correctly.  Customers with enterprise server machines will likely already have access control mechanisms and policies in place preventing all but a small subset of admins from locally accessing these machines. These controls help to reduce the relative risk associated with these flaws until OS vendors and others provide more stable patches for these security issues.

For ArcGIS Online, our cloud infrastructure providers have already patched their services and hosts for these vulnerabilities. What remains are some low risk issues that Esri will patch during our next release.

As additional patches are released which optimize and reduce performance impacts related to this issue, please reach out to your vendor (OS, browser, and database) for additional guidance – We will also continue updating this blog as we confirm more information.

- Esri’s Security Standards & Architecture Team

References:

This entry was posted in Security and tagged , . Bookmark the permalink.

Leave a Reply

39 Comments

  1. randallwilliams says:

    We are aware of issues related to geoprocessing services failing on Windows 2008r2 and Windows 7 associated with Microsoft Security Rollup kb4056894, and are investigating.

  2. johnmdye says:

    Do we know if the updates impact all geoprocessing services and tools or are there specific ones? Is there an error code?

  3. randallwilliams says:

    Hi John,

    The issue affects all GP Services I’ve tested.

    The error code # is 8273, the error reads like (for example):
    “Service containing process crashed for ‘System/PublishingTools.GPServer”.

    This error is generated when creating SOC processes.

    • johnmdye says:

      Thanks. We’ll be on the look out for these errors. Is the current remediation to just remove or delay installation of kb4056894?

      • randallwilliams says:

        Yes. Microsoft actually pulled kb4056894 due to issues with machines with AMD processors, but I think that they’re still available for machines with Intel processors. Note that both Spectre and Meltdown don’t have a publicly known network attack vector.

        Customers with enterprise server machines will likely already have access control mechanisms and policies in place preventing all but a small subset of admins from locally accessing these machines. These controls help to reduce the relative risk associated with these flaws. We intend to continue to keep customers abreast of updates and provide notifications of additional Esri response as the situation evolves.

  4. shahnazp_ess says:

    I had a user who was also unable to start the ArcGIS Server Printing Tool on Windows 2008R2.

  5. david_e says:

    So might this also be an issue with the display qualities of tiffs added to mxds? I’m getting a message I’ve never seen before when working with old formerly working images and wondering if the same issue is keeping me from visualizing similarly new images I’ve added even though they appear exactly the same in characteristics as the old ones. The message I’m now getting when touching old, working images states

    “Histogram does not exist. Click yes to estimate the histogram on resampled pixels or Click no to exit the dialog and compute histogram using Calculate Statistics Geoprocessing tool before display”.”
    I’ve used the tool on a new image as attest and it seems to not have teken effect or corrected the issue keeping me from visualizing all bands in the image.

    • randallwilliams says:

      I’m unsure about this one, David. if you’re running Windows 7 or Windows 2008 r2, can you check your list of recent updates and see if the Microsoft Patch 2018-01 Security Monthly Quality Rollup for Windows Server 2008 R2 for X64based Systems (KB4056894) is installed. If so, and if the patch time matches with the time you started seeing this issue, there may be a correlation.

      It would likely be worth contacting Esri Support Services. If this is related to these patches, they’ll want to build up a repro case in house and log it accordingly.

      • david_e says:

        I am, and we did. Tomorrow morning our IT staff is going to roll back the update for me and I’ll be able to test whether or not that’s the issue here once that’s done. I’ll follow up here. Thanks

        • randallwilliams says:

          I asked a Esri support to validate this issue. They weren’t able to reproduce. I’d recommend following up with Support Services. At this point, it doesn’t look related to the Meltdown patches for Windows 7 and Windows 2008r2.

          • david_e says:

            This morning we rolled back the windows patch and my tiffs behaved normally again. I’m not going to say that for sure this was the problem because I didn’t test anything else, just that removing it allowed me to work as expected with 4 new to me tiff files, and it eliminated the new popup requests for histograms/statistics from existing tiff files incorporated into mxds which never happened before the patch.

  6. tzschieter says:

    I can confirm problems on Windows Server 2008 R2 with the windows update.
    We were not able to publish or publish services and had similar error messages like randallwilliams.
    After we uninstalled the kb4056894 update, the publishing was possible again.

  7. randallwilliams says:

    Note that Esri Support Services are tracking this issue as Esri support services is tracking this as:

    [#BUG-000110662 Geoprocessing service instances crash after installing windows patches on Microsoft Windows Server 2008 R2 or Windows 7. ]

    • bfausel says:

      We are also experiencing the publish / print service problems on Windows Server 2008 R2 + ArcGIS Server 2008 R2 + kb4056894. I’m waiting to see if our IT will allow us to roll back the kb. Randall: do you recommend we send crash logs to Esri support, or just wait for updates?

  8. cenobite says:

    That was it for me. Uninstalled KB4056894 and I could publish again. Windows 2008r2, ArcGIS Server 10.4.1. Will wait for Esri fix before re-applying.

  9. grant_mullins says:

    Seeing the same issues this morning after a roll out of patches by IT last night. ArcSOC failures and GP Services that won’t start up. From what I can tell, it looks like our IT last patched it with KB4054955.

  10. christopherquick says:

    Are there performance hits to ArcGIS Desktop on Windows 7 or Windows 10?

  11. swiesmann says:

    Had same issue as randallwilliams with Win2008R2 / AGS1022 / KB4056894. Rollback of KB4056894 solved the problem.
    Different issue with Win2016 Standard / AGS1051 and corresponding KB4056890: publishing still possible, but after the update the HTTPS sitebindings to the SSL certificate got lost. As a result, server was not accessible via https anymore, no problem via http. Re-assigning the (same) SSL certificate in the sitebindings solved the problem.

  12. t523396 says:

    We issue the same problems. We have the issues on 4 servers. One server are patched up with KB4056894 and 3 server have patch KB4056897, I wonder if KB4056897 also give the same problem as KB4056894?

    • randallwilliams says:

      Yes. KB4056897 is a security rollup, which contains both KB4056568 and KB4056897. If both KB4056568 and KB4056897 are installed, this issue will occur.

  13. Jochen Harms says:

    Windows 2008R2 on Intel XEON / ArcServer 10.5.1 —- Some services didn´t start, especially publishing service. We removed Patch kb4056894, restartet the server. There are still application errors at the start-up of arcSOC.exe and the publishing service wasn´t running at first. I startet the Service with the ArcGIS Server Manager and it worked.

  14. codinggisuser says:

    Does any new information about this issues exist?

    I haven’t been able to find the mentioned ticket [#BUG-000110662 Geoprocessing service instances crash after installing windows patches on Microsoft Windows Server 2008 R2 or Windows 7] via the technical support search. Does it still exist?

    I also didn’t find any new blog entries related to this topic.

    We encountered the same problems with ArcGIS Server 10.2.2. on Windows 2008r2

    • bfausel says:

      Same config here: AGS 10.2.2 on Windows 2008 R2. We rolled back MS patches KB4056894, KB4056568, and KB4056897 and the ArcGIS Server publishing and printing services now work again. All we had to do was uninstall and then restart the server.

  15. randallwilliams says:

    Support services has released a parallel article regarding this issue:

    Bug: Geoprocessing service instances crash after installing Windows patches on Microsoft Windows Server 2008 R2 or Windows 7
    https://support.esri.com/en/Technical-Article/000017464

  16. randallwilliams says:

    From Product Management:

    We are actively preparing patches for all supported version of ArcGIS Server including: 10.3, 10.3.1, 10.4, 10.4.1, 10.5, 10.5.1, and 10.6.
     
    As an exception to our normal product life cycle policy we are also preparing patches for ArcGIS Server 10.2.1 and 10.2.2, because of the severity of the problem and the amount of customers affected on these releases. 

    ArcGIS 10.2.x is in mature support. All customers on 10.2.x are encouraged to upgrade to a fully supported version.
     
    No delivery date for the patches is available at this time. The development team is working on this with the highest urgency.

    • thorstenlowin says:

      It would be really beneficial for our customers to have your latest information about working on patches will be in the original post at the top (Esri’s Security Standards & Architecture Team)

    • kolster says:

      Can you estimate when the patches will be available? I have to tell that to our IT.
      How many weeks?
      We use ArcGIS 10.2.2

  17. kolster says:

    Windows 2008 R2
    ArcGIS Server 10.2.1
    Not only the Geoprossing Task did’t work, the PrintingTools and PublishingTools don’t work too.

  18. rspitzer says:

    A Customer of us is also affected:
    -> Win 2k8R2
    -> AGS 10.3.1

  19. ayamiranti says:

    Does 10.6 will fix this bug?

    • randallwilliams says:

      We are actively preparing patches for all supported version of ArcGIS Server including: 10.3, 10.3.1, 10.4, 10.4.1, 10.5, 10.5.1, and 10.6.

      As an exception to our normal product life cycle policy we are also preparing patches for ArcGIS Server 10.2.1 and 10.2.2, because of the severity of the problem and the amount of customers affected on these releases.

      Please note that ArcGIS 10.2.x is in mature support and would not normally receive further patches or updates at this point in the product life cycle. Please All customers on 10.2.x are urged to upgrade to a fully supported version.

  20. zhiyuli_byu says:

    Our ArcServer 10.4 on Windows Server 2008 R2 is affected. I tried uninstalling KB4056894, but after a system reboot it says “Failure configuring Windows updates. Reverting Changes…” . The patch was not able to get uninstalled and the Geoprocessing issue remains. I tried several times overnight but still no luck. Any suggestion would be highly appreciated. Thanks