Python scripting with 64-bit processing

Last week 64-bit Background Geoprocessing was made available for download. We’ve had a few questions from keen Python scripters who want to “get out of the application” and use their big data crunching scripts in 64-bit. If you’re one of those keen scripters, all you need to do is make sure you’re executing against 64-bit Python; no other special commands or tricks needed.

64-bit Background Geoprocessing installs a 64-bit version of Python 2.7.2, NumPy and Matplotlib if they are not already installed. By default they will be installed to C:\Python27\ArcGISx6410.1 (note the x64 in the path), an additional install from the 32-bit Python (C:\Python27\ArcGIS10.1) that Desktop and Engine install and use.

Note: If you have ArcGIS for Server 10.1 installed, 64-bit Background Processing will share Server’s 64-bit Python installation.

Now typically the last version of Python installed will be the one you execute against when double clicking a script from Windows Explorer. If you last installed 64-bit Background Processing, this means you’ll probably be executing against 64-bit Python. You can always change which program executes when double clicking a file in Windows.

If you need to ensure you’re calling the 64-bit version of Python (to make use of 64-bit processing) you can qualify which version of Python you execute against. Be sure to read the 64-bit help topic that applies to both scripting and ArcMap to learn about supported tools and data types.

The following script, “Intersect_Dissolve.py” is run in the 64-bit processing space by qualifying which Python installation to execute against at command line (c:\Python27\ArcGISx6410.1\python.exe).

The above text was printed running a script with the following python code

import arcpy
import sys

print "Running against: {}".format(sys.version)
print "Intersecting parcels..."
arcpy.Intersect_analysis(["C:/Demos/Parcels/data.gdb/parcels",
                         "C:/Demos/Parcels/data.gdb/landRecords"],
                         "C:/Demos/Parcels/scratch/out.gdb/int_out",
                         "ALL", "", "INPUT")
#...continued script...

3rd party Python modules

If you’ve written a Python script tool which makes use of a 3rd party module (one that is not included in the core Python installation, NumPy or Matplotlib) you will need to download and install the 64-bit version. Whether you are inside Desktop, or executing your script at command prompt, you need to ensure the proper module is installed and ready to go. For example, if your script tool requires the popular library, SciPy, you’ve probably already downloaded and installed the 32-bit version. You will have to download and install the 64-bit version of SciPy to successfully execute your tool when using 64-bit Background inside Desktop or when using 64-bit Python scripting. If you’re distributing your tool to a wide audience and are not confident your users will have the 64-bit module, you can disable background processing by checking the Always run in foreground option on the script tool’s general property page to force your tool to run against 32-bit Python.

Happy 64-bit scripting.

This entry was posted in Analysis & Geoprocessing, Python and tagged , , , . Bookmark the permalink.

Leave a Reply

7 Comments

  1. curtvprice says:

    Thanks Kevin, this is a useful post!

    I have found the assoc and ftype windows command line tools helpful in checking file associations. (For example, “ftype Python.File” shows the command executed when a .py file is double-clicked in windows or named in a command line or script.)

  2. csny490 says:

    Wow – this is very exciting. Questions:
    1. With all this RAM now avalable, does that mean that the “LargeOverlay” tiling schemes (like the one used in Union, Clip, etc.) will no longer tile up the data?
    2. The in_memory workspace size (which I now see suports rasters – way to go you guys!) is also only limited by the system RAM?

    • KenH says:

      1. Adaptive tiling will still take place if your data size and complexity requires it. However, there will be more cases where tiling is greatly reduced or eliminated (eliminated meaning all the data is processed in tile level 1 – one tile containing all the data).
      2. My understanding is that in_memory workspace size is only limited by the amount of memory available on the machine it is created on. I’m sure you are away, but beware of overloading your system memory with a huge in_memory workspace. You can easily negate the advantages in_memory may provide. http://resources.arcgis.com/en/help/main/10.1/index.html#//002w0000005s000000

      Ken

  3. petersm6449 says:

    Build numbers (see http://support.esri.com/en/knowledgebase/techarticles/detail/30104)
    Build 3143 = ArcGIS for Desktop 10.1 Service Pack 1
    Build 3035 = ArcGIS for Desktop 10.1 final

    Running in 32-bit Python, arcpy.GetInstallInfo() reports
    ‘BuildNumber’: u’3143′
    ‘SPNumber’: u’1′
    ‘SPBuild’: u’10.1.1.3143′

    Under 64-bit Python, arcpy.GetInstallInfo() reports
    ‘BuildNumber’: u’3143′
    ‘SPNumber’: u’N/A’
    ‘SPBuild’: u’N/A’

    Is this correct? Is 64-bit Background Geoprocessing simultaneously build #3143 and not Service Pack 1?

  4. clj2289 says:

    Why does the interpreter output this message when it is invoked claiming that its on win32?

    Python 2.7.2 (default, Jun 12 2011, 14:24:46) [MSC v.1500 64 bit (AMD64)] on win32

  5. elvisds8 says:

    Hello!

    I’m having trouble in trying to execute the Python 2.7.2 64-bit inside the Arcmap.
    When I open the Python command window inside Arcmap and I type sys.version I get the following output:
    ’2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]‘
    And that is the Python 32-bit version.
    How can I run the Python 64-bit version whose path is C:\Python27\ArcGISx6410.1\python.exe, inside ArcMap?????
    I need to execute a script inside Arcmap but it only works for Python 64-bit version because it requieres too much memory
    and I get a “memory error” in Python 32-bit.

    Regards.
    Elvis Durán