Tag: Big Data
As a new member of the ArcGIS GeoAnalytics Server team, I have quickly learned about GeoAnalytics and what it has to offer. GeoAnalytics enables users to leverage distributed computation for big data analysis. If you feel as I did, these … Continue reading
The Create Space Time Cube tool included with GeoAnalytics Server summarizes a set of points into a netCDF file structure by aggregating them into space-time bins. In addition to summarizing the number of features, the tool can also calculate summary statistics for the attribute values of points within each space-time bin. Continue reading
When people think about managing data, I can’t imagine anything as intimidating to an IT person as imagery data. Why you may ask? Because the file size is large, very large, the formats can be unfamiliar, and after you’ve acquired … Continue reading
Two years ago the Big Data team released GIS Tools for Hadoop on GitHub. GIS Tools for Hadoop is an open source project that allows users to integrate Hadoop (a distributed big data platform) with big spatial data, complete distributed … Continue reading
The Big Data team is excited to offer a new tutorial on spatial aggregation (sometimes called spatial binning). Spatial aggregation is extremely useful in summarizing big data to gain a meaningful snapshot of patterns in your data. Spatial aggregation works … Continue reading
At the 2014 Esri User Conference, the Big Data team gave several presentations, including two technical workshops entitled: ‘Big Data and Analytics: The Fundamentals’ and ‘Big Data and Analytics with ArcGIS’. We presented our open source GIS Tools for Hadoop (shared on GitHub), as well as some research that we’re currently pursuing (exciting things to come!). We gave demos using both our open source tools as well as the prototype tools being currently researched.
For the demos (source data consisted of > 170 million data points that represent all the taxi cab trips in New York City in 2013), we ran all of our analytics on a Hadoop cluster back in Redlands. A twenty node cluster may seem like a big investment (and it can be); but, it doesn’t have to be. Enter the DREDD cluster… Continue reading
The Big Data development team at Esri is excited to announce a major performance speedup in ST_Geometry for Hive, which is part of Esri’s open-source Spatial Framework for Hadoop. The amount of performance gain depends on the type of spatial query run and on the size of the table in Hive. The biggest gain comes with relational operations such as ST_Contains and ST_Overlaps. In general, the performance gain will be greater with larger tables — exactly where it helps the most.
This post also appears in Esri Insider. Last update: April 1, 2014. With all the recent excitement and good hopes over the White House Climate Data Initiative, and the ongoing progress of the Group on Earth Observation System of Systems (GEOSS), … Continue reading
Posted on behalf of and authored by Dr. Peter Urich, Managing Director of CLIMsystems Ltd. Esri’s Silver Business Partner CLIMsystems Ltd., located in Hamilton, New Zealand, has been exhaustively researching and processing the latest ocean data from the Coupled Model … Continue reading