At the 2014 Esri User Conference, the Big Data team gave several presentations, including two technical workshops entitled: ‘Big Data and Analytics: The Fundamentals’ and ‘Big Data and Analytics with ArcGIS’. We presented our open source GIS Tools for Hadoop (shared on GitHub), as well as some research that we’re currently pursuing (exciting things to come!). We gave demos using both our open source tools as well as the prototype tools being currently researched.
For the demos (source data consisted of > 170 million data points that represent all the taxi cab trips in New York City in 2013), we ran all of our analytics on a Hadoop cluster back in Redlands. A twenty node cluster may seem like a big investment (and it can be); but, it doesn’t have to be. Enter the DREDD cluster… Continue reading
The Big Data development team at Esri is excited to announce a major performance speedup in ST_Geometry for Hive, which is part of Esri’s open-source Spatial Framework for Hadoop. The amount of performance gain depends on the type of spatial query run and on the size of the table in Hive. The biggest gain comes with relational operations such as ST_Contains and ST_Overlaps. In general, the performance gain will be greater with larger tables — exactly where it helps the most.
We are pleased to announce that the ST_Geometry aggregate functions are now available for Hive, in the Spatial Framework for Hadoop. The aggregate functions can be used to perform a convex-hull, intersection, or union operation on geometries from multiple records of a dataset.
An interesting task in highway management is to study potential impact of driver carpooling, based on an analysis of automatically collected automobile GPS position data. To identify potential enhancements to carpool participation, we set out to study places that have the highest numbers of trips with similar origin and destination locations. The source data for this experiment consists of nearly 40 million vehicle position records assembled from a single day of GPS-collected vehicle positions. The position data consists of longitude and latitude along with date, time, and speed. We used the Hadoop MapReduce framework for distributed parallel computation, considering its capability for analyzing larger data sets. Continue reading
The 2013 Esri International User Conference starts next week down at the San Diego Convention Center. There will be 15,000 attendees eager to learn and to see our path towards the future of GIS. Hopefully you’re one of them!
The Geodatabase Team will be down there giving technical sessions and demo theater presentations. We’ll be available to answer your questions and discuss what projects you’re working on too, so come visit us in the convention’s showcase area.
The project contains an open source framework and API that enables big data developers to author custom spatial applications for Hadoop. Continue reading
The Geodatabase Team is rolling down to Palm Springs next week for the 2013 Developer Summit. Swing by our section of the showcase area to meet the team’s developers, ask questions, and talk about your projects. Esri Showcase hours: Monday … Continue reading
By Damien Demaj, Cartographer The statistical component of sport has always provided a fascinating way to analyze performance and success. This might simply be the final score, but for some sports, such as football, baseball, cricket, golf and tennis, meaningful … Continue reading
In an ongoing effort to improve the quality of ArcGIS, I’m happy to announce the release of ArcGIS 10.0 Service Pack 5
This Service Pack includes many fixes since the 10.0 release. Here is list of issues fixed in SP5. If you have any feedback or questions regarding this service pack, please post in the ArcGIS Resource Center Forums or contact Esri Support.
If you have additional enhancements or ideas that you would like to see included in future service packs or releases, please post them on the ArcGIS Ideas site.
A key focus of ArcGIS 10.1 is to simplify and streamline how you work with databases and geodatabases. We have listened to enhancement requests from you and developed easy-to-use interfaces for routine administration tasks such as managing privileges.
If you are the owner of a dataset, you can control which users can view or modify data in a database or geodatabase with privileges on datasets. Privileges can be set by user or by roles. Roles are a handy way to group users who have common workflows and responsibilities and should share a common set of privileges.