LandQv2: A MapReduce-based system for processing arable land quality big data

Xiaochuang Yao, Mohamed F. Mokbel, Sijing Ye, Guoqing Li, Louai Alarabi, Ahmed Eldawy, Zuliang Zhao, Long Zhao, Dehai Zhu

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Arable land quality (ALQ) data are a foundational resource for national food security. With the rapid development of spatial information technologies, the annual acquisition and update of ALQ data covering the country have become more accurate and faster. ALQ data are mainly vector-based spatial big data in the ESRI (Environmental Systems Research Institute) shapefile format. Although the shapefile is the most common GIS vector data format, unfortunately, the usage of ALQ data is very constrained due to its massive size and the limited capabilities of traditional applications. To tackle the above issues, this paper introduces LandQv2, which is a MapReduce-based parallel processing system for ALQ big data. The core content of LandQv2 is composed of four key technologies including data preprocessing, the distributed R-tree index, the spatial range query, and the map tile pyramid model-based visualization. According to the functions in LandQv2, firstly, ALQ big data are transformed by a MapReduce-based parallel algorithm from the ESRI Shapefile format to the GeoCSV file format in HDFS (Hadoop Distributed File System), and then, the spatial coding-based partition and R-tree index are executed for the spatial range query operation. In addition, the visualization of ALQ big data with a GIS (Geographic Information System) web API (Application Programming Interface) uses the MapReduce program to generate a single image or pyramid tiles for big data display. Finally, a set of experiments running on a live system deployed on a cluster of machines shows the efficiency and scalability of the proposed system. All of these functions supported by LandQv2 are integrated into SpatialHadoop, and it is also able to efficiently support any other distributed spatial big data systems.

Original languageEnglish (US)
Article number271
JournalISPRS International Journal of Geo-Information
Volume7
Issue number7
DOIs
StatePublished - Jul 2018

Bibliographical note

Funding Information:
Funding: This work was sponsored by National Key Research and Development Program of China from MOST (2016YFB0501503).

Funding Information:
This work was sponsored by National Key Research and Development Program of China from MOST (2016YFB0501503). We would like to thank the anonymous reviewers and editors for commenting on this paper.

Keywords

  • Arable land quality (ALQ)
  • GIS
  • MapReduce
  • Parallel processing
  • Spatial big data

Fingerprint Dive into the research topics of 'LandQ<sup>v2</sup>: A MapReduce-based system for processing arable land quality big data'. Together they form a unique fingerprint.

Cite this