Abstract
Street view imagery is a valuable resource for understanding the physical environment and public health. Open-access street view imagery platforms offer street view imagery along with metadata, such as geographic coordinates and capture details (e.g., the times-tamp of when the photo was taken and camera product type). Meta-data improves the utility of street view imagery for various location-based tasks, including urban studies and geospatial search. Recent research has further improved street view imagery by incorporating additional metadata, including human perceptions, weather conditions, and seasonal details. However, connecting street view imagery with Points of Interest (POI) data, including their names and amenity types, remains challenging. In this paper, we propose MulMapper, an automated system that enhances street view imagery metadata by extracting multilingual text, identifying text alignment, and matching POI attributes to the corresponding entity in the image. We develop two modules on top of one of the existing text spotter models: (1) a ‘text alignment detector’ to capture text alignment types and (2) a ‘character-wise text classification loss’ to overcome long-tail recognition issues, which result from imbalanced data distribution across diverse character sets. The proposed method greatly enhances the accuracy of matching between POIs and street view imagery while also enabling more semantically rich location searches within the images.
Original language | English (US) |
---|---|
Title of host publication | GeoSearch 2024 - Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Searching and Mining Large Collections of Geospatial Data |
Editors | Hao Li, Abhishek Potnis, Wenwen Li, Dalton Lunga, Martin Werner, Andreas Zufle |
Publisher | Association for Computing Machinery, Inc |
Pages | 29-35 |
Number of pages | 7 |
ISBN (Electronic) | 9798400711480 |
State | Published - Oct 29 2024 |
Event | 3rd ACM SIGSPATIAL International Workshop on Searching and Mining Large Collections of Geospatial Data, GeoSearch 2024 - Atlanta, United States Duration: Oct 29 2024 → … |
Publication series
Name | GeoSearch 2024 - Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Searching and Mining Large Collections of Geospatial Data |
---|
Conference
Conference | 3rd ACM SIGSPATIAL International Workshop on Searching and Mining Large Collections of Geospatial Data, GeoSearch 2024 |
---|---|
Country/Territory | United States |
City | Atlanta |
Period | 10/29/24 → … |
Bibliographical note
Publisher Copyright:© 2024 Copyright held by the owner/author(s).