Towards a Microblogs Data Management System

Amr Magdy; Mohamed F Mokbel

doi:10.1109/MDM.2015.24

Towards a Microblogs Data Management System

Amr Magdy, Mohamed F Mokbel

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

11 Scopus citations

Abstract

This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

Original language	English (US)
Title of host publication	Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	271-278
Number of pages	8
ISBN (Electronic)	9781479999729
DOIs	https://doi.org/10.1109/MDM.2015.24
State	Published - Sep 11 2015
Event	16th IEEE International Conference on Mobile Data Management, MDM 2015 - Pittsburgh, United States Duration: Jun 15 2015 → Jun 18 2015

Publication series

Name	Proceedings - IEEE International Conference on Mobile Data Management
Volume	1
ISSN (Print)	1551-6245

Other

Other	16th IEEE International Conference on Mobile Data Management, MDM 2015
Country/Territory	United States
City	Pittsburgh
Period	6/15/15 → 6/18/15

Bibliographical note

Publisher Copyright:
© 2015 IEEE.

Keywords

Big Data
Data Management Systems
Data Stream
Indexing
Keyword
Microblogs
Query Processing
Spatial
Temporal
Top-k

Publisher link

10.1109/MDM.2015.24

Find It

Find It at U of M Libraries

Cite this

Magdy, A., & Mokbel, M. F. (2015). Towards a Microblogs Data Management System. In Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015 (pp. 271-278). Article 7264332 (Proceedings - IEEE International Conference on Mobile Data Management; Vol. 1). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2015.24

Towards a Microblogs Data Management System. / Magdy, Amr; Mokbel, Mohamed F.
Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 271-278 7264332 (Proceedings - IEEE International Conference on Mobile Data Management; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Magdy, A & Mokbel, MF 2015, Towards a Microblogs Data Management System. in Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015., 7264332, Proceedings - IEEE International Conference on Mobile Data Management, vol. 1, Institute of Electrical and Electronics Engineers Inc., pp. 271-278, 16th IEEE International Conference on Mobile Data Management, MDM 2015, Pittsburgh, United States, 6/15/15. https://doi.org/10.1109/MDM.2015.24

@inproceedings{e5537f5916574009bbbb3755b83cda2d,

title = "Towards a Microblogs Data Management System",

abstract = "This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.",

keywords = "Big Data, Data Management Systems, Data Stream, Indexing, Keyword, Microblogs, Query Processing, Spatial, Temporal, Top-k",

author = "Amr Magdy and Mokbel, {Mohamed F}",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 16th IEEE International Conference on Mobile Data Management, MDM 2015 ; Conference date: 15-06-2015 Through 18-06-2015",

year = "2015",

month = sep,

day = "11",

doi = "10.1109/MDM.2015.24",

language = "English (US)",

series = "Proceedings - IEEE International Conference on Mobile Data Management",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "271--278",

booktitle = "Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015",

}

TY - GEN

T1 - Towards a Microblogs Data Management System

AU - Magdy, Amr

AU - Mokbel, Mohamed F

PY - 2015/9/11

Y1 - 2015/9/11

N2 - This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

AB - This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

KW - Big Data

KW - Data Management Systems

KW - Data Stream

KW - Indexing

KW - Keyword

KW - Microblogs

KW - Query Processing

KW - Spatial

KW - Temporal

KW - Top-k

UR - http://www.scopus.com/inward/record.url?scp=84958169629&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958169629&partnerID=8YFLogxK

U2 - 10.1109/MDM.2015.24

DO - 10.1109/MDM.2015.24

M3 - Conference contribution

AN - SCOPUS:84958169629

T3 - Proceedings - IEEE International Conference on Mobile Data Management

SP - 271

EP - 278

BT - Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 16th IEEE International Conference on Mobile Data Management, MDM 2015

Y2 - 15 June 2015 through 18 June 2015

ER -

Towards a Microblogs Data Management System

Abstract

Publication series

Other

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this