Residential Collegefalse
Status已發表Published
Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop
Rui Tang; Simon Fong
2018-03-10
Source PublicationFuture Generation Computer System
ISSN0167-739X
Volume86Pages:1395-1412
Abstract

Clustering algorithms are an important branch of data mining family which has been applied widely in loT applications such as finding similar sensing patterns, detecting outliers, and segmenting large behavioral groups in real-time. Traditional full batch k-means for clustering loT big data is confronted by large scaled storage and high computational complexity problems. In order to overcome the latency inherited from full batch k-means, two big data processing methods were often used: the first method is to use small batches as the input data to multiple computers for reducing the computation efforts. However, depending on the sensed data which may be heterogeneously fused from different sources in an IoT network, the size of each mini batch may vary in each iteration of clustering process. When these input data are subject to clustering their centers would shift drastically, which affects the final clustering results. The second method is parallel computing, it decreases the runtime while the overall computational effort remains the same. Furthermore, some centroid based clustering algorithm such as k-means converges easily into local optima. In light of this, in this paper, a new partitioned clustering method that is optimized by metaheuristic is proposed for IoT big data environment. The method has three main activities: Firstly, a sample of the dataset is partitioned into mini batches. It is followed by adjusting the centroids of the mini batches of data. The third step is collating the mini batches to form clusters, so the quality of the clusters would be maximized. How the positions of the centroids could be optimally attuned at the mini batches are governed by a metaheuristic called Dynamic Group Optimization. The data are processed in parallel in Hadoop. Extensive experiments are conducted to investigate the performance. The results show that our proposed method is a promising tool for clustering fused loT data efficiently. 

KeywordMetaheuristic Partitioning Clustering Hadoop Iot Data Data Fusion
DOI10.1016/j.future.2018.03.006
URLView the original
Indexed BySCIE ; SSCI
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Theory & Methods
WOS IDWOS:000437555800120
PublisherELSEVIER, RADARWEG 29, 1043 NX AMSTERDAM, NETHERLANDS
The Source to ArticleWOS
Scopus ID2-s2.0-85045328626
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorSimon Fong
AffiliationDepartment of Computer and Information Science University of Macau, Taipa, Macau SAR
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Rui Tang,Simon Fong. Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop[J]. Future Generation Computer System, 2018, 86, 1395-1412.
APA Rui Tang., & Simon Fong (2018). Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop. Future Generation Computer System, 86, 1395-1412.
MLA Rui Tang,et al."Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop".Future Generation Computer System 86(2018):1395-1412.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Rui Tang]'s Articles
[Simon Fong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Rui Tang]'s Articles
[Simon Fong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Rui Tang]'s Articles
[Simon Fong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.