Distributing deep neural networks with containerized partitions at the edge

Li Zhou, Hao Wen, Radu Teodorescu, David H.C. Du

Research output: Contribution to conferencePaperpeer-review

9 Scopus citations

Abstract

Deploying machine learning on edge devices is becoming increasingly important, driven by new applications such as smart homes, smart cities, and autonomous vehicles. Unfortunately, it is challenging to deploy deep neural networks (DNNs) on resource-constrained devices. These workloads are computationally intensive and often require cloud-like resources. Prior solutions attempted to address these challenges by either sacrificing accuracy or by relying on cloud resources for assistance. In this paper, we propose a containerized partition-based runtime adaptive convolutional neural network (CNN) acceleration framework for Internet of Things (IoT) environments. The framework leverages spatial partitioning techniques through convolution layer fusion to dynamically select the optimal partition according to the availability of computational resources and network conditions. By containerizing each partition, we simplify the model update and deployment with Docker and Kubernetes to efficiently handle runtime resource management and scheduling of containers.

Original languageEnglish (US)
StatePublished - 2019
Event2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2019, co-located with USENIX ATC 2019 - Renton, United States
Duration: Jul 9 2019 → …

Conference

Conference2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2019, co-located with USENIX ATC 2019
CountryUnited States
CityRenton
Period7/9/19 → …

Bibliographical note

Funding Information:
The authors would like to thank the anonymous reviewers for their feedback. We also extend special thanks to Irfan Ahmad for suggestions on the camera ready. This work was supported in part by NSF Award 1439622, 1812537 and NSF XPS Award 60053525.

Publisher Copyright:
© 2019 USENIX Association. All rights reserved.

Fingerprint Dive into the research topics of 'Distributing deep neural networks with containerized partitions at the edge'. Together they form a unique fingerprint.

Cite this