Computer Science
Accurate hierarchical traffic measurement in datacenters through differentiated memory allocation
Document Type
Conference Paper
Abstract
A hierarchical flow model provides us the opportunity to monitor network traffic in datacenters, at once, from coarser (aggregate flows) to finer granularity (individual flows). Per-flow traffic measurement based on an idea of counter-sharing is memory-efficient by recording the sizes of all flows in a shared array of physical counters. Through this approach, flows at the same level record their sizes in uniformly sized virtual counter arrays constructed from the shared memory. However, flows at the same level have vastly different sizes. The virtual counter arrays for large flows must be big enough to uniformly distribute errors it contains. However, when large virtual counter arrays are used to record small flow sizes, the arrays accumulate more errors than the recorded flows' sizes, thereby affecting the accuracy of the estimations. The paper introduces the concept of differentiated virtual counter arrays, where the number of virtual counters allocated to each flow is based on its historical size. We derive a mathematical formula for flow-size estimation when recording flow sizes with flexible number of virtual counters and develop a heuristic algorithm to calculate how many virtual counters are assigned to each flow. Through experiments, we demonstrate that our solution is more accurate by up to 75% for aggregate flows and by up to 100% for small base flows compared with the prior art.
Publication Title
Proceedings - 2020 6th International Conference on Big Data Computing and Communications, BigCom 2020
Publication Date
2020
First Page
122
Last Page
130
ISBN
9781728182759
DOI
10.1109/BigCom51056.2020.00025
Keywords
servers, size measurement, estimation, resource management, aggregates, monitoring, task analysis
Repository Citation
Odegbile, Olufemi; Chen, Shigang; Wang, Yuanda; and Melissourgos, Dimitrios, "Accurate hierarchical traffic measurement in datacenters through differentiated memory allocation" (2020). Computer Science. 178.
https://commons.clarku.edu/faculty_computer_sciences/178
APA Citation
Odegbile, O., Chen, S., Wang, Y., & Melissourgos, D. (2020, July). Accurate hierarchical traffic measurement in datacenters through differentiated memory allocation. In 2020 6th International Conference on Big Data Computing and Communications (BIGCOM) (pp. 122-130). IEEE.