Computer Science

Virtual filter for non-duplicate sampling

Document Type

Conference Paper

Abstract

Sampling is key to handling mismatch between the line rate and the throughput of a network traffic measurement module. Flow-spread measurement requires non-duplicate sampling, which only samples the elements (carried in packet header or payload) in each flow when they appear for the first time and blocks them for subsequent appearances. The only prior work for non-duplicate sampling incurs considerable overhead, and has two practical limitations: It lacks a mechanism to set an appropriate sampling probability under dynamic traffic conditions, and it cannot efficiently handle multiple concurrent sampling tasks. This paper proposes a virtual filter design for non-duplicate sampling, which reduces the processing overhead by about half and reduces the memory overhead by an order of magnitude or more under some practical settings. It has a mechanism to automatically adapt its sampling probability to the traffic dynamics. It can be extended to solve a new problem called non-duplicate distribution sampling, which samples packets based on a probability distribution to support multiple concurrent measurement tasks.

Publication Title

Proceedings - International Conference on Network Protocols, ICNP

Publication Date

2021

Volume

2021-November

ISSN

1092-1648

ISBN

9781665441315

DOI

10.1109/ICNP52444.2021.9651974

Keywords

protocols, heuristic algorithms, memory management, telecommunication traffic, filtering algorithms, throughput, time measurement

APA Citation

Ma, C., Wang, H., Odegbile, O. O., & Chen, S. (2021, November). Virtual filter for non-duplicate sampling. In 2021 IEEE 29th International Conference on Network Protocols (ICNP) (pp. 1-11). IEEE.

Link to Full Text

Share

COinS