2021
|
Yao, Zhiyuan; Ding, Zihan; Clausen, Thomas Heide Reinforced Workload Distribution Fairness Inproceedings Forthcoming In: Machine Learning for Systems at 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Forthcoming. @inproceedings{yao2021reinforced,
title = {Reinforced Workload Distribution Fairness},
author = {Zhiyuan Yao and Zihan Ding and Thomas Heide Clausen},
url = {https://www.thomasclausen.net/wp-content/uploads/2021/11/2111.00008-1.pdf},
year = {2021},
date = {2021-12-01},
urldate = {2021-01-01},
booktitle = {Machine Learning for Systems at 35th Conference on Neural Information Processing Systems (NeurIPS 2021)},
abstract = {Network load balancers are central components in data centers, that distributes workloads across multiple servers and thereby contribute to offering scalable services. However, when load balancers operate in dynamic environments with limited monitoring of application server loads, they rely on heuristic algorithms that require manual configurations for fairness and performance. To alleviate that, this paper proposes a distributed asynchronous reinforcement learning mechanism to-with no active load balancer state monitoring and limited network observations-improve the fairness of the workload distribution achieved by a load balancer. The performance of proposed mechanism is evaluated and compared with stateof-the-art load balancing algorithms in a simulator, under configurations with progressively increasing complexities. Preliminary results show promise in RLbased load balancing algorithms, and identify additional challenges and future research directions, including reward function design and model scalability.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Network load balancers are central components in data centers, that distributes workloads across multiple servers and thereby contribute to offering scalable services. However, when load balancers operate in dynamic environments with limited monitoring of application server loads, they rely on heuristic algorithms that require manual configurations for fairness and performance. To alleviate that, this paper proposes a distributed asynchronous reinforcement learning mechanism to-with no active load balancer state monitoring and limited network observations-improve the fairness of the workload distribution achieved by a load balancer. The performance of proposed mechanism is evaluated and compared with stateof-the-art load balancing algorithms in a simulator, under configurations with progressively increasing complexities. Preliminary results show promise in RLbased load balancing algorithms, and identify additional challenges and future research directions, including reward function design and model scalability. |
Yao, Zhiyuan; Desmouceaux, Yoann; Townsley, Mark; Clausen, Thomas Heide Towards Intelligent Load Balancing in Data Centers Inproceedings Forthcoming In: Machine Learning for Systems at 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Dec 2021, Sydney, Australia, Forthcoming. @inproceedings{yao2021intelligent,
title = {Towards Intelligent Load Balancing in Data Centers},
author = {Zhiyuan Yao and Yoann Desmouceaux and Mark Townsley and Thomas Heide Clausen},
url = {https://www.thomasclausen.net/wp-content/uploads/2021/11/2110.15788.pdf},
year = {2021},
date = {2021-12-01},
urldate = {2021-12-01},
booktitle = {Machine Learning for Systems at 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Dec 2021, Sydney, Australia},
abstract = {Network load balancers are important components in data centers to provide scalable services. Workload distribution algorithms are based on heuristics, e.g., Equal-Cost Multi-Path (ECMP), Weighted-Cost Multi-Path (WCMP) or naive machine learning (ML) algorithms, e.g., ridge regression. Advanced ML-based approaches help achieve performance gain in different networking and system problems. However, it is challenging to apply ML algorithms on networking problems in real-life systems. It requires domain knowledge to collect features from low-latency, high-throughput, and scalable networking systems, which are dynamic and heterogenous. This paper proposes Aquarius to bridge the gap between ML and networking systems and demonstrates its usage in the context of network load balancers. This paper demonstrates its ability of conducting both offline data analysis and online model deployment in realistic systems. The results show that the ML model trained and deployed using Aquarius improves load balancing performance yet they also reveals more challenges to be resolved to apply ML for networking systems.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Network load balancers are important components in data centers to provide scalable services. Workload distribution algorithms are based on heuristics, e.g., Equal-Cost Multi-Path (ECMP), Weighted-Cost Multi-Path (WCMP) or naive machine learning (ML) algorithms, e.g., ridge regression. Advanced ML-based approaches help achieve performance gain in different networking and system problems. However, it is challenging to apply ML algorithms on networking problems in real-life systems. It requires domain knowledge to collect features from low-latency, high-throughput, and scalable networking systems, which are dynamic and heterogenous. This paper proposes Aquarius to bridge the gap between ML and networking systems and demonstrates its usage in the context of network load balancers. This paper demonstrates its ability of conducting both offline data analysis and online model deployment in realistic systems. The results show that the ML model trained and deployed using Aquarius improves load balancing performance yet they also reveals more challenges to be resolved to apply ML for networking systems. |
Rizzi, Carmine; Yao, Zhiyuan; Desmouceaux, Yoann; Townsley, Mark; Clausen, Thomas Heide Charon: Load-Aware Load-Balancing in P4 Inproceedings In: 1st Joint International Workshop on Network Programmability & Automation (NetPA) at 17th International Conference on Network and Service Management (CNSM 2021),, 2021. @inproceedings{rizzi2021charon,
title = {Charon: Load-Aware Load-Balancing in P4},
author = {Carmine Rizzi and Zhiyuan Yao and Yoann Desmouceaux and Mark Townsley and Thomas Heide Clausen},
url = {https://www.thomasclausen.net/wp-content/uploads/2021/11/2110.14389.pdf},
year = {2021},
date = {2021-10-01},
urldate = {2021-01-01},
booktitle = {1st Joint International Workshop on Network Programmability & Automation (NetPA) at 17th International Conference on Network and Service Management (CNSM 2021),},
abstract = {Load-Balancers play an important role in data centers as they distribute network flows across application servers and guarantee per-connection consistency. It is hard however to make fair load balancing decisions so that all resources are efficiently occupied yet not overloaded. Tracking connection states allows to infer server load states and make informed decisions, but at the cost of additional memory space consumption. This makes it hard to implement on programmable hardware, which has constrained memory but offers line-rate performance. This paper presents Charon, a stateless load-aware load balancer that has line-rate performance implemented in P4-NetFPGA. Charon passively collects load states from application servers and employs the power-of-2-choices scheme to make data-driven load balancing decisions and improve resource utilization. Perconnection consistency is preserved statelessly by encoding server ID in a covert channel. The prototype design and implementation details are described in this paper. Simulation results show performance gains in terms of load distribution fairness, quality of service, throughput and processing latency.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Load-Balancers play an important role in data centers as they distribute network flows across application servers and guarantee per-connection consistency. It is hard however to make fair load balancing decisions so that all resources are efficiently occupied yet not overloaded. Tracking connection states allows to infer server load states and make informed decisions, but at the cost of additional memory space consumption. This makes it hard to implement on programmable hardware, which has constrained memory but offers line-rate performance. This paper presents Charon, a stateless load-aware load balancer that has line-rate performance implemented in P4-NetFPGA. Charon passively collects load states from application servers and employs the power-of-2-choices scheme to make data-driven load balancing decisions and improve resource utilization. Perconnection consistency is preserved statelessly by encoding server ID in a covert channel. The prototype design and implementation details are described in this paper. Simulation results show performance gains in terms of load distribution fairness, quality of service, throughput and processing latency. |
Hawari, Mohammed; Clausen, Thomas OP4T: Bringing Advanced Network Packet Timestamping into the Field Inproceedings In: 2021 International Conference on Information Networking (ICOIN), pp. 137-142, 2021. @inproceedings{Hawari2021,
title = {OP4T: Bringing Advanced Network Packet Timestamping into the Field},
author = {Mohammed Hawari and Thomas Clausen},
url = {https://www.thomasclausen.net/wp-content/uploads/2021/03/paper.pdf},
doi = {10.1109/ICOIN50884.2021.9333927},
year = {2021},
date = {2021-01-13},
booktitle = {2021 International Conference on Information Networking (ICOIN)},
pages = {137-142},
abstract = {Because it is very bursty, the microsecond-scale temporal behaviour of network traffic in data-centres is chal- lenging to measure and understand. To bring observability into data-centre networks, this paper introduces the Open Platform for Programmable Precise Packet Timestamping (OP4T), a hardware architecture, targeting Field-Programmable Gateway Arrays (FPGAs), integrated into data-centre servers as a Smart Network Interface Card (SmartNIC), and flexible enough to enable advanced latency diagnosis.
In this paper, OP4T is specified, and an open-source im- plementation of that architecture is proposed, targeting the NetFPGA SUME prototyping board. By leveraging the P4 programming language, and partial reconfiguration, that open- source implementation is experimentally shown to enable in-band, precise packet timestamping, without sacrificing the achievable throughput. As an illustration, OP4T is shown to be usable to measure fine-grained properties of a software packet forwarder, e.g., packet batching.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Because it is very bursty, the microsecond-scale temporal behaviour of network traffic in data-centres is chal- lenging to measure and understand. To bring observability into data-centre networks, this paper introduces the Open Platform for Programmable Precise Packet Timestamping (OP4T), a hardware architecture, targeting Field-Programmable Gateway Arrays (FPGAs), integrated into data-centre servers as a Smart Network Interface Card (SmartNIC), and flexible enough to enable advanced latency diagnosis.
In this paper, OP4T is specified, and an open-source im- plementation of that architecture is proposed, targeting the NetFPGA SUME prototyping board. By leveraging the P4 programming language, and partial reconfiguration, that open- source implementation is experimentally shown to enable in-band, precise packet timestamping, without sacrificing the achievable throughput. As an illustration, OP4T is shown to be usable to measure fine-grained properties of a software packet forwarder, e.g., packet batching. |
2020
|
Hawari, Mohammed; Cordero, Juan Antonio; Clausen, Thomas High-Accuracy Packet Pacing on Commodity Servers for Constant-Rate Flows Journal Article In: IEEE/ACM Transactions on Networking, pp. 1-15, 2020, ISSN: 1558-2566. @article{9130915,
title = {High-Accuracy Packet Pacing on Commodity Servers for Constant-Rate Flows},
author = {Mohammed Hawari and Juan Antonio Cordero and Thomas Clausen},
url = {https://www.thomasclausen.net/wp-content/uploads/2020/08/2020-IEEE-TNET-High-Accuracy-Packet-Pacing-on-Commodity-Servers-for-Constant-Rate-Flows.pdf},
doi = {10.1109/TNET.2020.3001672},
issn = {1558-2566},
year = {2020},
date = {2020-01-01},
journal = {IEEE/ACM Transactions on Networking},
pages = {1-15},
abstract = {This addresses the problem of high-quality packet pacing for constant-rate packet consumption systems, with strict buffering limitations. A mostly-software pacing architecture is developed, which has minimal hardware requirements, satisfied by commodity servers - rendering the proposed solution easily deployable in existing (data-centre) infrastructures. Two algorithms (free-running and frequency-controlled pacing, for explicitly and implicitly indicated target rates, respectively) are specified, and formally analysed. The proposed solution, including both algorithms, is implemented, and is tested on real hardware and under real conditions. The performance of these implementations is experimentally evaluated and compared to existing mechanisms, available in general-purpose hardware. Results of both exhaustive experiments, and of an analytical modeling, indicate that the proposed approach is able to perform low-jitter packet pacing on commodity hardware, being thus suitable for constant rate transmission and consumption in media production scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This addresses the problem of high-quality packet pacing for constant-rate packet consumption systems, with strict buffering limitations. A mostly-software pacing architecture is developed, which has minimal hardware requirements, satisfied by commodity servers - rendering the proposed solution easily deployable in existing (data-centre) infrastructures. Two algorithms (free-running and frequency-controlled pacing, for explicitly and implicitly indicated target rates, respectively) are specified, and formally analysed. The proposed solution, including both algorithms, is implemented, and is tested on real hardware and under real conditions. The performance of these implementations is experimentally evaluated and compared to existing mechanisms, available in general-purpose hardware. Results of both exhaustive experiments, and of an analytical modeling, indicate that the proposed approach is able to perform low-jitter packet pacing on commodity hardware, being thus suitable for constant rate transmission and consumption in media production scenarios. |
2018
|
Desmouceaux, Yoann; Toubaline, Sonia; Clausen, Thomas Flow-Aware Workload Migration in Data Centers Journal Article In: Springer - Journal of Network and Systems Management (JONS), 2018. @article{Desmouceaux2018a,
title = {Flow-Aware Workload Migration in Data Centers},
author = {Yoann Desmouceaux and Sonia Toubaline and Thomas Clausen},
url = {https://link.springer.com/epdf/10.1007/s10922-018-9452-5?author_access_token=qm_40d91CsNLlZ_vZ0tZFPe4RwlQNchNByi7wbcMAY4xSrvbLplDMLQ3AN9vWEoUIxtZAIdnOGAzJH5W3YOrbGteOLvaEXsEE1xFv66lVxTKlL40BAS25fsaLf8w1RJAvY69owHWqhJkTmAZpvdCkQ%3D%3D
https://www.epizeuxis.net/wp-content/uploads/2018/03/jons-2018.pdf},
doi = {10.1007/s10922-018-9452-5},
year = {2018},
date = {2018-03-10},
journal = {Springer - Journal of Network and Systems Management (JONS)},
abstract = {In data centers, subject to workloads with heterogeneous (and sometimes short) lifetimes, workload migration is a way of attaining a more efficient utilization of the underlying physical machines.
To not introduce performance degradation, such workload migration must take into account not only machine resources, and per-task resource requirements, but also application dependencies in terms of network communication.
This articleformat presents a workload migration model capturing all of these constraints.
A linear programming framework is developed allowing accurate representation of per-task resources requirements and inter-task network demands. Using this, a multi-objective problem is formulated to compute a re-allocation of tasks that (i) maximizes the total inter-task throughput, while (ii) minimizing the cost incurred by migration and (iii) allocating the maximum number of new tasks.
A baseline algorithm, solving this multi-objective problem using the $epsilon$-constraint method is proposed, in order to generate the set of Pareto-optimal solutions. As this algorithm is compute-intensive for large topologies, a heuristic, which computes an approximation of the Pareto front, is then developed, and evaluated on different topologies and with different machine load factors. These evaluations show that the heuristic can provide close-to-optimal solutions, while reducing the solving time by one to two order of magnitudes.
},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In data centers, subject to workloads with heterogeneous (and sometimes short) lifetimes, workload migration is a way of attaining a more efficient utilization of the underlying physical machines.
To not introduce performance degradation, such workload migration must take into account not only machine resources, and per-task resource requirements, but also application dependencies in terms of network communication.
This articleformat presents a workload migration model capturing all of these constraints.
A linear programming framework is developed allowing accurate representation of per-task resources requirements and inter-task network demands. Using this, a multi-objective problem is formulated to compute a re-allocation of tasks that (i) maximizes the total inter-task throughput, while (ii) minimizing the cost incurred by migration and (iii) allocating the maximum number of new tasks.
A baseline algorithm, solving this multi-objective problem using the $epsilon$-constraint method is proposed, in order to generate the set of Pareto-optimal solutions. As this algorithm is compute-intensive for large topologies, a heuristic, which computes an approximation of the Pareto front, is then developed, and evaluated on different topologies and with different machine load factors. These evaluations show that the heuristic can provide close-to-optimal solutions, while reducing the solving time by one to two order of magnitudes.
|
Desmouceaux, Yoann; Pfister, Pierre; Tollet, Jérôme; Townsley, W. Mark; Clausen, Thomas 6LB: Scalable and Application-Aware Load Balancing with Segment Routing Journal Article In: IEEE/ACM Transactions on Networking, 26 (2), pp. 819-834, 2018, ISSN: 1063-6692. @article{Desmouceaux2018,
title = {6LB: Scalable and Application-Aware Load Balancing with Segment Routing},
author = {Yoann Desmouceaux and Pierre Pfister and Jérôme Tollet and W. Mark Townsley and Thomas Clausen},
url = {http://www.thomasclausen.net/wp-content/uploads/2018/02/2018-IEEE-Transactions-on-Networking-6LB-Scalable-and-Application-Aware-Load-Balancing-with-Segment-Routing.pdf},
doi = {10.1109/TNET.2018.2799242},
issn = {1063-6692},
year = {2018},
date = {2018-02-15},
journal = {IEEE/ACM Transactions on Networking},
volume = {26},
number = {2},
pages = {819-834},
abstract = {Network load-balancers generally either do not take application state into account, or do so at the cost of a central- ized monitoring system. This paper introduces a load-balancer running exclusively within the IP forwarding plane, i.e. in an application protocol agnostic fashion – yet which still provides application-awareness and makes real-time, decentralized deci- sions. To that end, IPv6 Segment Routing is used to direct data packets from a new flow through a chain of candidate servers, until one decides to accept the connection, based solely on its local state. This way, applications themselves naturally decide on how to fairly share incoming connections, while incurring minimal network overhead, and no out-of-band signaling. A consistent hashing algorithm, as well as an in-band stickiness protocol, allow for the proposed solution to be able to be reliably distributed across a large number of instances.
Performance evaluation by means of an analytical model and actual tests on different workloads (including a Wikipedia replay as a realistic workload) show significant performance benefits in terms of shorter response times, when compared to a traditional random load-balancer. In addition, this paper introduces and compares kernel bypass high-performance implementations of both 6LB and a state-of-the-art load-balancer, showing that the significant system-level benefits of 6LB are achievable with a negligible data-path CPU overhead.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Network load-balancers generally either do not take application state into account, or do so at the cost of a central- ized monitoring system. This paper introduces a load-balancer running exclusively within the IP forwarding plane, i.e. in an application protocol agnostic fashion – yet which still provides application-awareness and makes real-time, decentralized deci- sions. To that end, IPv6 Segment Routing is used to direct data packets from a new flow through a chain of candidate servers, until one decides to accept the connection, based solely on its local state. This way, applications themselves naturally decide on how to fairly share incoming connections, while incurring minimal network overhead, and no out-of-band signaling. A consistent hashing algorithm, as well as an in-band stickiness protocol, allow for the proposed solution to be able to be reliably distributed across a large number of instances.
Performance evaluation by means of an analytical model and actual tests on different workloads (including a Wikipedia replay as a realistic workload) show significant performance benefits in terms of shorter response times, when compared to a traditional random load-balancer. In addition, this paper introduces and compares kernel bypass high-performance implementations of both 6LB and a state-of-the-art load-balancer, showing that the significant system-level benefits of 6LB are achievable with a negligible data-path CPU overhead. |
2017
|
Desmouceaux, Yoann; Pfister, Pierre; Tollet, Jerome; Townsley, W. Mark; Clausen, Thomas SRLB: The Power of Choices in Load Balancing with Segment Routing Inproceedings In: In Proceedings of the 37th IEEE International Conference on Distributed Computing Systems (ICDCS), 2017. @inproceedings{Desmouceaux2017b,
title = {SRLB: The Power of Choices in Load Balancing with Segment Routing},
author = {Yoann Desmouceaux and Pierre Pfister and Jerome Tollet and W. Mark Townsley and Thomas Clausen},
url = {http://www.thomasclausen.net/wp-content/uploads/2017/05/camera-ready-ieeepdfexpress.pdf},
year = {2017},
date = {2017-06-05},
booktitle = {In Proceedings of the 37th IEEE International Conference on Distributed Computing Systems (ICDCS)},
abstract = {Network load-balancers generally either do not take application state into account, or do so at the cost of a central- ized monitoring system. This paper introduces a load-balancer running exclusively within the IP forwarding plane, i.e. in an application protocol agnostic fashion – yet which still provides application-awareness and makes real-time, decentralized deci- sions. To that end, IPv6 Segment Routing is used to direct data packets from a new flow through a chain of candidate servers, until one decides to accept the connection, based on its local state. This way, applications themselves naturally decide on how to share incoming connections, while incurring minimal network overhead, and no out-of-band signaling.
Tests on different workloads – including realistic workloads such as replaying actual Wikipedia access traffic towards a set of replica Wikipedia instances – show significant performance benefits, in terms of shorter response times, when compared to a traditional random load-balancer.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Network load-balancers generally either do not take application state into account, or do so at the cost of a central- ized monitoring system. This paper introduces a load-balancer running exclusively within the IP forwarding plane, i.e. in an application protocol agnostic fashion – yet which still provides application-awareness and makes real-time, decentralized deci- sions. To that end, IPv6 Segment Routing is used to direct data packets from a new flow through a chain of candidate servers, until one decides to accept the connection, based on its local state. This way, applications themselves naturally decide on how to share incoming connections, while incurring minimal network overhead, and no out-of-band signaling.
Tests on different workloads – including realistic workloads such as replaying actual Wikipedia access traffic towards a set of replica Wikipedia instances – show significant performance benefits, in terms of shorter response times, when compared to a traditional random load-balancer. |
