Table of Contents
Data Transport in High-Performance Networks
Most data on the Internet is transferred with the TCP protocol (Transmission Control Protocol). It is known, that TCP does not work efficiently in networks with high bandwidths or delays and therefore cannot use the available bandwidth effectively. In the scope of the bwNET100G+ project, we investigate how the performance of TCP can be improved in high speed networks with 100 Gbit/s. In order to achieve this, new possibilities for data transport are created, evaluated, improved, and compared with existing solutions. Possible solutions are reviewed and compared, so applications in BelWü are enabled to use high speed data transport.
Congestion Control in High-Performance Networks
A central part of TCP is congestion control. The main task of congestion control is to regulate the sending rate in the end systems and to protect the network from excessive load. Currently, the Internet is dominated by loss-based congestion control algorithms, i.e. Cubic TCP. Big disadvantages of these established congestion controls are on the one hand the insufficient ability to scale at high data rates and on the other hand the high delays experienced by the end users. However, high-quality video conference and other demanding applications require both abilities, so we need new approaches for congestion control.
During the bwNET100G+ project, a new congestion control named TCP LoLa was developed, which was designed especially for the use in high performance networks. The main goals of TCP LoLa to achieve a high sending rate and a low latency. Furthermore, TCP LoLa tries to divide the available bandwidth fairly among competing flows. With the novel „Fair Flow Balancing“ mechanism, the senders can estimate their share of the bandwidth und adjust their sending rate step-by-step, so the network eventually converges to a fair state. The evaluation of TCP LoLa was performed on the NeIF infrastructure of the bwNET100G+ project.
TCP LoLa was published on the „LCN2017“ conference and presented on a meeting of the Internet-standardizing IETF and at a meeting of the ICCRG (Internet Congestion Control Research Group). The code is publicly available under a free software license.
With programmable data planes, we can expect, that the intermediate systems in a network will get more flexibility and more abilities. This offers a potential for a tighter collaboration between end systems and intermediate systems for congestion control. To study the chances of such a collaboration, TCP LoLa was extended to use explicit congestion feedback from the network. In our case, TCP LoLa requests the state of the intermediate systems, especially the length of the queues on the network path. With the help of this information, TCP LoLa is able to construct are more detailed model of the network, which would not be achievable with classic indicators, like RTT or throughput rates. With this ability, we could improve the performance of TCP LoLa in multi-bottleneck scenarios.
In order to accomplish low delays, we can use active queue management (AQM) in the intermediate systems (routers/switches), in addition to congestion control. With AQM, the filling level of the buffers is controlled actively, so the length of a standing queue is restricted by discarding packets or the setting of congestion notification flags. A disadvantage of AQMs is, that with traditional congestion controls, like Reno and Cubic TCP, only low data rates can be achieved if only a few flows participate during a congestion situation. In order to achieve an easy configuration for a trade-off between the allowed delay and the planed utilization, we developed „AQM steering“. AQM steering is a mechanism that adjusts the parameters of an AQM algorithm in relation to the desired trade-offs. AQM steering was published on the IFIP Networking Conference 2018 and presented on the 102nd IETF meeting in July 2018 of the ICCRG.
With „BBR“, Google published an congestion control in 2016, which also tries to achieve a high throughput rate and a low delay, which makes it relevant for the context of the bwNET100G+ project. Because of this, the performance and characteristics of BBR were researched and experimentally and conceptually evaluated. The results were published on the ICNP2017 conference. We showed, that BBR has conceptual weaknesses in its current version. This is illustrated by a high queueing delay, extremely high packet losses in certain scenarios, low fairness between BBR flows, and high unfairness when used with today's standard congestion controls. The latter is to be judged critically, because Google's BBR is already used on the public Internet. The results of the investigation were presented in two IETF meetings in the scope of the ICCRG and contributed an essential contribution to the evaluation of BBR. Currently, Google is working by their own statement on the further development of BBR, which tries to address the presented problems.
TCP at 100 Gbit/s
Another research question of the project is, whether and how transfer rates of 100 Gbit/s are achievable with a sole sender and/or receiver. To reach this goal, several tuning options were investigated. We found, that the following parameters play an important role:
- the number of senders and receivers
- the size of the MTU
- the choice of the right CPU socket for the interrupts and the applications
- the distribution of the interrupts on the different processor cores
Despite the intensive tuning, packet losses in the receiving end system are inevitable. These is a special challenge for congestion control, because common congestion control algorithms use packet loss as a congestion indicator. Novel congestion controls like TCP LoLa and BRR avoid this problem, because they do not react on packet loss. The results of the investigation will be published soon on the IEEE conference LCN2019.