Repository navigation
Wrong Traffic Shaping in Secondary NIC over L2 networks #10007
Description
ISSUE TYPE
- Bug Report
COMPONENT NAME
L2 Networks
CLOUDSTACK VERSION
4.19.1
CONFIGURATION
Advance networking, VLANs and VMware ESXi 8.0.0 hypervisor
SUMMARY
When the same VM is connected to two or more L2 networks, the secondary NIC gets a different bandwidth (vmware traffic shapping) than the bandwidth of the default NIC, even if all the L2 networks use the same network offer.
STEPS TO REPRODUCE
1) Have an L2 network offer configured with a network rate (Mb/s) different from the global setting “Vm network throttling rate”( vm.network.throttling.rate).
2) Create two or more L2 networks from the same network offer.
3) Connect the same VM to the two o more L2 networks.
EXPECTED RESULTS
All the NICs of the VM in question should have the same bandwidth allocated (VMware Traffic Shapping).
ACTUAL RESULTS
The default NIC takes a different bandwidth value (VMware Traffic Shapping) than the secondary NICs.
The default NIC gets a Traffic Shapping in VMware according to the value of the global setting vm.network.throttling.rate while the secondary NIC(s) takes the value of the network offer.
Activity
iishitahere commentedon Nov 28, 2024
Hi @luganofer , I’d like to work on this issue. From my understanding, the problem involves incorrect traffic shaping in secondary NICs over L2 networks. Could you provide more details on the expected traffic shaping behavior or any specific test cases to validate the fix? Once assigned, I’ll dive in and start working on this. Thank you!
luganofer commentedon Nov 28, 2024
Hi @iishitahere . Thank you for your comment. Yes, the use case is relatively simple.
A user creates two L2 type networks (from the same network offering for example) and the bandwidth that cloudstack applies to the default NIC (Traffic Shapping in VMware) differs from the one configured in the secondary NIC, when it should be the same bandwidth since both networks were created from the same network offering.
From what I could find out, CloudStack would NOT be using the value configured in the vm.network.throttling.rate parameter of the global settings for the secondary NICs as it does for the default NIC. Let me know if this helps in understanding the case and thanks for taking an interest in it.
iishitahere commentedon Nov 28, 2024
Hi @luganofer , thank you for the detailed explanation! I understand now that the issue arises because CloudStack does not apply the vm.network.throttling.rate parameter to secondary NICs, even though both NICs are created from the same network offering. The expected behavior is that the bandwidth should be consistent across all NICs.
I’ll investigate this further and look into how the vm.network.throttling.rate parameter is being applied. If there are any specific areas of the codebase or logs that you recommend focusing on, please let me know. I'll keep you updated on my progress. Thanks again for your guidance!
BryanMLima commentedon Nov 28, 2024
Hello, @luganofer and @iishitahere
From what I remember of the code, the throttling is applied differently if the NIC of the VM is the default one or not.
cloudstack/server/src/main/java/com/cloud/network/NetworkModelImpl.java
Line 1037 in db2e89a
For the default NIC, the network throttling is applied based on the compute offering network bandwidth of the VM. If there is no limit set in the compute offering, the value of the configuration
vm.network.throttling.ratewill be used.For additional NICs, the network throttling is applied based on the network offering bandwidth. If there is no limit set in the network offering, then the value of the configuration
network.throttling.ratewill be used.To be fair, this behavior is kind of confusing, and maybe a discussion should be done before any actual implementation.
What do you guys think? cc @DaanHoogland @JoaoJandre @GutoVeronezi @weizhouapache
iishitahere commentedon Nov 28, 2024
Hello @BryanMLima , @luganofer
Thank you for pointing out the behavior of network throttling for default and additional NICs. I agree that the current approach can be somewhat confusing. I’d like to explore this issue further to clarify and potentially improve the implementation.
Here’s how I plan to proceed:
Analyze the existing logic in NetworkModelImpl.java to ensure a comprehensive understanding of the throttling rules for both NIC types.
Propose a consistent or more intuitive mechanism for throttling, balancing functionality and ease of configuration.
Incorporate input from the team to ensure alignment with the broader system design.
Could you assign this issue to me so I can take the lead on the discussion and possible implementation? I’m excited to contribute!
Looking forward to your thoughts.
Best regards,
Ishita
iishitahere commentedon Nov 29, 2024
Thank you for assigning me to this issue! I’ll start working on it right away and keep you updated on my progress. Please let me know if there’s anything specific you’d like me to focus on. Thanks again for the opportunity!
JoaoJandre commentedon Nov 29, 2024
You're correct on the throttling behavior @BryanMLima.
I also think that this behavior is very unintuitive for users, the throttling should happen either only on the network offering or on the compute offering. Personally I think that only having the setting in the network offering makes more sense. However, we could also have a setting to determine if the throttle (for all NICs of a VM) will be based on the compute or the network offering.
@iishitahere I would wait for some other opinions before implementing anything, if you're not familiar with ACS and its code, I would start by getting used to it. Once the way forward is established, I would start the actual code changes (this is just my opinion, feel free to ignore it). In any case, I think that the easiest way forward would be creating that configuration that I mentioned.
Furthermore, for maximum configurability, we could eventually add a feature to allow configuring the throttling of each NIC of a given VM separately, regardless of offerings. I know that @hsato03 was working on this specific feature.
DaanHoogland commentedon Dec 4, 2024
@iishitahere , very welcome to the community (again). I have seen your name on several issues. Can you (introduce yourself and) tell us which ones you are taking on first, on dev@cloudstack.apache.org?
I hope you are successful in submitting code, soon.
DaanHoogland commentedon Dec 4, 2024
oh and if it is this one, do you plan to submit your code for 4.19.2 or later (just an administrative question, you can always change your mind later)
iishitahere commentedon Dec 4, 2024
Hi @DaanHoogland,
Thank you for reaching out and for your warm welcome to the community!
I am currently working on Issue #10007 - Wrong Traffic Shaping in Secondary NIC over L2 Networks. My primary focus is on analyzing the disparity between the bandwidth allocation for the default NIC and the secondary NICs in L2 networks, as described in the issue. I am addressing this by reviewing the configuration of VMware Traffic Shaping in relation to the global setting (vm.network.throttling.rate) and the network offer parameters.
I plan to submit my code for this fix by December 6th, targeting version 4.19.2 for now. Please let me know if there are any specific guidelines or suggestions for this fix.
Looking forward to contributing and receiving feedback from the community!
iishitahere commentedon Dec 4, 2024
Hi @DaanHoogland ,
Thank you so much for the warm welcome to the community! I am thrilled to be a part of this amazing group and contribute to the Apache CloudStack project.
A bit about me: I am Ishita Jaiswal, a first-year B.Tech student with a strong interest in open-source projects. I specialize in Java development and am particularly enthusiastic about debugging and resolving issues in complex systems like CloudStack.
I’m looking forward to contributing and learning more from this incredible community. Please feel free to share any feedback or suggestions as I work on this!
Best regards,
Ishita Jaiswal
iishitahere commentedon Dec 7, 2024
Hi @DaanHoogland ,
I hope this message finds you well. I apologize for the delay in submitting this fix; I was one day late due to unforeseen circumstances. Thank you for your patience.
Regarding the issue, I’ve made the following changes in the VmwareTrafficLabel class to resolve the inconsistent bandwidth allocation for secondary NICs in VMware L2 networks:
Traffic Label Parsing Enhancement: Updated the _parseLabel method to ensure consistent interpretation of the traffic label, irrespective of NIC type.
Default Values Adjustment: Ensured that both the default NIC and secondary NICs adhere to the same bandwidth specifications derived from the network offering or global throttling settings.
Error Handling Improvement: Added checks to validate and handle unexpected or invalid traffic label fields gracefully, reducing potential misconfigurations.
Code Cleanup: Simplified logic and ensured compliance with the project's coding standards for better readability and maintainability.
These changes aim to align the bandwidth configuration across all NICs of a VM, as per the expected behavior described in the issue report. I have tested the changes locally to verify their correctness.
Please let me know if there’s anything else I need to address or refine. I look forward to your feedback and further guidance.
Thank you again for your time and support!
Best regards,
Ishita Jaiswal
DaanHoogland commentedon Dec 8, 2024
hi @iishitahere (Ishita),
this works sounds awesome, but we deal (communicate largely through code and/or illustrated by code. Can I ask you to submit a PR and discuss the changes there?
You do not have to apologise for anything. We do not expect things and everything is a bonus. We are very glad that you are willing to work on the codebase.
regards,
7 remaining items
luganofer commentedon Jun 2, 2025
I add a comment, I observed the same behaviour with secondary NICs connected to an isolated network.
Example: A VM is connected to 2 networks; NIC1 takes the traffic shapping value of the ‘vm.network.throttling.rate’ parameter while NIC2 takes the value of ‘the network offer’. In this example the value of vm.network.throttling.rate is 10,000 (10 Gbps) while the bandwidth value of ‘the network offer’ used by NIC 2 is 1,000 (1 Gbps). In NIC 1, VMware uses the value 10,000 for traffic shaping, while in NIC2 it takes the value of ‘the network offer’ which is 1,000. This does not seem to be consistent.