Skip to content

VM not picking assigned IP through VR #10608

Closed
Closed
VM not picking assigned IP through VR#10608
Milestone
unplanned

Description

@rbaweja
rbaweja
opened on Mar 22, 2025

problem

A VM deployed on Apache CloudStack (ACS) is unable to acquire a DHCP-assigned IP address from the Virtual Router (VR). The DHCP request is visible in the VR’s tcpdump, but the VM does not receive an IP. However, manually assigning an IP to the VM allows it to reach both the VR and the internet.

versions

Environment Details
• CloudStack Version: 4.20
• Hypervisor: KVM
• Virtual Router IP: 10.0.0.1/24
• Network Setup: Advanced network with VR acting as DHCP server

The steps to reproduce the bug

  1. VM sends a DHCP renew request.
  2. VR receives the request, as confirmed via tcpdump -i eth2 port 67 or port 68 -n.
  3. VR responds with a DHCP reply to the VM.
  4. The VM does not acquire the IP.
  5. Manually assigning an IP allows the VM to reach both VR and the internet.

What to do about it?

No response

Activity

  1. boring-cyborg commented on Mar 22, 2025

    @boring-cyborg
    boring-cyborgbot
    on Mar 22, 2025 – with boring-cyborg

    Thanks for opening your first issue here! Be sure to follow the issue template!

  2. weizhouapache commented on Mar 22, 2025

    @weizhouapache
    weizhouapache
    on Mar 22, 2025
    Member

    This may be same as #10436

  3. rbaweja commented on Mar 22, 2025

    @rbaweja
    rbaweja
    on Mar 22, 2025
    Author

    This may be same as #10436

    i saw this post earlier, but didn't get solution by that method..

    Do you suggest that i post it there or continue here?

  4. DaanHoogland commented on Mar 24, 2025

    @DaanHoogland
    DaanHoogland
    on Mar 24, 2025
    Contributor

    @rbaweja , so the dhcp reply is sent. Can you check on the client logs if anything goes wrong there? I think in the case of #10436 the lease is not acquired at all, so this is a different issue it seems.

  5. rbaweja commented on Mar 24, 2025

    @rbaweja
    rbaweja
    on Mar 24, 2025
    Author

    On VM:

    Image

    New VR Logs:

    Tue Mar 18 09:43:11 PM UTC 2025 Starting guest services for kvm
    Tue Mar 18 09:43:13 PM UTC 2025 acpiphp and pci_hotplug module already compiled in
    Tue Mar 18 09:43:14 PM UTC 2025 Received a new non-empty cmdline file from qemu-guest-agent
    Tue Mar 18 09:43:14 PM UTC 2025 Booting from cloudstack, remove old configuration files in /etc/cloudstack/
    Tue Mar 18 09:43:14 PM UTC 2025 Applying iptables rules
    Tue Mar 18 09:43:14 PM UTC 2025 Setting up interface: eth0
    Tue Mar 18 09:43:14 PM UTC 2025 Set up route for management network: via local gateway: for device eth0 for hypervisor: kvm
    Tue Mar 18 09:43:16 PM UTC 2025 Executing cloud-early-config
    Tue Mar 18 09:43:16 PM UTC 2025 Scripts checksum detected: oldmd5=0e40239dbb8573201b0a80a411b03b12 newmd5=0e40239dbb8573201b0a80a411b03b12
    Tue Mar 18 09:43:18 PM UTC 2025 Could not find patch file, retrying
    Tue Mar 18 09:43:20 PM UTC 2025 Could not find patch file, retrying
    Tue Mar 18 09:43:22 PM UTC 2025 Could not find patch file, retrying
    Tue Mar 18 09:43:24 PM UTC 2025 Could not find patch file, retrying
    Tue Mar 18 09:43:24 PM UTC 2025 Scripts checksum detected: oldmd5=0e40239dbb8573201b0a80a411b03b12 newmd5=82d3395ea6108bf4a1ceaefdffb7a3f9
    Tue Mar 18 09:43:24 PM UTC 2025 Patched scripts using /var/cache/cloud/cloud-scripts.tgz
    Tue Mar 18 09:43:24 PM UTC 2025 Bootstrapping systemvm appliance
    ...skipping...
    2025-03-24 14:07:48,627 INFO Executing: ip link show eth2 | grep ' state '
    2025-03-24 14:07:48,631 INFO Executing2: arping -c 1 -I eth2 -A -U -s 10.10.10.1 10.10.10.1
    2025-03-24 14:07:48,632 INFO Adding route: dev eth2 table: Table_eth2 network: 10.10.10.0/24 if not present
    2025-03-24 14:07:48,632 INFO Executing: ip route show 10.10.10.0/24 table Table_eth2 proto static
    2025-03-24 14:07:48,636 INFO Executing: sudo ip route flush cache
    2025-03-24 14:07:48,647 ERROR Not able to setup source-nat for a regular router yet
    2025-03-24 14:07:48,647 INFO Not making dns publicly available
    2025-03-24 14:07:48,649 INFO Executing: systemctl start cloud-password-server@10.10.10.1
    2025-03-24 14:07:48,659 INFO Service cloud-password-server@10.10.10.1 start
    2025-03-24 14:07:48,660 INFO Nothing to commit. The /etc/monitor.conf file did not change
    2025-03-24 14:07:48,660 INFO Nothing to commit. The /etc/cron.d/process file did not change
    2025-03-24 14:07:48,661 INFO Executing: systemctl stop conntrackd
    2025-03-24 14:07:48,674 INFO Service conntrackd stop
    2025-03-24 14:07:48,674 INFO Executing: systemctl stop keepalived
    2025-03-24 14:07:48,688 INFO Service keepalived stop
    2025-03-24 14:07:48,688 INFO Executing: mount

  6. DaanHoogland commented on Mar 24, 2025

    @DaanHoogland
    DaanHoogland
    on Mar 24, 2025
    Contributor

    @rbaweja, can you check /var/log/dnsmasq.log on the router for any errors? I thought an answer went out of the VR as per you initial description but on second read it seems only the request may have come in.

  7. rbaweja commented on Mar 24, 2025

    @rbaweja
    rbaweja
    on Mar 24, 2025
    Author

    @rbaweja, can you check /var/log/dnsmasq.log on the router for any errors? I thought an answer went out of the VR as per you initial description but on second read it seems only the request may have come in.

    the worst part is there are 2 routers and both have same issue, i tried with default 5.5 centos template but same

    Mar 24 15:58:34 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:34 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:34 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:43 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPDISCOVER(eth2) 02:01:00:de:00:01
    Mar 24 15:58:59 dnsmasq-dhcp[3006]: DHCPOFFER(eth2) 10.0.0.138 02:01:00:de:00:01

  8. DaanHoogland commented on Mar 24, 2025

    @DaanHoogland
    DaanHoogland
    on Mar 24, 2025
    Contributor

    that log looks ok,

    you said

    The DHCP request is visible in the VR’s tcpdump, but the VM does not receive an IP

    do you mean you see the request coming in, but no response going out from the VR using tcpdump?

  9. rbaweja commented on Mar 24, 2025

    @rbaweja
    rbaweja
    on Mar 24, 2025
    Author

    as per response i see that the response is also tehre but VM is not receiving the response.

    tcpdump for VR:

    17:15:21.881789 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:21.881790 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:21.881790 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:21.881790 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:21.882263 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:21.882528 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:21.882773 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:21.883042 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:29.882073 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:29.882074 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:29.882074 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:29.882074 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:29.882559 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:29.882815 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:29.882950 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:29.883171 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:41.883434 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:41.883694 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:41.883694 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:41.883694 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:41.884023 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:41.884335 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:41.884541 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:41.884779 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:51.883875 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:51.883876 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:51.884042 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:51.884042 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:01:00:cc:00:16, length 300
    17:15:51.884429 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:51.884779 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:51.885070 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344
    17:15:51.885342 IP 10.10.10.1.67 > 10.10.10.169.68: BOOTP/DHCP, Reply, length 344

  10. weizhouapache commented on Mar 24, 2025

    @weizhouapache
    weizhouapache
    on Mar 24, 2025
    Member

    are there firewall rules in the vm ?

    are there firewall rules with the kvm host ?

  11. rbaweja commented on Mar 24, 2025

    @rbaweja
    rbaweja
    on Mar 24, 2025
    Author

    are there firewall rules in the vm ?

    are there firewall rules with the kvm host ?

    No firewall rules. Even the fresh VM of centos 5.5 doesn’t get IP

  12. weizhouapache commented on Mar 24, 2025

    @weizhouapache
    weizhouapache
    on Mar 24, 2025
    Member

    are there firewall rules in the vm ?

    are there firewall rules with the kvm host ?

    No firewall rules. Even the fresh VM of centos 5.5 doesn’t get IP

    Are the vm and vr running on the same host ?

  13. rbaweja commented on Mar 25, 2025

    @rbaweja
    rbaweja
    on Mar 25, 2025
    Author

    are there firewall rules in the vm ?
    are there firewall rules with the kvm host ?

    No firewall rules. Even the fresh VM of centos 5.5 doesn’t get IP

    Are the vm and vr running on the same host ?

    all on one host for now.

  14. weizhouapache commented on Mar 25, 2025

    @weizhouapache
    weizhouapache
    on Mar 25, 2025
    Member

    all on one host for now.

    @rbaweja
    can you restart network with cleanup and retry ?

    please use tcpdump -vvv instead of tcpdump

  15. rbaweja commented on Mar 25, 2025

    @rbaweja
    rbaweja
    on Mar 25, 2025
    Author

    tcpdump -vvv instead of tcpdump

    this is what i get -

    09:35:47.319133 IP (tos 0xc0, ttl 64, id 6910, offset 0, flags [none], proto UDP (17), length 372)
    10.10.10.1.67 > 10.10.10.169.68: [udp sum ok] BOOTP/DHCP, Reply, length 344, xid 0xc11f8f51, secs 20, Flags [none] (0x0000)
    Your-IP 10.10.10.169
    Server-IP 10.10.10.1
    Client-Ethernet-Address 02:01:00:cc:00:16
    Vendor-rfc1048 Extensions
    Magic Cookie 0x63825363
    DHCP-Message (53), length 1: Offer
    Server-ID (54), length 4: 10.10.10.1
    Lease-Time (51), length 4: 4294967295
    BR (28), length 4: 10.10.10.255
    Hostname (12), length 39: "vm-d0d0ba5c-24e0-4acb-8cf0-785a1bef0e49"
    Subnet-Mask (1), length 4: 255.255.255.0
    Default-Gateway (3), length 4: 10.10.10.1
    Domain-Name-Server (6), length 8: 10.10.10.1,172.27.5.11
    Domain-Name (15), length 17: "cs2cloud.internal"
    END (255), length 0
    09:35:47.319469 IP (tos 0xc0, ttl 64, id 6911, offset 0, flags [none], proto UDP (17), length 372)
    10.10.10.1.67 > 10.10.10.169.68: [udp sum ok] BOOTP/DHCP, Reply, length 344, xid 0xc11f8f51, secs 20, Flags [none] (0x0000)
    Your-IP 10.10.10.169
    Server-IP 10.10.10.1
    Client-Ethernet-Address 02:01:00:cc:00:16
    Vendor-rfc1048 Extensions
    Magic Cookie 0x63825363
    DHCP-Message (53), length 1: Offer
    Server-ID (54), length 4: 10.10.10.1
    Lease-Time (51), length 4: 4294967295
    BR (28), length 4: 10.10.10.255
    Hostname (12), length 39: "vm-d0d0ba5c-24e0-4acb-8cf0-785a1bef0e49"
    Subnet-Mask (1), length 4: 255.255.255.0
    Default-Gateway (3), length 4: 10.10.10.1
    Domain-Name-Server (6), length 8: 10.10.10.1,172.27.5.11
    Domain-Name (15), length 17: "cs2cloud.internal"
    END (255), length 0

  16. 2 remaining items

  17. weizhouapache commented on Mar 25, 2025

    @weizhouapache
    weizhouapache
    on Mar 25, 2025
    Member

    VR looks ok

    any packets received in the vm ?

  18. rbaweja commented on Mar 25, 2025

    @rbaweja
    rbaweja
    on Mar 25, 2025
    Author

    VR looks ok

    any packets received in the vm ?

    Image (on VM side)

  19. weizhouapache commented on Mar 25, 2025

    @weizhouapache
    weizhouapache
    on Mar 25, 2025
    Member

    VR looks ok
    any packets received in the vm ?

    Image (on VM side)

    it looks like no packets received.

    do you use linux bridge or openvswitch ?

    if you use linux bridge, can you capture the packets of the linux bridge br<devicename>-<vlanid> and the physical interface <devicename>.<vlanid> ?

  20. rbaweja commented on Mar 25, 2025

    @rbaweja
    rbaweja
    on Mar 25, 2025
    Author

    VR looks ok
    any packets received in the vm ?

    Image (on VM side)

    it looks like no packets received.

    do you use linux bridge or openvswitch ?

    if you use linux bridge, can you capture the packets of the linux bridge br<devicename>-<vlanid> and the physical interface <devicename>.<vlanid> ?

    that also seems to be passing

    10:42:23.421152 IP (tos 0xc0, ttl 64, id 11870, offset 0, flags [none], proto UDP (17), length 372)
    10.10.10.1.67 > 10.10.10.169.68: [udp sum ok] BOOTP/DHCP, Reply, length 344, xid 0xf4d7fe4b, secs 12, Flags [none] (0x0000)
    Your-IP 10.10.10.169
    Server-IP 10.10.10.1
    Client-Ethernet-Address 02:01:00:cc:00:16
    Vendor-rfc1048 Extensions
    Magic Cookie 0x63825363
    DHCP-Message (53), length 1: Offer
    Server-ID (54), length 4: 10.10.10.1
    Lease-Time (51), length 4: 4294967295
    BR (28), length 4: 10.10.10.255
    Hostname (12), length 39: "vm-d0d0ba5c-24e0-4acb-8cf0-785a1bef0e49"
    Subnet-Mask (1), length 4: 255.255.255.0
    Default-Gateway (3), length 4: 10.10.10.1
    Domain-Name-Server (6), length 8: 10.10.10.1,172.27.5.11
    Domain-Name (15), length 17: "cs2cloud.internal"
    10:42:23.599399 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:06:7b:c1.8098, length 42
    message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 0, port-role Designated
    10:42:23.599491 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:06:7b:c1.8097, length 42
    message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 0, port-role Designated
    10:42:24.527682 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:9c:98:fc.809b, length 42
    message-age 1.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 2, port-role Designated
    10:42:24.527816 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:9c:98:fc.8098, length 42
    message-age 1.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 2, port-role Designated
    10:42:25.646244 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:06:7b:c1.8098, length 42
    message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 0, port-role Designated
    10:42:25.646370 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:06:7b:c1.8097, length 42
    message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 0, port-role Designated
    10:42:26.568470 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:9c:98:fc.809b, length 42
    message-age 1.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 2, port-role Designated
    10:42:26.568640 STP 802.1w, Rapid STP, Flags [Learn, Forward], bridge-id 84a7.00:2a:6a:9c:98:fc.8098, length 42
    message-age 1.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
    root-id 84a7.00:2a:6a:06:7b:c1, root-pathcost 2, port-role Designated

  21. rbaweja commented on Mar 25, 2025

    @rbaweja
    rbaweja
    on Mar 25, 2025
    Author

    explaining the scenario:
    i have 3 network cards on my KVM
    Cloudbr0 - Access mode for management
    Cloudbr1 - guest network in trunk mode
    cloudbr2 - public network in trunk mode.

    i was using it with above scenario.

    now i tried with all different versions but nothing worked. so i used cloudbr0 for both management and guest the VM got the IP.

    since cloudbr0 is in access mode the traffic of other VLAN will not go to other hosts since it will not communicate.

    now my question is how to keep all 3 networks isolated? shall i configure cloudbr1 with management vlan in native and rest as trunk ? or all should be like that?

  22. DaanHoogland commented on Mar 26, 2025

    @DaanHoogland
    DaanHoogland
    on Mar 26, 2025
    Contributor

    @rbaweja @weizhouapache , could this be a matter of asymetric routing? (just having a wild stab at it)

  23. rbaweja commented on Mar 26, 2025

    @rbaweja
    rbaweja
    on Mar 26, 2025
    Author

    @rbaweja @weizhouapache , could this be a matter of asymetric routing? (just having a wild stab at it)

    Not really sure.

  24. weizhouapache commented on Mar 26, 2025

    @weizhouapache
    weizhouapache
    on Mar 26, 2025
    Member

    explaining the scenario: i have 3 network cards on my KVM Cloudbr0 - Access mode for management Cloudbr1 - guest network in trunk mode cloudbr2 - public network in trunk mode.

    i was using it with above scenario.

    now i tried with all different versions but nothing worked. so i used cloudbr0 for both management and guest the VM got the IP.

    since cloudbr0 is in access mode the traffic of other VLAN will not go to other hosts since it will not communicate.

    now my question is how to keep all 3 networks isolated? shall i configure cloudbr1 with management vlan in native and rest as trunk ? or all should be like that?

    @rbaweja
    have you set the network traffic label of physical networks ?

    refer to "Choose which traffic types will be carried by the physical network" section on https://docs.cloudstack.apache.org/en/latest/installguide/configuration.html

  25. rbaweja commented on Mar 27, 2025

    @rbaweja
    rbaweja
    on Mar 27, 2025
    Author

    yes the label and configuration is perfect.

    below for your reference:

    network:
    version: 2
    ethernets:
    ens160: {} # MNGT Network
    ens192: {} # GUEST Trunk
    ens224: {} # PUBLIC Trunk
    bridges:
    cloudbr0:
    addresses:
    - 172.27.5.31/24
    dhcp4: false
    routes:
    - to: default
    via: 172.27.5.1
    nameservers:
    addresses:
    - 8.8.8.8
    - 1.1.1.1
    search: []
    interfaces:
    - ens160
    parameters:
    stp: false
    forward-delay: 5
    cloudbr1:
    dhcp4: false
    interfaces:
    - ens192
    cloudbr2:
    dhcp4: false
    interfaces:
    - ens224
    parameters:
    stp: false
    forward-delay: 5

    /etc/cloudstack/agent/agent.properties
    private.network.device=cloudbr0
    guest.network.device=cloudbr1
    hypervisor.type=kvm
    port=8250
    public.network.device=cloudbr2

  26. weizhouapache commented on Mar 27, 2025

    @weizhouapache
    weizhouapache
    on Mar 27, 2025
    Member

    yes the label and configuration is perfect.

    below for your reference:

    network: version: 2 ethernets: ens160: {} # MNGT Network ens192: {} # GUEST Trunk ens224: {} # PUBLIC Trunk bridges: cloudbr0: addresses: - 172.27.5.31/24 dhcp4: false routes: - to: default via: 172.27.5.1 nameservers: addresses: - 8.8.8.8 - 1.1.1.1 search: [] interfaces: - ens160 parameters: stp: false forward-delay: 5 cloudbr1: dhcp4: false interfaces: - ens192 cloudbr2: dhcp4: false interfaces: - ens224 parameters: stp: false forward-delay: 5

    /etc/cloudstack/agent/agent.properties private.network.device=cloudbr0 guest.network.device=cloudbr1 hypervisor.type=kvm port=8250 public.network.device=cloudbr2

    @rbaweja
    can you check if the cloudbrX and its interface ensX have the same mac addresses ?

  27. rbaweja commented on Mar 27, 2025

    @rbaweja
    rbaweja
    on Mar 27, 2025
    Author

    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    valid_lft forever preferred_lft forever
    2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master cloudbr0 state UP group default qlen 1000
    link/ether 00:50:56:95:74:67 brd ff:ff:ff:ff:ff:ff
    altname enp3s0
    3: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master cloudbr1 state UP group default qlen 1000
    link/ether 00:50:56:95:58:d7 brd ff:ff:ff:ff:ff:ff
    altname enp11s0
    4: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master cloudbr2 state UP group default qlen 1000
    link/ether 00:50:56:95:31:af brd ff:ff:ff:ff:ff:ff
    altname enp19s0
    5: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 3e:f7:89:e8:5f:f3 brd ff:ff:ff:ff:ff:ff
    inet 172.27.5.31/24 brd 172.27.5.255 scope global cloudbr0
    valid_lft forever preferred_lft forever
    inet6 fe80::3cf7:89ff:fee8:5ff3/64 scope link
    valid_lft forever preferred_lft forever
    6: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 42:6a:c2:33:d5:7e brd ff:ff:ff:ff:ff:ff
    inet6 fe80::406a:c2ff:fe33:d57e/64 scope link
    valid_lft forever preferred_lft forever
    7: cloudbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether aa:fa:6d:75:e2:8d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a8fa:6dff:fe75:e28d/64 scope link
    valid_lft forever preferred_lft forever
    8: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ee:0e:49:fd:f7 brd ff:ff:ff:ff:ff:ff
    inet 169.254.0.1/16 scope global cloud0
    valid_lft forever preferred_lft forever
    inet6 fe80::e0ee:eff:fe49:fdf7/64 scope link
    valid_lft forever preferred_lft forever

  28. weizhouapache commented on Apr 10, 2025

    @weizhouapache
    weizhouapache
    on Apr 10, 2025
    Member

    @rbaweja
    Can you set the mac address in netplan so that the linux bridge and ethernet use the same mac address ?

    for example

        bridges:
        cloudbr0:   <=== linux bridge
            dhcp4: yes
            interfaces:
                - eth0      <====ethernet 
            macaddress: 1e:00:47:00:04:68   <========mac address of the ethernet
  29. apache
    locked and limited conversation to collaborators on Apr 10, 2025
  30. DaanHoogland
    converted this issue into a discussion #10693 on Apr 10, 2025
  31. DaanHoogland
    added this to the unplanned milestone on Sep 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions