Skip to content

Routed VR: accept VRRP and conntrackd sync on redundant routers - #14446

Open
hulucc wants to merge 1 commit into
apache:4.20from
hulucc:4.20-routed-vr-redundant-vrrp
Open

hulucc wants to merge 1 commit into
apache:4.20from
hulucc:4.20-routed-vr-redundant-vrrp

Conversation

@hulucc

@hulucc hulucc commented Oct 20, 2026

Copy link
Copy Markdown

Description

This PR fixes redundant VRs of ROUTED isolated networks and VPCs coming up as PRIMARY on both routers.

In ROUTED mode the VR flushes iptables and builds nftables tables (ip4_firewall for isolated networks, ip4_acl for VPCs) whose INPUT chain has a drop policy. The INPUT whitelist built by fw_router_routing and fw_vpcrouter_routing lacks VRRP (224.0.0.18) and conntrackd (225.0.0.50) multicast, which fw_router / fw_vpcrouter accept for non-routed networks. Both routers drop each other's VRRP advertisements and both become PRIMARY.

This adds iifname <guest dev> ip daddr 224.0.0.18 counter accept and iifname <guest dev> ip daddr 225.0.0.50 counter accept to the INPUT whitelist of both functions, plus unit tests. The rules match the destination address rather than the protocol because keepalived uses AH authentication, so the advertisements are IP protocol 51, not 112.

Redundant ROUTED offerings can be created on 4.20 (the RedundantRouter capability on the Gateway service was added for ROUTED in #9470), so this targets 4.20 as a bug fix, following CONTRIBUTING.md. If you would rather take it as part of #11565 on main, I can retarget the PR; the commit cherry-picks cleanly onto main and the new tests pass there.

Fixes: #14445

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Unit tests: the two new tests in TestCsAddress.TestCsIPRouted pass. pycodestyle (--max-line-length=179) and pylint report nothing new for the changed files; the two TestCsRoute failures in a container are pre-existing (they need a real default route).

Test environment: management server 4.20.3.1, KVM on CentOS 8, systemvm template 4.20.2. The change was applied to /usr/share/cloudstack-common/vms/cloud-scripts.tgz on the host, which the agent copies into the VR at boot (the new md5 showed up as the routers' scripts version).

  • Isolated network, ROUTED Static, redundant offering (created via API, RedundantRouter capability on the Gateway service): before the change both VRs were PRIMARY; after, PRIMARY/BACKUP, and the INPUT counters of the new rules show VRRP and conntrackd traffic being accepted. Forced stop of the primary: about 3 s of ping loss, and the stopped VR rejoins as BACKUP when started again (nopreempt).
  • Isolated network, ROUTED Dynamic, with an FRR peer: PRIMARY/BACKUP. The backup's BGP session stays Active (its public interface is down), and after a failover the new primary establishes the session.
  • VPC, ROUTED Dynamic, redundant VPC offering: PRIMARY/BACKUP, about 6 s of ping loss on failover.

How did you try to break this feature and the system with this change?

  • Only fw_router_routing and fw_vpcrouter_routing change, and both return early unless the network is ROUTED, so non-routed VRs are not affected.
  • The new rules accept only the two link-local multicast groups on the guest interface, the same addresses the iptables path already accepts.
  • On non-redundant routed networks the rules never match anything.

Not addressed here, but possibly useful for #11565: during a BGP failover the peer may refuse the new primary with a Connection Collision until the stale session from the old primary is torn down. With default timers this caused about 20 s without the route in our test, and with keepalive 3 s / hold 9 s on the peer the total failover was about 5 s.

🤖 Generated with Claude Code

In ROUTED mode the VR replaces iptables with nftables (table ip4_firewall
for isolated networks, ip4_acl for VPCs) whose INPUT chain has a drop
policy. The INPUT whitelist does not accept VRRP (224.0.0.18) and
conntrackd (225.0.0.50) multicast, which the iptables path allows via
"-A INPUT -d 224.0.0.18/32 -j ACCEPT" and "-d 225.0.0.50/32". With a
redundant router offering both VRs drop each other's advertisements and
both become PRIMARY, holding the guest gateway and public IPs at the
same time.

Accept both on the guest interface in fw_router_routing and
fw_vpcrouter_routing, and add unit tests for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boring-cyborg

boring-cyborg Bot commented Oct 20, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

@weizhouapache

Copy link
Copy Markdown
Member

@hulucc
thanks for raising the issue and creating the PR.

To be frank, when I developed the feature, I did not take redundant VRs into consideration. I will review and test the PR.

I am glad to see the feature is being tested or used. Are you doing PoC of routed network/vpc, or running it in production ?

@weizhouapache

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@weizhouapache
weizhouapache requested review from weizhouapache and a balanced review from Copilot October 20, 2026 10:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.41%. Comparing base (376c1b4) to head (8637dc4).

Additional details and impacted files
@@             Coverage Diff              @@
##               4.20   #14446      +/-   ##
============================================
- Coverage     16.41%   16.41%   -0.01%     
+ Complexity    13657    13655       -2     
============================================
  Files          5669     5669              
  Lines        501628   501628              
  Branches      60942    60942              
============================================
- Hits          82336    82323      -13     
- Misses       410064   410078      +14     
+ Partials       9228     9227       -1     
Flag Coverage Δ
uitests 4.16% <ø> (ø)
unittests 17.27% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants