I am currently setting up kubernetes HA production cluster and am running into some issues with the CloudInit images on the DHCP network and moving the control-plane nodes to static ip addresses.
I have still yet to find out what the problem was for this issue. After messing around with some configuration settings on the firewall boxes, I have come to the conclusion that it was hardware offloading on the firewalls causing the issue. When I was configuring HA Sync for the firewalls I think I must have forgotten to disable hardware offloading.
Either way - with these two boxes being the control planes, I decided against the cloudinit images and went with the plain old unbunt 20.04 image and made them special boxes with static IPs.
I’d like to dig into this issue more to troubleshoot the connection issues, but as my time stands there isn’t really a problem anymore now that hw offloading is disabled and I don’t want to waste my own time.