Direct Server Return Loadbalancing in ACI

Direct Server Loadbalancing is a loadbalancing mechanism in which traffic is sent via a Virtual IP (VIP) to one or more Real Servers. These servers are in the same Layer 2 domain as the loadbalancer itself and will respond directly to the client without the return traffic going through the loadbalancer on the way back. This differs from loadbalancing based on source NAT or when using the loadbalancer as gateway. The following image shows Direct Server Return (DSR) Loadbalancing.

Direct Server Loadbalancing

In normal networks this is not a big issue as long as both loadbalancer and real server are in the same vlan and the realservers don’t respond to ARP messages.

ACI works a bit different. Where a normal switch does not learn the location of an IP address, the ACI fabric does. And whats worse, it learns based on every IP packet, not just ARP packets. This will cause a DSR loadbalancing system to fail in an ACI fabric. Because of this the fabric must be told not to learn the VIP addres. This can be done under the EPG. The IP address of the loadbalancer must be added as a L4-L7 Virtual IP. For this to work the following requirements must be met:

  • The EPG must be configured to use a contract (provide or consume).
  • The option EP Move Detection Mode, GARP based detection must be enabled to be able to perform failover.
  • The real servers must be configured not to reply to ARP requests for the VIP address.

Step 1: Configure the L4-L7 Virtual IP

  • Go to Tenant, Application Profiles, Your Application Profile, Your EPG, L4-L7 Virtual IPs
  • Right click and choose Create L4-L7 Virtual IP
  • Enter the IP address of the VIP
  • click Submit

See the following images.

L4-L7 Virtual IPs

L4-L7 Virtual IP

Step 2: Configure GARP based EP move detection

  • Go to Tenant, Networking, Bridge Domains, Your Bridge Domain, L3 Configuration
  • Check the box at GARP based detection
  • Click Submit

See the following image

GARP based EP move detection

Conclusion and Verification

Now you should have a Virtual IP configured for the EPG. This IP address is only learned using ARP messages and will move based on GARP messages. This will enable failover in the event of a loadbalancer failure.

You can verify that the fabric learns the correct mac address of the loadbalancer by using the command:

aci-leaf-101# show system internal epm endpoint ip a.b.c.d

This will show you the mac address of the system. This should match the mac address of the loadbalancer. Furthermore you will see the line EP Flags at the bottom, this should show VIP. An example is the following (I don’t really have a DSR loadbalancer in my lab, so the below example is crafted and could differ from your environment):

MAC : 0000.0164.6464 ::: Num IPs : 1
IP# 0 : 10.10.10.10 ::: IP# 0 flags : VIP
Vlan id : 110 ::: Vlan vnid : 15171544 ::: VRF name : Michael:default
BD vnid : 15171544 ::: VRF vnid : 2260994
Phy If : 0x901006e ::: Tunnel If : 0
Interface : Eth1/1
Flags : 0x4000404 ::: sclass : 0 ::: Ref count : 3
EP Create Timestamp : 05/03/2018 14:45:12.659640
EP Update Timestamp : 05/03/2018 14:45:12.659640
EP Flags : VIP|psvi|
::::