FortiOS v3.0 HA Cluster virtual MAC addresses

When a FortiOS v3.0 cluster is operating, the FGCP assigns virtual MAC addresses to each primary unit interface. The FGCP uses virtual MAC addresses so that if a failover occurs, the new primary unit interfaces will have the same MAC addresses as the failed primary unit interfaces. If the MAC addresses change after a failover, the network would take longer to recover because all attached network devices would have to learn the new MAC addresses before they could communicate with the cluster.

 

If a cluster is operating in NAT/Route mode, the FGCP assigns a different virtual MAC address to each primary unit interface. VLAN subinterfaces are assigned the same virtual MAC address as the physical interface that the VLAN subinterface is added to. Redundant interfaces or 802.3ad aggregate interfaces are assigned the virtual MAC address of the first interface in the redundant or aggregate list.

 

If a cluster is operating in Transparent mode, the FGCP assigns a virtual MAC address for the primary unit management IP address. Since you can connect to the management IP address from any interface, all of the FortiGate interfaces appear to have the same virtual MAC address.

 

When a cluster starts up, after a failover, the primary unit sends gratuitous ARP packets to update the switches connected to the cluster interfaces with the virtual MAC address. The switches update their MAC forwarding tables with this MAC address. As a result, the switches direct all network traffic to the primary unit. Depending on the cluster configuration, the primary unit either processes this network traffic itself or load balances the network traffic among all of the cluster units.

 

Note After a failover, because the new primary unit has the same IP addresses and MAC addresses as the failed primary unit, once the switches updated their MAC forwarding tables no information about the failover needs to be communicated to other network devices.

• FortiGate units 60 and above.
• FortiOS v3.0 all maintenance releases

The cluster virtual MAC addresses depend on the cluster group ID. In most cases you can operate the cluster with the default group ID of zero. However, if you have more than one FortiGate cluster on the same network, each cluster should have a different group ID. If two clusters on the same network have the same group ID, duplicate MAC addresses could cause addressing conflicts on the network. You can change the group ID from the FortiGate CLI using the following command:

 

config system ha
    set group-id <id_integer>
end

How the virtual MAC address is determined

The virtual MAC address is determined based on following formula:

 

00-09-0f-06-<group-id_hex>-<vcluster_integer><idx>

where

 

<group-id_hex> is the HA group ID for the cluster converted to hexadecimal.

 

<vcluster_integer> is 0 for virtual cluster 1 and 2 for virtual cluster 2. If virtual domains are not enabled, HA sets the virtual cluster to 1 and by default all interfaces are in the root virtual domain. Including virtual cluster and virtual domain factors in the virtual MAC address formula means that the same formula can be used whether or not virtual domains and virtual clustering is enabled.

 

<idx> iIn NAT/Route mode, interfaces are numbered from 0 to x (where x is the number of interfaces). The interfaces are listed in alphabetical order on the web-based manager and CLI. The interface at the top of the interface list is first in alphabetical order by name and has an index of 0. The second interface in the list has an index of 1 and so on. In Transparent mode, the index number foe the management IP address is 0.

 

The second last part of the virtual MAC address depends on the HA group ID and is the same for each cluster interface. The last part of the virtual MAC address is different for each cluster interface.

Example virtual MAC addresses

A FortiGate-500 operating in HA mode where the HA group ID has not been changed (default=0) and virtual domains have not been enabled would have the following virtual MAC addresses:

 

• dmz interface virtual MAC: 00-09-0f-09-00-00
• external interface virtual MAC: 00-09-0f-09-00-01
• ha interface virtual MAC: 00-09-0f-09-00-02
• Internal interface virtual MAC: 00-09-0f-09-00-03
• port1 interface virtual MAC: 00-09-0f-09-00-04
• port2 interface virtual MAC: 00-09-0f-09-00-05
• port3 interface virtual MAC: 00-09-0f-09-00-06
• port4 interface virtual MAC: 00-09-0f-09-00-07
• port5 interface virtual MAC: 00-09-0f-09-00-08
• port6 interface virtual MAC: 00-09-0f-09-00-09
• port7 interface virtual MAC: 00-09-0f-09-00-0a
• port8 interface virtual MAC: 00-09-0f-09-00-0b

If the group ID is changed to 34 these virtual MAC addresses change to:

• dmz interface virtual MAC: 00-09-0f-09-22-00
• external interface virtual MAC: 00-09-0f-09-22-01
• ha interface virtual MAC: 00-09-0f-09-22-02
• Internal interface virtual MAC: 00-09-0f-09-22-03
• port1 interface virtual MAC: 00-09-0f-09-22-04
• port2 interface virtual MAC: 00-09-0f-09-22-05
• port3 interface virtual MAC: 00-09-0f-09-22-06
• port4 interface virtual MAC: 00-09-0f-09-22-07
• port5 interface virtual MAC: 00-09-0f-09-22-08
• port6 interface virtual MAC: 00-09-0f-09-22-09
• port7 interface virtual MAC: 00-09-0f-09-22-0a
• port8 interface virtual MAC: 00-09-0f-09-22-0b

All of the interfaces of a FortiGate-800 HA cluster operating in Transparent mode with group ID set to 10 have the virtual MAC 00-09-0f-09-0a-00.

A FortiGate-5001SX operating in HA mode with virtual domains enabled where the HA group ID has been changed to 23, port5 and port 6 are in the root virtual domain (which is in virtual cluster1), and port7 and port8 are in the vdom_1 virtual domain (which is in virtual cluster 2) would have the following virtual MAC addresses:

 

• port5 interface virtual MAC: 00-09-0f-09-23-05
• port6 interface virtual MAC: 00-09-0f-09-23-06
• port7 interface virtual MAC: 00-09-0f-09-23-27
• port8 interface virtual MAC: 00-09-0f-09-23-28

Virtual MAC address conflicts

If two or more clusters are operating on the same network, there is a possibility that a MAC address conflict can occur. Because all clusters use the same formula to calculate cluster virtual MAC addresses, a MAC address conflict can occur in the following configurations:

 

• Two clusters are operating on the same network in NAT/Route mode and both clusters have the cluster interface with the same index number connected to the network. For example, both clusters could be using the same FortiGate model and the same interface of each cluster could be connected to the network. This can also happen if each cluster is using a different FortiGate model but the interfaces connected to the network have the same network index.
• Two clusters are operating on the same network in Transparent mode. In this case, all interfaces of both clusters have the same MAC address.
• Two clusters are operating on the same network, one in NAT/Route mode and one in Transparent mode. In this case a conflict can occur of NAT/Route mode cluster interface with interface index 0 is connected to the same network as the cluster operating in Transparent mode.

The solution to all of these conflicts is to use the config system ha group-id CLI command to change the HA group ID of one or both of the clusters. In general it is recommended that you change the group-id if you are connecting two clusters to the same network.