A how-to manual: bonding modes, active-backup and LACP, link monitoring, hashing, failover tests, netplan and NetworkManager, VLANs and bridges, all tested on a lab
▶Prefer to watch? This video runs the guide's main commands on a real Ubuntu 24.04 machine, with on-screen captions and no voiceover. Click to play, or watch it on YouTube. It is part of the Networking and Firewalls playlist.
NIC bonding on Linux: two cables, one logical interface
cat /proc/net/bonding/bond0
Bonding (also called teaming, link aggregation or "NIC trunking", depending on whose brochure you read) joins two or more network cards into ONE logical interface, bond0. The server gets a single IP address and a single MAC address, and the kernel decides which physical card carries each frame. Done right, a dead cable, a dead card or a dead switch port becomes a line in the log instead of a phone call. Done wrong, you get a server with two cables that is exactly as fragile as a server with one, plus the confidence of a person who thinks it is redundant.
This guide builds bonds from scratch, tests failure on purpose, and explains the three things that trip everybody up: the default setting that does not notice a dead link, the myth that two cables make one connection twice as fast, and the switch that must be configured to match.
Every example below ran for real on a lab machine with the Linux bonding driver (kernel 7.0, Ubuntu 24.04 userland). The "network cards" were virtual ethernet pairs (nic1 and nic2) and the "switch" was a second network namespace, so no real hardware was harmed. Where the other end must speak LACP, the lab's peer was a second Linux bond; the real switch configuration is shown but was not run. On real hardware a failover loses about one monitoring interval of traffic (a few packets); the lab's virtual cables fail more politely than copper does. The lab ran as root; use sudo on a real server. Anything not run in the lab is marked "not run in the lab".
What bonding is, and what it is not
The kernel's bonding driver creates a virtual interface (bond0) and enslaves ("slaves", "members" or "ports", all the same thing) physical interfaces to it. Traffic goes out through the bond; the driver picks a member. Two different goals hide under the same name, and it is worth choosing one before typing anything:
redundancy survive a failed cable, card or switch. The simple, boring and valuable goal. Modes active-backup, broadcast.
more bandwidth carry more TOTAL traffic by using several cables at once. Modes 802.3ad (LACP), balance-xor, balance-tlb, balance-alb, balance-rr.
What it is not: a way to make one download faster. A single TCP connection is sent over one member, so a bond of two 1 Gbit cards gives you 2 Gbit across many connections, and still 1 Gbit for any one of them (the measurements are in the hashing section). It also does not protect you from the switch itself dying, unless the two cables go to two different switches, which needs either active-backup or a switch pair that supports multi-chassis link aggregation (MLAG, vPC, stacking). And it is not magic: both ends must agree about what is going on.
The bond takes the MAC address of its first member, and every member shows that same address:
$ ip -br link show | grep -E "nic|bond"
bond0 UP 5e:5d:52:85:06:41 <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP>
nic1@if4 UP 5e:5d:52:85:06:41 <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP>
nic2@if6 UP 5e:5d:52:85:06:41 <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP>
The flags tell the roles: MASTER is the bond, SLAVE is a member. Every member's real hardware address is still kept ("Permanent HW addr" in the next sections), which is how you find out which physical card is which.
The seven modes
The mode is the first thing you choose, and it cannot be changed while the bond has members. The numbers are the old way of naming them; the names work everywhere:
0 balance-rr round robin: frame 1 on member A, frame 2 on member B, and so on. Only mode where ONE connection uses all cables. Reorders packets. Needs a switch port group ("static etherchannel").
1 active-backup one member works, the others wait. If it fails, the next takes over. Needs NOTHING from the switch, works across two separate switches. The safe default.
2 balance-xor each frame goes to the member chosen by a hash of the addresses (source MAC xor destination MAC by default). Needs a static port group on the switch.
3 broadcast every frame is sent on EVERY member. Redundancy for the paranoid; halves nothing, doubles everything.
4 802.3ad LACP, the standard: both ends negotiate which cables form the group, and notice when one stops answering. Needs a LACP port-channel on the switch. The one to use for bandwidth.
5 balance-tlb adaptive transmit balancing: outgoing traffic spread by load, incoming on one member. No switch support needed.
6 balance-alb tlb plus receive balancing done with ARP tricks. No switch support needed, IPv4 only for the receive part, and not friendly to bridges.
The decision in two lines: no say over the switch (or two switches) => active-backup. A switch you control that can do LACP => 802.3ad. The rest exist, and each has a story, usually a long one.
The kernel driver accepts all seven; this is each name as the running kernel reports it:
$ ip link set bm type bond mode 802.3ad && grep "Bonding Mode" /proc/net/bonding/bm
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
balance-rr -> load balancing (round-robin) active-backup -> fault-tolerance (active-backup) balance-xor -> load balancing (xor)
broadcast -> fault-tolerance (broadcast) balance-tlb -> transmit load balancing balance-alb -> adaptive load balancing
A first bond: active-backup
The smallest useful bond, with iproute2 (everything here is lost at reboot; see the persistence sections for the permanent version). The members should be down and unconfigured (no IP address on them) when you add them; the bond brings them up:
$ ip link add bond0 type bond mode active-backup miimon 100
$ ip link set nic1 down
$ ip link set nic2 down
$ ip link set nic1 master bond0
$ ip link set nic2 master bond0
$ ip addr add 10.77.0.1/24 dev bond0
$ ip link set bond0 up
$ ip -br addr show bond0
bond0 UP 10.77.0.1/24 fe80::5c5d:52ff:fe85:641/64
$ ping -c 2 10.77.0.2 | tail -2
2 packets transmitted, 2 received, 0% packet loss, time 1028ms
rtt min/avg/max/mdev = 0.077/0.098/0.120/0.021 ms
Read it as: create the bond with its mode and the link check (miimon 100, explained below), release each card, attach it, give the BOND the address (never a member), bring it up. The IP address, default route and DNS belong to bond0 and nothing else. A member with its own IP is a classic mistake: it still answers, in a way nobody expects.
Take a card out again, or throw the whole bond away:
$ ip link set nic2 nomaster # remove one member, live
$ ip link del bond0 # remove the bond; the members are released and come back down
Reading /proc/net/bonding/bond0
This file is the control panel. It tells you the mode, which member is active, what each member's link is doing and how many times it has failed:
$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v7.0.0-38-generic
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: nic1
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
Slave Interface: nic1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 5e:5d:52:85:06:41
Slave queue ID: 0
Slave Interface: nic2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 62:db:7b:51:02:6d
Slave queue ID: 0
What to look at first:
Currently Active Slave in active-backup, the member carrying traffic. After a failover this is what changes.
MII Status up or down, per member and for the bond. Down on a member you did not expect = the cable, the card or the switch port.
MII Polling Interval how often the link is checked. 0 means NEVER. See the next section, it matters.
Link Failure Count how many times each member has dropped. A number that keeps growing is a flapping cable or a bad port: replace it before it picks a bad moment.
Speed, Duplex a member that negotiated 100 Mbps half duplex among gigabit friends is a bad cable or a wrong switch port, and LACP will refuse to group unequal members.
Permanent HW addr the real MAC of the card, to tell the cards apart when the bond hides them.
More ways to look, all read only: ip -d link show bond0 (all the settings on one long line), ip -s link show nic1 (per-card packet and error counters), ethtool bond0 (the aggregate speed, 20000Mb/s for two 10 Gbit members), cat /sys/class/net/bond0/bonding/slaves and mode, dmesg or journalctl -k for the driver's own story ("link status definitely down, disabling slave").
Link monitoring: the default that does not monitor
Here is the setting that makes a two-cable server exactly as available as a one-cable server. When you create a bond without miimon, the link is not checked. The file says it in plain sight and almost nobody reads it:
$ ip link add bond0 type bond mode active-backup # no miimon given
$ grep -E "MII Polling|Currently" /proc/net/bonding/bond0
Currently Active Slave: nic1
MII Polling Interval (ms): 0
Now the cable of the ACTIVE card is pulled. The card sees NO-CARRIER, and the bond carries on sending into the void:
$ ip -br link show nic1
nic1@if4 DOWN 5e:5d:52:85:06:41 <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP>
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic1
$ ping -c 3 -W 1 10.77.0.2 | tail -2 | head -1
3 packets transmitted, 0 received, 100% packet loss, time 2066ms
Two cables, no redundancy. Switch on the monitor on the running bond and it fixes itself within a monitoring interval:
$ ip link set bond0 type bond miimon 100
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic2
$ ping -c 3 -W 1 10.77.0.2 | tail -2 | head -1
3 packets transmitted, 3 received, 0% packet loss, time 2082ms
miimon 100 means "ask each card every 100 ms whether it has a link". That is the right value for nearly everyone. The tools that create bonds for you (netplan, NetworkManager) usually set it; a hand-written bond or an old script often does not. Always read MII Polling Interval after you build a bond.
There are two ways to detect a failure, and they do not catch the same failures:
miimon checks the LINK: "is a cable plugged in and does the switch port answer?" Fast, cheap, catches cable, card and port failures.
ARP monitor (arp_interval and arp_ip_target) sends real ARP requests to a target address and expects an answer, so it catches failures BEYOND the link: the switch is up but its uplink is dead, a VLAN is not forwarded, the gateway is gone. It cannot be combined with miimon: one or the other.
Here is the case that miimon cannot see. The link of nic1 stays up, but the path behind it is broken (in the lab, nic1's far end was taken out of the switch):
$ ip -br link show nic1 | cut -c1-90
nic1@if4 UP 5e:5d:52:85:06:41 <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP>
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic1
$ ping -c 3 -W 1 10.77.0.2 | tail -2 | head -1
3 packets transmitted, 0 received, 100% packet loss, time 2044ms
Link up, active member selected, nothing gets through. Switch to the ARP monitor, pointing at the gateway or another address that must always answer:
$ ip link set bond0 type bond miimon 0 arp_interval 200 arp_ip_target 10.77.0.2 arp_validate all
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic2
$ ping -c 3 -W 1 10.77.0.2 | tail -2 | head -1
3 packets transmitted, 3 received, 0% packet loss, time 2067ms
$ ip link set bond0 type bond miimon 100 arp_interval 200
Error: ARP monitoring cannot be used with MII monitoring.
Pick a target that is always there (the default gateway is the usual choice, or two of them separated by commas), because if the target goes away the ARP monitor thinks every member has died. Use miimon for ordinary cabling, the ARP monitor when the switch can fail without dropping the link. Two more knobs smooth flapping: updelay and downdelay (milliseconds a link must stay up or down before the bond believes it; use a multiple of miimon). A member that bounces for a moment then does not cause a failover to and fro. (Not run in the lab: updelay and downdelay.)
Pull the cable: a failover test
A bond you have not unplugged is a bond you have not tested. Do this in a maintenance window, or at least on a machine where nobody is working, with a ping running to something beyond the bond. In the lab, with miimon 100, a ping every 0.1 s, and nic1 (the active member) losing its link after one and a half seconds:
$ ping -i 0.1 -c 60 -W 1 10.77.0.2 | tail -2 | head -1 & # in the lab: no packets lost; on real hardware expect a gap of up to the miimon interval
60 packets transmitted, 60 received, 0% packet loss, time 6135ms
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic2
$ grep -A6 "Slave Interface: nic1" /proc/net/bonding/bond0 | grep -E "MII Status|Link Failure"
MII Status: down
Link Failure Count: 1
The driver says it too (journalctl -k or dmesg):
bond0: (slave nic1): link status definitely down, disabling slave
bond0: (slave nic2): making interface the new active one
Plug the cable back. The bond does NOT go back to nic1 by itself: nic2 is working, so the bond leaves it alone (changing the active member needlessly would break connections twice). If one card is better, say the 10 Gbit one next to a 1 Gbit one, name it primary and the bond returns to it when it is healthy again:
(the cable is plugged back in)
$ grep Currently /proc/net/bonding/bond0
Currently Active Slave: nic2
$ ip link set bond0 type bond primary nic1
$ grep -E "Currently|Primary" /proc/net/bonding/bond0
Currently Active Slave: nic1
Primary Slave: nic1 (primary_reselect always)
primary_reselect controls when the primary takes over again: always (the default), better (only if it is faster) or failure (only when the current member fails). Remember the bond sends gratuitous ARPs after a failover (num_grat_arp, default 1), so the switch learns the new port quickly. A switch that does not move the MAC address on a gratuitous ARP is the reason a failover works on paper and not in production: test with the real switch.
LACP (802.3ad): the switch must agree
LACP is the standards-based way to bundle cables. The server and the switch exchange small LACP messages (LACPDUs), agree which ports form one group, and notice if one end stops answering. The mode needs the other end to be configured too: a port-channel on the switch with LACP "active" (or "passive" on one side at least). A bond in 802.3ad mode against a switch with plain ports never forms a group, and a switch with a port-channel against a server in active-backup mode drops frames in confusing ways. Generic switch side, here as a Cisco-style example (not run in the lab; the syntax of your vendor differs but the ideas do not):
interface range GigabitEthernet1/0/1-2
channel-group 1 mode active
interface Port-channel1
switchport mode trunk
The server side, in the lab against a second Linux bond acting as the switch:
$ ip link add bond0 type bond mode 802.3ad miimon 100 lacp_rate fast xmit_hash_policy layer2
$ ip link set nic1 master bond0
$ ip link set nic2 master bond0
$ ip link set bond0 up
$ sed -n '11,23p' /proc/net/bonding/bond0
802.3ad info
LACP active: on
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 5e:5d:52:85:06:41
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 15
Partner Key: 15
Partner Mac Address: 86:da:05:66:7f:0c
What it tells you: Number of ports: 2 means both cables joined the group. A Partner Mac Address of 00:00:00:00:00:00 means nobody answered LACP: the switch side is wrong. Each member section further down has its own aggregator ID; two members with DIFFERENT aggregator IDs did not agree on a group, usually because of different speeds or a mis-wired port.
lacp_rate is how often LACPDUs are sent: slow (every 30 s, the default) or fast (every second). fast notices a silent partner in seconds instead of minutes: use it when both ends support it. A member failure in LACP, as seen in the lab (one cable pulled):
$ grep -E "Number of ports" /proc/net/bonding/bond0 | head -1
Number of ports: 1
$ ping -c2 -W1 10.77.0.2 | tail -2 | head -1
2 packets transmitted, 2 received, 0% packet loss, time 1037ms
(plug it back, and a few seconds later:)
Number of ports: 2
min_links sets how many members must be in the group for the bond to count as up. Zero means "even one is fine". Setting it to 2 makes the bond go down when a member dies, which is what you want when one cable cannot carry the load and you would rather fail over to another path. (Not run in the lab.)
Hashing: why one connection never gets two cables
In 802.3ad and the xor modes the bond decides the outgoing member by hashing parts of each packet. The same connection always has the same hash, so it always uses the same member. That keeps packets of a connection in order, which TCP likes, and it is the reason one download never goes faster than one cable. xmit_hash_policy chooses which fields are hashed:
layer2 source and destination MAC addresses (the default). Between two hosts that is ONE pair of addresses, so ALL traffic between them goes through ONE member, however many connections there are.
layer2+3 adds the IP addresses. Better for many different peers, no help for one peer.
layer3+4 adds the TCP and UDP ports, so different connections between the same two hosts can use different members. The usual choice for servers. (Fragmented traffic and some tunnels fall back to layer 3.)
Measured, with iperf3 sending 1 and then 8 parallel connections to the same peer, and counting bytes transmitted by each member:
nic1 nic2
layer2, 1 connection 0 MB 9707 MB
layer2, 8 connections 0 MB 26288 MB
layer3+4, 1 connection 9280 MB 0 MB
layer3+4, 8 connections 10005 MB 16554 MB
Read the table slowly. With layer2 even eight connections used one cable. With layer3+4 one connection still used one cable, and eight connections used both (not evenly: hashing is statistics, not a distribution plan; with only a few connections a lopsided result is normal). To change the policy on a running bond:
$ ip link set bond0 type bond xmit_hash_policy layer3+4
$ cat /sys/class/net/bond0/bonding/xmit_hash_policy
layer3+4 1
The policy only controls what THIS machine sends. What the switch sends back is decided by the switch's own load balancing setting (often by source and destination MAC or IP by default). Set both ends, or half of the traffic is pinned to one cable.
balance-rr: one connection over both cables
Round robin is the only mode where a single connection can use more than one cable, because it sends packet after packet on alternate members. Both ends were rebuilt in balance-rr for this test:
$ grep "Bonding Mode" /proc/net/bonding/bond0
Bonding Mode: load balancing (round-robin)
$ iperf3 -c 10.77.0.2 -t 3 # ONE connection; the bytes sent by each member:
balance-rr, 1 connection nic1 4775 MB nic2 4775 MB
A perfect split. The catch, which a lab with virtual cables cannot show you (this test had no retransmissions and no reordering): over real cables and real switches the two paths take slightly different times, packets arrive out of order, and TCP sees that as loss. It slows down, retransmits and sometimes ends up slower than a single cable. Watch for it with nstat -az | grep -i reorder and the Retr column in iperf3. That is why round-robin is rare outside a point-to-point cable between two machines (a backup link, a storage link) that both run the same bond mode.
The remaining modes in a line each. balance-xor: the same hashing as 802.3ad, but with no negotiation, so a cable plugged into the wrong port fails silently: prefer 802.3ad. broadcast: sends everything everywhere; for special cases only. balance-tlb and balance-alb: bandwidth without switch support, at the cost of tricks with MAC addresses; they are used in homelabs and on switches that cannot do LACP, and are avoided in the data center. (Not run in the lab: tlb and alb.)
Making it permanent: netplan
Everything above vanishes at reboot. On Ubuntu, netplan writes the permanent version. This is a LACP bond with a default route, DNS and a VLAN on top. The interface names are examples; use yours (ip -br link). The file must not be readable by others (chmod 600):
# /etc/netplan/01-bond.yaml
network:
version: 2
renderer: networkd
ethernets:
eno1: {}
eno2: {}
bonds:
bond0:
interfaces: [eno1, eno2]
addresses: [192.0.2.10/24]
routes:
- to: default
via: 192.0.2.1
nameservers:
addresses: [192.0.2.53]
parameters:
mode: 802.3ad
lacp-rate: fast
mii-monitor-interval: 100
transmit-hash-policy: layer3+4
vlans:
bond0.100:
id: 100
link: bond0
addresses: [198.51.100.10/24]
Under "ethernets" the members are listed with no settings of their own, and ONLY under the bond do they get an address. Check the file without applying it:
$ sudo netplan generate
$ ls /run/systemd/network/
10-netplan-bond0.100.netdev 10-netplan-bond0.100.network 10-netplan-bond0.netdev 10-netplan-bond0.network 10-netplan-eno1.network 10-netplan-eno2.network
(in the lab, with no systemd running, netplan also complained "Failed to connect to bus"; on a real server there is no such line.) The netdev file is what the networking service really builds, and a good way to check you wrote what you meant:
$ cat /run/systemd/network/10-netplan-bond0.netdev
[NetDev]
Name=bond0
Kind=bond
[Bond]
Mode=802.3ad
LACPTransmitRate=fast
MIIMonitorSec=100ms
TransmitHashPolicy=layer3+4
$ cat /run/systemd/network/10-netplan-eno1.network
[Match]
Name=eno1
[Network]
LinkLocalAddressing=no
Bond=bond0
Netplan catches typos before they cost you a connection:
/etc/netplan/02-typo.yaml:19:9: Error in network definition: unknown key 'mii-monitor-intervl'
/etc/netplan/02-typo.yaml:17:15: Error in network definition: unknown bond mode 'lacp'
** (generate:3910): WARNING **: Permissions for /etc/netplan/01-bond.yaml are too open. Netplan configuration should NOT be accessible by others.
The mode names netplan accepts are the kernel's: 802.3ad, active-backup, balance-rr and so on, NOT "lacp". The safe way to apply a change on a remote machine: sudo netplan try. It applies the change and rolls back by itself after 120 seconds unless you confirm, so a broken bond does not lock you out. (Not run in the lab: it needs a running network service.) Then sudo netplan apply, and read /proc/net/bonding/bond0 again.
Making it permanent: NetworkManager
On RHEL, Fedora and any desktop-style install, nmcli does the same thing. A bond connection, two port connections and the address (not run in the lab: it needs the NetworkManager service):
nmcli con add type bond con-name bond0 ifname bond0 bond.options "mode=802.3ad,miimon=100,lacp_rate=fast,xmit_hash_policy=layer3+4"
nmcli con add type ethernet slave-type bond con-name bond0-port1 ifname eno1 master bond0
nmcli con add type ethernet slave-type bond con-name bond0-port2 ifname eno2 master bond0
nmcli con mod bond0 ipv4.addresses 192.0.2.10/24 ipv4.gateway 192.0.2.1 ipv4.dns 192.0.2.53 ipv4.method manual
nmcli con up bond0
nmcli con show --active the bond and both ports should be there
Delete any old connection that still claims eno1 or eno2 (nmcli con show), otherwise it fights the new ones for the card. The settings are the same ones as everywhere: mode, miimon, lacp_rate, xmit_hash_policy. If you have used a distribution's own network scripts (ifcfg files, /etc/network/interfaces with bond-slaves), the same words appear in different clothes. The kernel does the real work, and /proc/net/bonding/bond0 is the same on all of them.
VLANs and bridges on a bond
Treat the bond as an ordinary interface. A VLAN on top of it:
$ ip link add link bond0 name bond0.100 type vlan id 100
$ ip addr add 198.51.100.10/24 dev bond0.100
$ ip link set bond0.100 up
$ ip -br addr show bond0.100
bond0.100@bond0 UP 198.51.100.10/24 fe80::5c5d:52ff:fe85:641/64
$ ip -d link show bond0.100 | grep -oE "vlan protocol 802.1Q id [0-9]+"
vlan protocol 802.1Q id 100
The switch ports behind the bond must carry that VLAN (a trunk). A bridge on top of the bond is how a virtualization host (KVM, containers) shares the cables with its guests. The IP address moves from the bond to the bridge:
$ ip addr flush dev bond0
$ ip link add br0 type bridge
$ ip link set bond0 master br0
$ ip addr add 10.77.0.1/24 dev br0
$ ip link set br0 up
$ ip -br link show master br0 | cut -c1-70
bond0 UP 5e:5d:52:85:06:41 <BROADCAST,MULTICAST
$ ping -c1 -W1 10.77.0.2 | grep received
1 packets transmitted, 1 received, 0% packet loss, time 0ms
Bridges work well on top of active-backup and 802.3ad. They do not work well with balance-alb and balance-tlb, because those modes rewrite MAC addresses.
Changing a live bond, and the errors you will see
Most settings can be changed on the running bond with ip link set bond0 type bond OPTION VALUE, or by writing to /sys/class/net/bond0/bonding/OPTION. The mode cannot, while the bond has members. The messages are clear once you have seen them:
$ ip link set bond0 type bond mode balance-rr
Error: unable to set option because the bond device has slaves.
$ echo 1 > /sys/class/net/bond0/bonding/mode
bash: echo: write error: Directory not empty
$ ip link set bond0 type bond lacp_rate 9
Error: argument "9" is wrong: invalid lacp_rate
$ ip link add bondX type bond mode nosuch
Error: argument "nosuch" is wrong: invalid mode
To change the mode: take the members out (ip link set nic1 nomaster, and nic2), set the mode, put them back. Or rebuild the bond, which is easier to reason about. Do it from the console or a management port, not over the bond you are rebuilding.
THE SILENT ONE. Giving an interface that already belongs to a bond to ANOTHER bond works without a word, and takes it away from the first:
$ ip link add bond9 type bond mode active-backup
$ ip link set nic1 master bond9 ; echo "rc=$?"
rc=0
$ ip -br link show | grep -E "nic|bond0" # nic1 left bond0, which has lost a cable without anyone saying so
nic1@if4 DOWN 5e:5d:52:85:06:41 <BROADCAST,MULTICAST>
Look at /proc/net/bonding/bond0 after any change: it must list every member you expect.
The sysfs interface is the same settings as text files, handy in scripts and for reading everything at once:
$ ls /sys/class/net/bond0/bonding
active_slave ad_actor_key ad_actor_sys_prio ad_actor_system ad_aggregator ad_num_ports ad_partner_key ad_partner_mac ad_select ... mii_status miimon min_links mode primary primary_reselect slaves updelay use_carrier xmit_hash_policy
$ cat /sys/class/net/bonding_masters
bond0
$ cat /sys/class/net/bond0/bonding/slaves
nic1 nic2
Which mode should I use?
one server, one switch, no say over the switch config active-backup. It cannot go wrong in interesting ways.
two switches that are NOT stacked or MLAG'd (or you cannot tell) active-backup, one cable to each switch. Add the ARP monitor if the uplinks can die separately.
a switch you control, LACP supported, wanting bandwidth + failover 802.3ad with xmit_hash_policy layer3+4, lacp_rate fast, miimon 100.
two switches in a proper MLAG / vPC / stack 802.3ad across both. This is what the feature exists for.
hypervisor with a bridge active-backup or 802.3ad. Avoid tlb and alb.
one big transfer between two machines on a cable you own balance-rr on both, and measure it.
a database or storage link that must not lose a single packet broadcast is the unfashionable answer. Think hard about cost.
Common problems
Bond works, but a dead cable is not noticed MII Polling Interval is 0: set miimon 100 (or the ARP monitor)
Two cables, one connection is not faster that is how hashing works; use more connections, layer3+4, or a faster card (or balance-rr, and measure)
All traffic uses one member in 802.3ad xmit_hash_policy layer2 and a single peer (a router or a storage box): switch to layer3+4. Remember the switch's own hash for return traffic
Partner Mac Address is 00:00:00:00:00:00 LACP: no answer from the switch. No port-channel, wrong ports, or the switch is in "on" (static) instead of "active". Check the cabling map
Number of ports: 1 when both cables are in members negotiated different speeds, or one is on a different switch (without MLAG), or the switch group has only one port. Compare Speed and Aggregator ID of each member
Network lost after switching to a bond the IP is still on the old interface, the default route points at it, or the switch ports are not in the right VLAN. From the console: ip -br addr, ip route, cat /proc/net/bonding/bond0
Failover works for ping but TCP sessions break expected for connections that were open; they recover when the application retries. If NEW connections also fail, the switch has not learned the MAC on the new port: check num_grat_arp, port security (sticky MAC) and MAC move limits
A member keeps going up and down Link Failure Count keeps rising: cable, SFP, port. Use updelay/downdelay so one flap does not shake the bond
MAC address changed and DHCP gave a new IP the bond takes the MAC of its FIRST member. Set it on purpose (macaddress in netplan, or ip link set bond0 address ...), or reserve the lease per bond MAC
Switch complains of MAC flapping the same MAC appears on two ports at once: a static port group missing on the switch for balance-rr or xor, or LACP not configured
Bridge and balance-alb misbehave alb rewrites MACs: use active-backup or 802.3ad under bridges
ip link set nicX master bondY "worked", yet bondX lost a member moving a card between bonds is silent. Re-read /proc/net/bonding after every change
netplan apply made the SSH session hang you changed the network under your own connection. Use netplan try, or a console
Link up, no traffic through the active member the failure is behind the link. Use the ARP monitor, with the gateway as target
After reboot the bond is back to old settings the runtime change (ip link set) was never saved to netplan/NetworkManager. Edit the config, then apply
Tips
Choose the goal first (redundancy or bandwidth), then the mode. Redundancy alone: active-backup.
Always set miimon 100, and read MII Polling Interval after the bond exists.
Pull a cable, then the other one, then put them back. Watch /proc/net/bonding/bond0 and a running ping. Do it before the server goes into production, and again after any switch change.
Put the two cables on two different switches whenever you can (active-backup, or MLAG). A bond on one switch survives a cable and not the switch.
Give the IP address to the bond, never to a member. Members carry no address.
For LACP, configure BOTH ends and document which switch ports they are. "Port-channel 7" is not an address a person can find at 3 a.m.
layer3+4 on servers, lacp_rate fast, and tell the network team about both.
Use netplan try (or a console) for changes on a remote machine.
Match the members: same speed, same duplex, same driver. A bond of unequal cards is a puzzle.
Record the bond's expected state (members, mode, Speed) in your monitoring. A bond that has quietly lost one member still passes ping, and that is exactly how it is found out too late.