Lines Matching +full:switch +full:- +full:mode
1 .. SPDX-License-Identifier: GPL-2.0
11 Corrections, HA extensions: 2000/10/03-15:
13 - Willy Tarreau <willy at meta-x.org>
14 - Constantine Gavrilov <const-g at xpert.com>
15 - Chad N. Tindel <ctindel at ieee dot org>
16 - Janice Girouard <girouard at us dot ibm dot com>
17 - Jay Vosburgh <fubar at us dot ibm dot com>
22 - Mitch Williams <mitch.a.williams at intel.com>
29 The behavior of the bonded interfaces depends upon the mode; generally
35 the original tools from extreme-linux and beowulf sites will not work
59 3.7 Configuring LACP for 802.3ad mode in a more secure way
65 5. Switch Configuration
81 10. Promiscuous mode
84 11.1 High Availability in a Single Switch Topology
85 11.2 High Availability in a Multiple Switch Topology
86 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
87 11.2.2 HA Link Monitoring for Multiple Switch Topology
90 12.1 Maximum Throughput in a Single Switch Topology
91 12.1.1 MT Bonding Mode Selection for Single Switch Topology
92 12.1.2 MT Link Monitoring for Single Switch Topology
93 12.2 Maximum Throughput in a Multiple Switch Topology
94 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
95 12.2.2 MT Link Monitoring for Multiple Switch Topology
97 13. Switch Behavior Issues
119 -----------------------------------------------
135 ---------------------------
148 ``/etc/modprobe.d/*.conf`` configuration files, or in a distro-specific
156 configuring a bond, it is recommended "tail -f /var/log/messages" be
166 "mode=802.3ad" and "mode=4" set the same mode.
173 (active-backup, balance-alb and balance-tlb). Possible values
185 the current mode does not use an active slave.
190 is 1 - 65535. If the value is not specified, it takes 65535 as the
193 This parameter has effect only in 802.3ad mode and is available through
198 In an AD system, this specifies the mac-address for the actor in
200 address. If the all-zeroes MAC is specified, bonding will internally
202 local-admin bit set for this mac but driver does not enforce it. If
206 This parameter has effect only in 802.3ad mode and is available through
230 - A slave is added to or removed from the bond
232 - Any slave's link state changes
234 - Any slave's 802.3ad association state changes
236 - The bond's administrative state changes to up
253 In an AD system, the port-key has three parts as shown below -
259 01-05 Speed
260 06-15 User-defined
264 from 0 - 1023. If not given, the system defaults to 0.
266 This parameter has effect only in 802.3ad mode and is available through
288 bonding mode, and the state of the slave). Regular traffic is
295 If ARP monitoring is used in an etherchannel compatible mode
296 (modes 0 and 2), the switch should be configured in a mode
298 switch is configured to distribute the packets in an XOR
330 validated in any mode that supports arp monitoring, or whether
331 non-ARP traffic should be filtered (disregarded) for link
377 active slave. It is possible that some switch or network
389 beyond a common switch. Should the link between the switch and
390 target fail (but not the switch itself), the probe traffic
421 This option affects only active-backup mode for slaves with
445 The default value is 2, and the allowable range is 1 - 255.
458 Specifies whether active-backup mode should set all slaves to
468 bonding to set all slaves of an active-backup bond to
487 vs. just updating a switch or set of switches (which
489 traffic, if the switch snoops incoming traffic to
542 to transmit LACPDU packets in 802.3ad mode. Possible values
575 asserting carrier. It is similar to the Cisco EtherChannel min-links
577 must be up (link-up state) before marking the bond device as up
581 mode.
584 802.3ad mode) whenever there is an active aggregator, regardless of the
589 mode
592 balance-rr (round robin). Possible values are:
594 balance-rr or 0
596 Round-robin policy: Transmit packets in sequential
598 last. This mode provides load balancing and fault
601 active-backup or 1
603 Active-backup policy: Only one slave in the bond is
607 to avoid confusing the switch.
610 occurs in active-backup mode, bonding will issue one
618 This mode provides fault tolerance. The primary
620 mode.
622 balance-xor or 2
631 This mode provides load balancing and fault tolerance.
636 interfaces. This mode provides fault tolerance.
650 regards to the packet mis-ordering requirements of
660 2. A switch that supports IEEE 802.3ad Dynamic link
664 to enable 802.3ad mode.
666 balance-tlb or 5
669 does not require any special switch support.
671 In tlb_dynamic_lb=1 mode; the outgoing traffic is
675 In tlb_dynamic_lb=0 mode; the load balancing based on
688 balance-alb or 6
690 Adaptive load balancing: includes balance-tlb plus
692 does not require any special switch support. The
718 and when an inactive slave is re-activated. The
727 be set to a value equal or greater than the switch's
729 peers will not be blocked by the switch.
753 bonding device and each VLAN sub-device. This is repeated at
757 The valid range is 0 - 255; the default value is 1. These options
758 affect only the active-backup mode. These options were added for
771 The valid range is 0 - 65535; the default value is 1. This option
772 has effect only in balance-rr mode.
782 The valid range is 0 - 300000. The default value is 0, which means
791 for active-backup(1), balance-tlb (5) and balance-alb (6) mode.
801 off-line will alternate devices be used. This is useful when
805 The primary option is only valid for active-backup(1),
806 balance-tlb (5) and balance-alb (6) mode.
813 occurs. This option is designed to prevent flip-flopping between
851 or alb mode. The value has no effect on any other modes.
853 The default behavior of tlb mode is to shuffle active flows across
855 characteristics but can cause packet reordering. If re-ordering is
858 xmit-hash-policy can be used to select the appropriate hashing for
903 balance-xor, 802.3ad, and tlb modes. Possible values are:
936 network peer on the same slave. For non-IP traffic,
969 information is omitted. For non-IP traffic, the
1006 hash to load-balance traffic per-vlan, with failover
1009 use their own vlan, to give lacp-like functionality
1010 without requiring lacp-capable switching hardware.
1027 The valid range is 0 - 255; the default value is 1. A value of 0
1031 This option is useful for bonding modes balance-rr (0), active-backup
1032 (1), balance-tlb (5) and balance-alb (6), in which a failover can
1033 switch the IGMP traffic from one slave to another. Therefore a fresh
1034 IGMP report must be issued to cause the switch to forward the incoming
1042 driver sends learning packets to each slaves peer switch.
1044 The valid range is 1 - 0x7fffffff; the default value is 1. This Option
1045 has effect only in balance-tlb and balance-alb modes.
1073 $ rpm -qf /sbin/ifup
1088 ----------------------------------------
1101 ifcfg-id file for each slave device. The simplest way to accomplish
1103 file ifcfg-id file created; see below for some issues with DHCP). The
1106 ifcfg-id-xx:xx:xx:xx:xx:xx
1111 Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been
1121 _nm_name='bus-pci-0001:61:01.0'
1131 Once the ifcfg-id-xx:xx:xx:xx:xx:xx files have been modified,
1133 itself. This file is named ifcfg-bondX, where X is the number of the
1135 ifcfg-bond0, the second is ifcfg-bond1, and so on. The sysconfig
1139 The contents of the ifcfg-bondX file is as follows::
1149 BONDING_MODULE_OPTS="mode=active-backup miimon=100"
1151 BONDING_SLAVE1="bus-pci-0000:06:08.1"
1180 for the bonding mode, link monitoring, and so on here. Do not include
1190 (bus-pci-0000:06:08.1 in the example above) specify the physical
1221 -------------------------------
1231 -----------------------------------------------
1235 bonding instance to have an appropriately configured ifcfg-bondX file
1239 ifcfg-bondX files.
1242 options in the ifcfg-bondX file, it is not necessary to add them to
1246 ------------------------------------------
1259 network-script file for all physical adapters that will be members of
1262 /etc/sysconfig/network-scripts
1264 The file name must be prefixed with "ifcfg-eth" and suffixed
1266 for eth0 would be named /etc/sysconfig/network-scripts/ifcfg-eth0.
1277 must correspond with the name of the file, i.e., ifcfg-eth1 must have
1285 script will be /etc/sysconfig/network-scripts/ifcfg-bondX where X is
1286 the number of the bond. For bond0 the file is named "ifcfg-bond0",
1287 for bond1 it is named "ifcfg-bond1", and so on. Within that file,
1304 and, indeed, preferable, to specify the bonding options in the ifcfg-bond0
1307 BONDING_OPTS="mode=active-backup arp_interval=60 arp_ip_target=192.168.1.254"
1330 options bond0 mode=balance-alb miimon=100
1340 ---------------------------------
1353 -------------------------------------------------
1357 specifying the appropriate BONDING_OPTS= in ifcfg-bondX where X is the
1365 -----------------------------------------------
1384 modprobe bonding mode=balance-alb miimon=100
1407 enabled without re-running the entire global init script.
1423 -----------------------------------------
1452 options bond0 -o bond0 mode=balance-rr miimon=100
1455 options bond1 -o bond1 mode=balance-alb miimon=50
1458 named "bond0" and creates the bond0 device in balance-rr mode with an
1460 bond1 device in balance-alb mode with an miimon of 50.
1467 install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
1468 mode=balance-alb miimon=50
1474 to rename modules at load time (the "-o bond1" part). Attempts to pass
1482 ------------------------------------------
1502 -----------------------------
1509 # echo -bar > /sys/class/net/bonding_masters
1522 --------------------------
1534 # echo -eth0 > /sys/class/net/bond0/bonding/slaves
1543 # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves
1548 -------------------------------
1552 The names of these files correspond directly with the command-
1561 To configure bond0 for balance-alb mode::
1564 # echo 6 > /sys/class/net/bond0/bonding/mode
1565 - or -
1566 # echo balance-alb > /sys/class/net/bond0/bonding/mode
1570 The bond interface must be down before the mode can be changed.
1579 monitoring is enabled, and vice-versa.
1592 # echo -192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
1601 the bonding driver sends learning packets to each slaves peer switch. The
1605 ---------------------
1616 echo balance-alb > /sys/class/net/bond0/bonding/mode
1623 active-backup mode, using ARP monitoring, add the following lines to
1628 echo active-backup > /sys/class/net/bond1/bonding/mode
1636 -----------------------------------------
1643 the box. The ifenslave-2.6 package should be installed to provide bonding
1644 support. Once installed, this package will provide ``bond-*`` options
1647 Note that ifenslave-2.6 package will load the bonding module and use
1651 ----------------------
1654 active-backup mode, with eth0 and eth1 as slaves::
1658 bond-slaves eth0 eth1
1659 bond-mode active-backup
1660 bond-miimon 100
1661 bond-primary eth0 eth1
1670 bond-slaves none
1671 bond-mode active-backup
1672 bond-miimon 100
1676 bond-master bond0
1677 bond-primary eth0 eth1
1681 bond-master bond0
1682 bond-primary eth0 eth1
1684 For a full list of ``bond-*`` supported options in /etc/network/interfaces and
1686 /usr/share/doc/ifenslave-2.6.
1689 ----------------------------------------------
1694 the selected bonding mode. On occasion however, it is helpful to direct certain
1712 Bonding Mode: fault-tolerance (active-backup)
1763 a pass-through for selecting output queues on the slave device rather than
1767 output slave selection was limited to round-robin and active-backup modes.
1769 3.7 Configuring LACP for 802.3ad mode in a more secure way
1770 ----------------------------------------------------------
1772 When using 802.3ad bonding mode, the Actor (host) and Partner (switch)
1778 domain can spoof LACPDU packets from other hosts to the switch and potentially
1779 cause mayhem by joining (from the point of view of the switch) another
1786 (a) ad_actor_system : You can set a random mac-address that can be used for
1788 Also it's preferable to set the local-admin bit. Following shell code
1789 generates a random mac-address as described above::
1801 is 65535, but system can take the value from 1 - 65535. Following shell
1807 (c) ad_user_port_key : Use the user portion of the port-key. The default
1808 keeps this empty. These are the upper 10 bits of the port-key and value
1809 ranges from 0 - 1023. Following shell code generates these 10 bits and
1820 -------------------------
1822 Each bonding device has a read-only file residing in the
1827 driver is loaded with parameters of mode=0 and miimon=1000 is
1831 Bonding Mode: load balancing (round-robin)
1850 -------------------------
1884 5. Switch Configuration
1887 For this section, "switch" refers to whatever system the
1889 the cable plugs into). This may be an actual dedicated switch device,
1893 The active-backup, balance-tlb and balance-alb modes do not
1894 require any specific configuration of the switch.
1896 The 802.3ad mode requires that the switch have the appropriate
1898 to configure this varies from switch to switch, but, for example, a
1899 Cisco 3550 series switch requires that the appropriate ports first be
1901 etherchannel is set to mode "lacp" to enable 802.3ad (instead of
1904 The balance-rr, balance-xor and broadcast modes generally
1905 require that the switch have the appropriate ports grouped together.
1908 group" or some other similar variation. For these modes, each switch
1909 will also have its own configuration options for the switch's transmit
1912 match. For these three modes, the bonding mode really selects a
1924 packets generated by either ALB mode or the ARP monitor mechanism, are
1936 "un-accelerated" by the bonding driver so the VLAN tag sits in the
1943 would pick up the all-zeroes hardware address. Once the first slave
1964 underlying device -- i.e. the bonding interface -- to promiscuous
1965 mode, which might not be what you want.
1980 -------------------------
1989 ------------------------------------
2012 -------------------------
2041 -------------------------
2076 ----------------------------
2087 options bond0 mode=some-mode miimon=50
2120 ---------------------------------------------------------
2144 beyond other ports of a switch, or if a switch is refusing to pass
2195 10. Promiscuous mode
2199 common to enable promiscuous mode on the device, so that all traffic
2201 The bonding driver handles promiscuous mode changes to the bonding
2205 For the balance-rr, balance-xor, broadcast, and 802.3ad modes,
2206 the promiscuous mode setting is propagated to all slaves.
2208 For the active-backup, balance-tlb and balance-alb modes, the
2209 promiscuous mode setting is propagated only to the active slave.
2211 For balance-tlb mode, the active slave is the slave currently
2214 For balance-alb mode, the active slave is the slave used as a
2215 "primary." This slave is used for mode-specific control traffic, for
2218 For the active-backup, balance-tlb and balance-alb modes, when
2232 11.1 High Availability in a Single Switch Topology
2233 --------------------------------------------------
2235 If two hosts (or a host and a single switch) are directly
2238 only one switch (or peer), so if it fails, there is no alternative
2246 11.2 High Availability in a Multiple Switch Topology
2247 ----------------------------------------------------
2250 network changes dramatically. In multiple switch topologies, there is
2258 +-----+----+ +-----+----+
2260 | switch A +--------------------------+ switch B |
2262 +-----+----+ +-----++---+
2264 | +-------+ |
2265 +-------------+ host1 +---------------+
2266 eth0 +-------+ eth1
2269 switches (ISL, or inter switch link), and multiple ports connecting to
2270 the outside world ("port3" on each switch). There is no technical
2271 reason that this could not be extended to a third switch.
2273 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
2274 -------------------------------------------------------------
2276 In a topology such as the example above, the active-backup and
2281 active-backup:
2282 This is generally the preferred mode, particularly if
2284 network configuration is such that one switch is specifically
2285 a backup switch (e.g., has lower capacity, higher cost, etc),
2290 This mode is really a special purpose mode, and is suitable
2294 necessary for some specific one-way traffic to reach both
2295 independent networks, then the broadcast mode may be suitable.
2297 11.2.2 HA Link Monitoring Selection for Multiple Switch Topology
2298 ----------------------------------------------------------------
2301 switch. If the switch can reliably fail ports in response to other
2306 thus detecting that failure without switch support.
2308 In general, however, in a multiple switch topology, the ARP
2313 one for each switch in the network). This will insure that,
2314 regardless of which switch is active, the ARP monitor has a suitable
2319 switch that causes the link state of a particular switch port to be set
2320 down (or up) when the state of another switch port goes down (or up).
2324 switch, but this can be a viable alternative to the ARP monitor when using
2330 12.1 Maximizing Throughput in a Single Switch Topology
2331 ------------------------------------------------------
2333 In a single switch configuration, the best method to maximize
2342 In a gatewayed configuration, the "switch" is acting primarily
2347 +----------+ +----------+
2349 | Host A +---------------------+ router +------------------->
2350 | +---------------------+ | Hosts B and C are out
2352 +----------+ +----------+
2369 In a local configuration, the "switch" is acting primarily as
2370 a switch, and the majority of traffic passes through this switch to
2374 +----------+ +----------+ +--------+
2375 | |eth0 port1| +-------+ Host B |
2376 | Host A +------------+ switch |port3 +--------+
2377 | +------------+ | +--------+
2378 | |eth1 port2| +------------------+ Host C |
2379 +----------+ +----------+port4 +--------+
2382 Again, the switch may be a dedicated switch device, or another
2398 mode is described below.
2401 12.1.1 MT Bonding Mode Selection for Single Switch Topology
2402 -----------------------------------------------------------
2405 although you will have to decide which bonding mode best suits your
2406 needs. The trade offs for each mode are detailed below:
2408 balance-rr:
2409 This mode is the only mode that will permit a single
2411 interfaces. It is therefore the only mode that will allow a
2426 networking interfaces, the switch, and the topology of the
2435 through the switch to a balance-rr bond will not utilize greater
2440 delivery, then this mode can allow for single stream datagram
2444 This mode requires the switch to have the appropriate ports
2447 active-backup:
2449 the active-backup mode, as the inactive backup devices are all
2451 load balancing mode (with link monitoring) will provide the
2453 available bandwidth. On the plus side, active-backup mode
2454 does not require any configuration of the switch, so it may
2458 balance-xor:
2459 This mode will limit traffic such that packets destined
2462 addresses involved, this mode works best in a "local" network
2464 the same local network. This mode is likely to be suboptimal
2468 As with balance-rr, the switch ports need to be configured for
2472 Like active-backup, there is not much advantage to this
2473 mode in this type of network topology.
2476 This mode can be a good choice for this type of network
2477 topology. The 802.3ad mode is an IEEE standard, so all peers
2480 so minimal manual configuration of the switch is needed
2485 packets. The 802.3ad mode does have some drawbacks: the
2488 balance modes other than balance-rr, no single connection will
2501 Finally, the 802.3ad mode mandates the use of the MII monitor,
2502 therefore, the ARP monitor is not available in this mode.
2504 balance-tlb:
2505 The balance-tlb mode balances outgoing traffic by peer.
2507 "gatewayed" configuration (as described above), this mode will
2509 "local" network configuration, this mode balances multiple
2511 manner (not a simple XOR as in balance-xor or 802.3ad mode),
2517 special switch configuration is required. On the down side,
2518 in this mode all incoming traffic arrives over a single
2519 interface, this mode requires certain ethtool support in the
2523 balance-alb:
2524 This mode is everything that balance-tlb is, and more.
2525 It has all of the features (and restrictions) of balance-tlb,
2530 The only additional down side to this mode is that the network
2534 12.1.2 MT Link Monitoring for Single Switch Topology
2535 ----------------------------------------------------
2538 mode you choose to use. The more advanced load balancing modes do not
2543 12.2 Maximum Throughput in a Multiple Switch Topology
2544 -----------------------------------------------------
2550 +-----------+
2552 +-+---+---+-+
2554 +--------+ | +---------+
2556 +------+---+ +-----+----+ +-----+----+
2557 | Switch A | | Switch B | | Switch C |
2558 +------+---+ +-----+----+ +-----+----+
2560 +--------+ | +---------+
2562 +-+---+---+-+
2564 +-----------+
2570 cost effective than a single larger switch, e.g., on a network with 24
2572 a single 72 port switch.
2578 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
2579 -------------------------------------------------------------
2581 In actual practice, the bonding mode typically employed in
2582 configurations of this type is balance-rr. Historically, in this
2587 packets has arrived). When employed in this fashion, the balance-rr
2588 mode allows individual connections between two hosts to effectively
2591 12.2.2 MT Link Monitoring for Multiple Switch Topology
2592 ------------------------------------------------------
2601 13. Switch Behavior Issues
2605 -------------------------------------------
2608 timing of link up and down reporting by the switch.
2614 during switch initialization (e.g., during recovery after a switch
2621 the switch is initializing. Again, an appropriate updelay value may
2634 In addition to the concerns about switch timings, if your
2635 switches take a long time to go into backup mode, it may be desirable
2640 --------------------------------
2652 For example, on a bond in active-backup mode with five slaves
2653 all connected to one switch, the output may appear as follows::
2655 # ping -n 10.0.4.2
2668 tables. Initially, the switch does not associate the MAC address in
2669 the packet with a particular switch port, and so it may send the
2672 single switch, when the switch (temporarily) floods the traffic to all
2676 The duplicated packet behavior is switch dependent, some
2679 most Cisco switches, the privileged command "clear mac address-table
2690 --------------------
2695 balance-rr, active-backup, balance-tlb and balance-alb modes. This is
2700 --------------------------------
2703 integrated on the planar (that's "motherboard" in IBM-speak). In the
2706 An add-on Broadcom daughter card can be installed on a JS20 to provide
2710 Each I/O Module may contain either a switch or a passthrough
2712 switch). Some bonding modes require a specific BladeCenter internal
2715 Additional BladeCenter-specific networking information can be
2718 - "IBM eServer BladeCenter Networking Options"
2719 - "IBM eServer BladeCenter Layer 2-7 Network Switching"
2722 ------------------------------------
2728 Normally, Ethernet Switch Modules (ESMs) are used in I/O
2735 switch. By using PMs in I/O module #1 and #2, the eth0 and eth1
2737 connected to a common external switch.
2740 appear to bonding as either a single switch topology (all PMs) or as a
2741 multiple switch topology (one or more ESMs, zero or more PMs). It is
2743 much like the example in "High Availability in a Multiple Switch
2747 -------------------------------
2749 The balance-rr mode requires the use of passthrough modules
2750 for devices in the bond, all connected to an common external switch.
2751 That switch must be configured for "etherchannel" or "trunking" on the
2752 appropriate ports, as is usual for balance-rr.
2754 The balance-alb and balance-tlb modes will function with
2755 either switch modules or passthrough modules (or a mix). The only
2761 The active-backup mode has no additional requirements.
2764 ----------------------
2766 When an Ethernet Switch Module is in place, only the ARP
2767 monitor will reliably detect link loss to an external switch. This is
2770 the system, when it fact there is a switch between these "external"
2779 --------------
2787 It may be desirable to disable spanning tree on the switch
2788 (either the internal Ethernet Switch Module, or an external switch) to
2789 avoid fail-over delay issues when using bonding.
2796 -------------------
2802 -----------------------------------------
2804 Any Ethernet type cards (you can even mix cards - a Intel
2809 slaves in active-backup mode.
2812 ----------------------------------------
2817 ----------------------------------------------
2824 ----------------------------------------
2827 disabled. The active-backup mode will fail over to a backup link, and
2830 manner is appropriate for the mode). See the sections on High
2831 Availability and the documentation for each mode for additional
2844 depends upon the bonding mode and network configuration.
2847 ----------------------------------------------
2852 ---------------------------------------------
2854 The full answer to this depends upon the desired mode.
2856 In the basic balance modes (balance-rr and balance-xor), it
2861 The advanced balance modes (balance-tlb and balance-alb) do
2862 not have special switch requirements, but do need device drivers that
2866 In 802.3ad mode, it works with systems that support IEEE
2870 The active-backup mode should work with any Layer-II switch.
2873 ---------------------------------------------------------
2895 # ifconfig bond0 down ; modprobe -r bonding
2903 from the bond (``ifenslave -d bond0 eth0``). The bonding driver will
2925 http://vger.kernel.org/vger-lists.html#netdev