Lines Matching +full:switch +full:- +full:mode
1 .. SPDX-License-Identifier: GPL-2.0
11 Corrections, HA extensions: 2000/10/03-15:
13 - Willy Tarreau <willy at meta-x.org>
14 - Constantine Gavrilov <const-g at xpert.com>
15 - Chad N. Tindel <ctindel at ieee dot org>
16 - Janice Girouard <girouard at us dot ibm dot com>
17 - Jay Vosburgh <fubar at us dot ibm dot com>
22 - Mitch Williams <mitch.a.williams at intel.com>
29 The behavior of the bonded interfaces depends upon the mode; generally
35 the original tools from extreme-linux and beowulf sites will not work
59 3.7 Configuring LACP for 802.3ad mode in a more secure way
65 5. Switch Configuration
81 10. Promiscuous mode
84 11.1 High Availability in a Single Switch Topology
85 11.2 High Availability in a Multiple Switch Topology
86 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
87 11.2.2 HA Link Monitoring for Multiple Switch Topology
90 12.1 Maximum Throughput in a Single Switch Topology
91 12.1.1 MT Bonding Mode Selection for Single Switch Topology
92 12.1.2 MT Link Monitoring for Single Switch Topology
93 12.2 Maximum Throughput in a Multiple Switch Topology
94 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
95 12.2.2 MT Link Monitoring for Multiple Switch Topology
97 13. Switch Behavior Issues
119 -----------------------------------------------
135 ---------------------------
148 ``/etc/modprobe.d/*.conf`` configuration files, or in a distro-specific
156 configuring a bond, it is recommended "tail -f /var/log/messages" be
166 "mode=802.3ad" and "mode=4" set the same mode.
173 (active-backup, balance-alb and balance-tlb). Possible values
185 the current mode does not use an active slave.
190 is 1 - 65535. If the value is not specified, it takes 65535 as the
193 This parameter has effect only in 802.3ad mode and is available through
198 In an AD system, this specifies the mac-address for the actor in
200 address. If the all-zeroes MAC is specified, bonding will internally
202 local-admin bit set for this mac but driver does not enforce it. If
206 This parameter has effect only in 802.3ad mode and is available through
230 - A slave is added to or removed from the bond
232 - Any slave's link state changes
234 - Any slave's 802.3ad association state changes
236 - The bond's administrative state changes to up
253 In an AD system, the port-key has three parts as shown below -
259 01-05 Speed
260 06-15 User-defined
264 from 0 - 1023. If not given, the system defaults to 0.
266 This parameter has effect only in 802.3ad mode and is available through
288 bonding mode, and the state of the slave). Regular traffic is
295 If ARP monitoring is used in an etherchannel compatible mode
296 (modes 0 and 2), the switch should be configured in a mode
298 switch is configured to distribute the packets in an XOR
330 validated in any mode that supports arp monitoring, or whether
331 non-ARP traffic should be filtered (disregarded) for link
377 active slave. It is possible that some switch or network
389 beyond a common switch. Should the link between the switch and
390 target fail (but not the switch itself), the probe traffic
421 This option affects only active-backup mode for slaves with
445 The default value is 2, and the allowable range is 1 - 255.
449 Specifies whether the LACP state machine's MUX in the 802.3ad mode
453 IEEE 802.1AX-2008 5.4.15 in addition to the existing coupled control
470 Specifies whether active-backup mode should set all slaves to
480 bonding to set all slaves of an active-backup bond to
499 vs. just updating a switch or set of switches (which
501 traffic, if the switch snoops incoming traffic to
554 to transmit LACPDU packets in 802.3ad mode. Possible values
587 asserting carrier. It is similar to the Cisco EtherChannel min-links
589 must be up (link-up state) before marking the bond device as up
593 mode.
596 802.3ad mode) whenever there is an active aggregator, regardless of the
601 mode
604 balance-rr (round robin). Possible values are:
606 balance-rr or 0
608 Round-robin policy: Transmit packets in sequential
610 last. This mode provides load balancing and fault
613 active-backup or 1
615 Active-backup policy: Only one slave in the bond is
619 to avoid confusing the switch.
622 occurs in active-backup mode, bonding will issue one
630 This mode provides fault tolerance. The primary
632 mode.
634 balance-xor or 2
643 This mode provides load balancing and fault tolerance.
648 interfaces. This mode provides fault tolerance.
662 regards to the packet mis-ordering requirements of
672 2. A switch that supports IEEE 802.3ad Dynamic link
676 to enable 802.3ad mode.
678 balance-tlb or 5
681 does not require any special switch support.
683 In tlb_dynamic_lb=1 mode; the outgoing traffic is
687 In tlb_dynamic_lb=0 mode; the load balancing based on
700 balance-alb or 6
702 Adaptive load balancing: includes balance-tlb plus
704 does not require any special switch support. The
730 and when an inactive slave is re-activated. The
739 be set to a value equal or greater than the switch's
741 peers will not be blocked by the switch.
765 bonding device and each VLAN sub-device. This is repeated at
769 The valid range is 0 - 255; the default value is 1. These options
770 affect only the active-backup mode. These options were added for
783 The valid range is 0 - 65535; the default value is 1. This option
784 has effect only in balance-rr mode.
794 The valid range is 0 - 300000. The default value is 0, which means
803 for active-backup(1), balance-tlb (5) and balance-alb (6) mode.
813 off-line will alternate devices be used. This is useful when
817 The primary option is only valid for active-backup(1),
818 balance-tlb (5) and balance-alb (6) mode.
825 occurs. This option is designed to prevent flip-flopping between
863 or alb mode. The value has no effect on any other modes.
865 The default behavior of tlb mode is to shuffle active flows across
867 characteristics but can cause packet reordering. If re-ordering is
870 xmit-hash-policy can be used to select the appropriate hashing for
915 balance-xor, 802.3ad, and tlb modes. Possible values are:
948 network peer on the same slave. For non-IP traffic,
981 information is omitted. For non-IP traffic, the
1018 hash to load-balance traffic per-vlan, with failover
1021 use their own vlan, to give lacp-like functionality
1022 without requiring lacp-capable switching hardware.
1039 The valid range is 0 - 255; the default value is 1. A value of 0
1043 This option is useful for bonding modes balance-rr (0), active-backup
1044 (1), balance-tlb (5) and balance-alb (6), in which a failover can
1045 switch the IGMP traffic from one slave to another. Therefore a fresh
1046 IGMP report must be issued to cause the switch to forward the incoming
1054 driver sends learning packets to each slaves peer switch.
1056 The valid range is 1 - 0x7fffffff; the default value is 1. This Option
1057 has effect only in balance-tlb and balance-alb modes.
1085 $ rpm -qf /sbin/ifup
1100 ----------------------------------------
1113 ifcfg-id file for each slave device. The simplest way to accomplish
1115 file ifcfg-id file created; see below for some issues with DHCP). The
1118 ifcfg-id-xx:xx:xx:xx:xx:xx
1123 Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been
1133 _nm_name='bus-pci-0001:61:01.0'
1143 Once the ifcfg-id-xx:xx:xx:xx:xx:xx files have been modified,
1145 itself. This file is named ifcfg-bondX, where X is the number of the
1147 ifcfg-bond0, the second is ifcfg-bond1, and so on. The sysconfig
1151 The contents of the ifcfg-bondX file is as follows::
1161 BONDING_MODULE_OPTS="mode=active-backup miimon=100"
1163 BONDING_SLAVE1="bus-pci-0000:06:08.1"
1192 for the bonding mode, link monitoring, and so on here. Do not include
1202 (bus-pci-0000:06:08.1 in the example above) specify the physical
1233 -------------------------------
1243 -----------------------------------------------
1247 bonding instance to have an appropriately configured ifcfg-bondX file
1251 ifcfg-bondX files.
1254 options in the ifcfg-bondX file, it is not necessary to add them to
1258 ------------------------------------------
1271 network-script file for all physical adapters that will be members of
1274 /etc/sysconfig/network-scripts
1276 The file name must be prefixed with "ifcfg-eth" and suffixed
1278 for eth0 would be named /etc/sysconfig/network-scripts/ifcfg-eth0.
1289 must correspond with the name of the file, i.e., ifcfg-eth1 must have
1297 script will be /etc/sysconfig/network-scripts/ifcfg-bondX where X is
1298 the number of the bond. For bond0 the file is named "ifcfg-bond0",
1299 for bond1 it is named "ifcfg-bond1", and so on. Within that file,
1316 and, indeed, preferable, to specify the bonding options in the ifcfg-bond0
1319 BONDING_OPTS="mode=active-backup arp_interval=60 arp_ip_target=192.168.1.254"
1342 options bond0 mode=balance-alb miimon=100
1352 ---------------------------------
1365 -------------------------------------------------
1369 specifying the appropriate BONDING_OPTS= in ifcfg-bondX where X is the
1377 -----------------------------------------------
1396 modprobe bonding mode=balance-alb miimon=100
1419 enabled without re-running the entire global init script.
1435 -----------------------------------------
1464 options bond0 -o bond0 mode=balance-rr miimon=100
1467 options bond1 -o bond1 mode=balance-alb miimon=50
1470 named "bond0" and creates the bond0 device in balance-rr mode with an
1472 bond1 device in balance-alb mode with an miimon of 50.
1479 install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
1480 mode=balance-alb miimon=50
1486 to rename modules at load time (the "-o bond1" part). Attempts to pass
1494 ------------------------------------------
1514 -----------------------------
1521 # echo -bar > /sys/class/net/bonding_masters
1534 --------------------------
1546 # echo -eth0 > /sys/class/net/bond0/bonding/slaves
1555 # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves
1560 -------------------------------
1564 The names of these files correspond directly with the command-
1573 To configure bond0 for balance-alb mode::
1576 # echo 6 > /sys/class/net/bond0/bonding/mode
1577 - or -
1578 # echo balance-alb > /sys/class/net/bond0/bonding/mode
1582 The bond interface must be down before the mode can be changed.
1591 monitoring is enabled, and vice-versa.
1604 # echo -192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
1613 the bonding driver sends learning packets to each slaves peer switch. The
1617 ---------------------
1628 echo balance-alb > /sys/class/net/bond0/bonding/mode
1635 active-backup mode, using ARP monitoring, add the following lines to
1640 echo active-backup > /sys/class/net/bond1/bonding/mode
1648 -----------------------------------------
1655 the box. The ifenslave-2.6 package should be installed to provide bonding
1656 support. Once installed, this package will provide ``bond-*`` options
1659 Note that ifenslave-2.6 package will load the bonding module and use
1663 ----------------------
1666 active-backup mode, with eth0 and eth1 as slaves::
1670 bond-slaves eth0 eth1
1671 bond-mode active-backup
1672 bond-miimon 100
1673 bond-primary eth0 eth1
1682 bond-slaves none
1683 bond-mode active-backup
1684 bond-miimon 100
1688 bond-master bond0
1689 bond-primary eth0 eth1
1693 bond-master bond0
1694 bond-primary eth0 eth1
1696 For a full list of ``bond-*`` supported options in /etc/network/interfaces and
1698 /usr/share/doc/ifenslave-2.6.
1701 ----------------------------------------------
1706 the selected bonding mode. On occasion however, it is helpful to direct certain
1724 Bonding Mode: fault-tolerance (active-backup)
1775 a pass-through for selecting output queues on the slave device rather than
1779 output slave selection was limited to round-robin and active-backup modes.
1781 3.7 Configuring LACP for 802.3ad mode in a more secure way
1782 ----------------------------------------------------------
1784 When using 802.3ad bonding mode, the Actor (host) and Partner (switch)
1790 domain can spoof LACPDU packets from other hosts to the switch and potentially
1791 cause mayhem by joining (from the point of view of the switch) another
1798 (a) ad_actor_system : You can set a random mac-address that can be used for
1800 Also it's preferable to set the local-admin bit. Following shell code
1801 generates a random mac-address as described above::
1813 is 65535, but system can take the value from 1 - 65535. Following shell
1819 (c) ad_user_port_key : Use the user portion of the port-key. The default
1820 keeps this empty. These are the upper 10 bits of the port-key and value
1821 ranges from 0 - 1023. Following shell code generates these 10 bits and
1832 -------------------------
1834 Each bonding device has a read-only file residing in the
1839 driver is loaded with parameters of mode=0 and miimon=1000 is
1843 Bonding Mode: load balancing (round-robin)
1862 -------------------------
1896 5. Switch Configuration
1899 For this section, "switch" refers to whatever system the
1901 the cable plugs into). This may be an actual dedicated switch device,
1905 The active-backup, balance-tlb and balance-alb modes do not
1906 require any specific configuration of the switch.
1908 The 802.3ad mode requires that the switch have the appropriate
1910 to configure this varies from switch to switch, but, for example, a
1911 Cisco 3550 series switch requires that the appropriate ports first be
1913 etherchannel is set to mode "lacp" to enable 802.3ad (instead of
1916 The balance-rr, balance-xor and broadcast modes generally
1917 require that the switch have the appropriate ports grouped together.
1920 group" or some other similar variation. For these modes, each switch
1921 will also have its own configuration options for the switch's transmit
1924 match. For these three modes, the bonding mode really selects a
1936 packets generated by either ALB mode or the ARP monitor mechanism, are
1948 "un-accelerated" by the bonding driver so the VLAN tag sits in the
1955 would pick up the all-zeroes hardware address. Once the first slave
1976 underlying device -- i.e. the bonding interface -- to promiscuous
1977 mode, which might not be what you want.
1992 -------------------------
2001 ------------------------------------
2024 -------------------------
2053 -------------------------
2088 ----------------------------
2099 options bond0 mode=some-mode miimon=50
2132 ---------------------------------------------------------
2156 beyond other ports of a switch, or if a switch is refusing to pass
2207 10. Promiscuous mode
2211 common to enable promiscuous mode on the device, so that all traffic
2213 The bonding driver handles promiscuous mode changes to the bonding
2217 For the balance-rr, balance-xor, broadcast, and 802.3ad modes,
2218 the promiscuous mode setting is propagated to all slaves.
2220 For the active-backup, balance-tlb and balance-alb modes, the
2221 promiscuous mode setting is propagated only to the active slave.
2223 For balance-tlb mode, the active slave is the slave currently
2226 For balance-alb mode, the active slave is the slave used as a
2227 "primary." This slave is used for mode-specific control traffic, for
2230 For the active-backup, balance-tlb and balance-alb modes, when
2244 11.1 High Availability in a Single Switch Topology
2245 --------------------------------------------------
2247 If two hosts (or a host and a single switch) are directly
2250 only one switch (or peer), so if it fails, there is no alternative
2258 11.2 High Availability in a Multiple Switch Topology
2259 ----------------------------------------------------
2262 network changes dramatically. In multiple switch topologies, there is
2270 +-----+----+ +-----+----+
2272 | switch A +--------------------------+ switch B |
2274 +-----+----+ +-----++---+
2276 | +-------+ |
2277 +-------------+ host1 +---------------+
2278 eth0 +-------+ eth1
2281 switches (ISL, or inter switch link), and multiple ports connecting to
2282 the outside world ("port3" on each switch). There is no technical
2283 reason that this could not be extended to a third switch.
2285 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
2286 -------------------------------------------------------------
2288 In a topology such as the example above, the active-backup and
2293 active-backup:
2294 This is generally the preferred mode, particularly if
2296 network configuration is such that one switch is specifically
2297 a backup switch (e.g., has lower capacity, higher cost, etc),
2302 This mode is really a special purpose mode, and is suitable
2306 necessary for some specific one-way traffic to reach both
2307 independent networks, then the broadcast mode may be suitable.
2309 11.2.2 HA Link Monitoring Selection for Multiple Switch Topology
2310 ----------------------------------------------------------------
2313 switch. If the switch can reliably fail ports in response to other
2318 thus detecting that failure without switch support.
2320 In general, however, in a multiple switch topology, the ARP
2325 one for each switch in the network). This will insure that,
2326 regardless of which switch is active, the ARP monitor has a suitable
2331 switch that causes the link state of a particular switch port to be set
2332 down (or up) when the state of another switch port goes down (or up).
2336 switch, but this can be a viable alternative to the ARP monitor when using
2342 12.1 Maximizing Throughput in a Single Switch Topology
2343 ------------------------------------------------------
2345 In a single switch configuration, the best method to maximize
2354 In a gatewayed configuration, the "switch" is acting primarily
2359 +----------+ +----------+
2361 | Host A +---------------------+ router +------------------->
2362 | +---------------------+ | Hosts B and C are out
2364 +----------+ +----------+
2381 In a local configuration, the "switch" is acting primarily as
2382 a switch, and the majority of traffic passes through this switch to
2386 +----------+ +----------+ +--------+
2387 | |eth0 port1| +-------+ Host B |
2388 | Host A +------------+ switch |port3 +--------+
2389 | +------------+ | +--------+
2390 | |eth1 port2| +------------------+ Host C |
2391 +----------+ +----------+port4 +--------+
2394 Again, the switch may be a dedicated switch device, or another
2410 mode is described below.
2413 12.1.1 MT Bonding Mode Selection for Single Switch Topology
2414 -----------------------------------------------------------
2417 although you will have to decide which bonding mode best suits your
2418 needs. The trade offs for each mode are detailed below:
2420 balance-rr:
2421 This mode is the only mode that will permit a single
2423 interfaces. It is therefore the only mode that will allow a
2438 networking interfaces, the switch, and the topology of the
2447 through the switch to a balance-rr bond will not utilize greater
2452 delivery, then this mode can allow for single stream datagram
2456 This mode requires the switch to have the appropriate ports
2459 active-backup:
2461 the active-backup mode, as the inactive backup devices are all
2463 load balancing mode (with link monitoring) will provide the
2465 available bandwidth. On the plus side, active-backup mode
2466 does not require any configuration of the switch, so it may
2470 balance-xor:
2471 This mode will limit traffic such that packets destined
2474 addresses involved, this mode works best in a "local" network
2476 the same local network. This mode is likely to be suboptimal
2480 As with balance-rr, the switch ports need to be configured for
2484 Like active-backup, there is not much advantage to this
2485 mode in this type of network topology.
2488 This mode can be a good choice for this type of network
2489 topology. The 802.3ad mode is an IEEE standard, so all peers
2492 so minimal manual configuration of the switch is needed
2497 packets. The 802.3ad mode does have some drawbacks: the
2500 balance modes other than balance-rr, no single connection will
2513 Finally, the 802.3ad mode mandates the use of the MII monitor,
2514 therefore, the ARP monitor is not available in this mode.
2516 balance-tlb:
2517 The balance-tlb mode balances outgoing traffic by peer.
2519 "gatewayed" configuration (as described above), this mode will
2521 "local" network configuration, this mode balances multiple
2523 manner (not a simple XOR as in balance-xor or 802.3ad mode),
2529 special switch configuration is required. On the down side,
2530 in this mode all incoming traffic arrives over a single
2531 interface, this mode requires certain ethtool support in the
2535 balance-alb:
2536 This mode is everything that balance-tlb is, and more.
2537 It has all of the features (and restrictions) of balance-tlb,
2542 The only additional down side to this mode is that the network
2546 12.1.2 MT Link Monitoring for Single Switch Topology
2547 ----------------------------------------------------
2550 mode you choose to use. The more advanced load balancing modes do not
2555 12.2 Maximum Throughput in a Multiple Switch Topology
2556 -----------------------------------------------------
2562 +-----------+
2564 +-+---+---+-+
2566 +--------+ | +---------+
2568 +------+---+ +-----+----+ +-----+----+
2569 | Switch A | | Switch B | | Switch C |
2570 +------+---+ +-----+----+ +-----+----+
2572 +--------+ | +---------+
2574 +-+---+---+-+
2576 +-----------+
2582 cost effective than a single larger switch, e.g., on a network with 24
2584 a single 72 port switch.
2590 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
2591 -------------------------------------------------------------
2593 In actual practice, the bonding mode typically employed in
2594 configurations of this type is balance-rr. Historically, in this
2599 packets has arrived). When employed in this fashion, the balance-rr
2600 mode allows individual connections between two hosts to effectively
2603 12.2.2 MT Link Monitoring for Multiple Switch Topology
2604 ------------------------------------------------------
2613 13. Switch Behavior Issues
2617 -------------------------------------------
2620 timing of link up and down reporting by the switch.
2626 during switch initialization (e.g., during recovery after a switch
2633 the switch is initializing. Again, an appropriate updelay value may
2646 In addition to the concerns about switch timings, if your
2647 switches take a long time to go into backup mode, it may be desirable
2652 --------------------------------
2664 For example, on a bond in active-backup mode with five slaves
2665 all connected to one switch, the output may appear as follows::
2667 # ping -n 10.0.4.2
2680 tables. Initially, the switch does not associate the MAC address in
2681 the packet with a particular switch port, and so it may send the
2684 single switch, when the switch (temporarily) floods the traffic to all
2688 The duplicated packet behavior is switch dependent, some
2691 most Cisco switches, the privileged command "clear mac address-table
2702 --------------------
2707 balance-rr, active-backup, balance-tlb and balance-alb modes. This is
2712 --------------------------------
2715 integrated on the planar (that's "motherboard" in IBM-speak). In the
2718 An add-on Broadcom daughter card can be installed on a JS20 to provide
2722 Each I/O Module may contain either a switch or a passthrough
2724 switch). Some bonding modes require a specific BladeCenter internal
2727 Additional BladeCenter-specific networking information can be
2730 - "IBM eServer BladeCenter Networking Options"
2731 - "IBM eServer BladeCenter Layer 2-7 Network Switching"
2734 ------------------------------------
2740 Normally, Ethernet Switch Modules (ESMs) are used in I/O
2747 switch. By using PMs in I/O module #1 and #2, the eth0 and eth1
2749 connected to a common external switch.
2752 appear to bonding as either a single switch topology (all PMs) or as a
2753 multiple switch topology (one or more ESMs, zero or more PMs). It is
2755 much like the example in "High Availability in a Multiple Switch
2759 -------------------------------
2761 The balance-rr mode requires the use of passthrough modules
2762 for devices in the bond, all connected to an common external switch.
2763 That switch must be configured for "etherchannel" or "trunking" on the
2764 appropriate ports, as is usual for balance-rr.
2766 The balance-alb and balance-tlb modes will function with
2767 either switch modules or passthrough modules (or a mix). The only
2773 The active-backup mode has no additional requirements.
2776 ----------------------
2778 When an Ethernet Switch Module is in place, only the ARP
2779 monitor will reliably detect link loss to an external switch. This is
2782 the system, when it fact there is a switch between these "external"
2791 --------------
2799 It may be desirable to disable spanning tree on the switch
2800 (either the internal Ethernet Switch Module, or an external switch) to
2801 avoid fail-over delay issues when using bonding.
2808 -------------------
2814 -----------------------------------------
2816 Any Ethernet type cards (you can even mix cards - a Intel
2821 slaves in active-backup mode.
2824 ----------------------------------------
2829 ----------------------------------------------
2836 ----------------------------------------
2839 disabled. The active-backup mode will fail over to a backup link, and
2842 manner is appropriate for the mode). See the sections on High
2843 Availability and the documentation for each mode for additional
2856 depends upon the bonding mode and network configuration.
2859 ----------------------------------------------
2864 ---------------------------------------------
2866 The full answer to this depends upon the desired mode.
2868 In the basic balance modes (balance-rr and balance-xor), it
2873 The advanced balance modes (balance-tlb and balance-alb) do
2874 not have special switch requirements, but do need device drivers that
2878 In 802.3ad mode, it works with systems that support IEEE
2882 The active-backup mode should work with any Layer-II switch.
2885 ---------------------------------------------------------
2907 # ifconfig bond0 down ; modprobe -r bonding
2915 from the bond (``ifenslave -d bond0 eth0``). The bonding driver will
2937 http://vger.kernel.org/vger-lists.html#netdev