The mux object can be thought of as a specialized switch which is optimized for many-to-one connectivity. The advantage of using a mux over a switch is that no packet inspection or routing decisions are required. This results in significant latency reductions.
A mux object has many downstream ports which are connected to one upstream port. The upstream port is typically connected to a shared connection - for example, an order line to an exchange. The downstream ports are usually connected to devices that need to share a connection, for example local trading servers.
The supported number of mux objects, upstream ports, and downstream ports varies
depending on the currently running firmware mode (see
Multiple mux objects for more details). The
object requires the
fastmux firmware to be loaded and
A switch license is not required to
mux objects in the
switch firmware mode. A license is only required
switch objects. Refer to Licensing for further
A typical Mux setup, showing a single upstream WAN connection (A1) to be shared amongst three local downstream servers (B5, B9, C5)
Note that traffic cannot be sent between downstream ports within a mux object. In the example depicted above B5 is not able to communicate with B9 or C5.
The primary function of a mux is to forward traffic from downstream ports up to the upstream port. However, most use-cases require a bidirectional connection, which means that traffic must be returned from the upstream to the downstream ports. Mux objects can be configured in several different modes that affect how return traffic is routed from the upstream port to the downstream ports.
layer2mode, traffic that arrives at the upstream port is routed to a downstream port according to the destination MAC address. The upstream port to downstream port latency in this configuration is approximately 107ns. In this mode downstream hosts will only see traffic destined for their MAC address, as well as broadcast traffic.
rawmode, traffic that arrives at the upstream port is broadcast via the layer 1 cross-point device to the downstream ports. This allows return traffic to be routed to hosts in around 5 ns. However, all downstream hosts will see all return data from the upstream connection. Usually the network card on the downstream host will perform this filtering transparently, so
rawmode can provide significant latency improvements. One consideration in the use of
rawmode is network security. In shared setups, where multiple independent parties are sharing an Cisco Nexus 3550-F Fusion (formerly ExaLINK Fusion), it may be undesirable for all local hosts to have visibility into all return data from the upstream port. In such setups,
layer2mode should be used. Note: when using a
layer2forwarding is not supported.
fast-vlanmode can be used on VLAN enabled mux objects to slightly improve the latency by not inspecting the VLAN tag of packets entering a downstream port. Refer to VLAN support for more details.
Note that when raw mode is used, all ports within the mux object operate at the same link speed (e.g. 10Gb/s). This is because all there is no opportunity to do rate conversion at layer 1. The CLI will issue warnings if ports of different speeds are added to a mux object in raw mode.
Configuring a mux object
A mux object must first be created and assigned a name. This must be done from
within the configuration mode, or by prefixing the command with
admin@N3550-F(config)> mux my_mux Mux "my_mux" created
A common next step is to add front panel ports to the mux. Any front panel port can be assigned as the upstream port, but currently only one upstream port can be assigned per mux. As an example, to assign port A1 as an upstream port:
admin@N3550-F(config-mux:my_mux)> port up A1 Added upstream port "A1" to mux "my_mux"
Any number of downstream ports can then be assigned to the mux. To add a downstream port simply omit the 'up' modifier to the port command:
admin@N3550-F(config-mux:my_mux)> port A2 Added downstream port "A2" to mux "my_mux"
Both upstream and downstream ports can be removed from the mux using the
form of the port command. It is recommended to remove unused ports from a mux
when they are no longer required.
admin@N3550-F(config-mux:my_mux)> no port A2 Removed port "A2" from mux "my_mux"
The mode of the mux can be changed between
layer2 (the default setting) and
raw using the mode command:
admin@N3550-F(config-mux:my_mux)> mode raw Set mux "my_mux" to raw mode
The mode can be changed back to
admin@N3550-F(config-mux:my_mux)> mode layer2 Set mux "my_mux" to layer2 mode
mux object using
fastmux firmware can only operate in
To display the current configuration of a mux object use the
admin@N3550-F(config-mux:my_mux)> show Mux name : my_mux Mode : layer2 VLAN tagging : disabled IGMP snooping : disabled Port Side ---- ---------- A1 upstream A2 downstream
The mux object should be deleted when no longer in use.
This is done using the
no form of the mux command:
admin@N3550-F(config)> no mux my_mux Mux "my_mux" deleted
Multiple mux objects
Mux objects can be created when running any of the three standard firmware
fastmux). The supported number of mux objects,
upstream ports, and downstream ports varies depending on the firmware mode.
These differences are summarized in the table below:
|Firmware||Mux Objects||Upstream Ports||Downstream Ports|
|FastMux||4||4||15, 11, 11, 7|
switch firmware, there is no restriction on the number of
mux objects that can be created. The number of downstream ports for each mux
object is also unlimited. For example, a single mux with 47 downstream ports can
be created (48 ports total). Similarly, one can create 6 muxes each with 7
downstream ports (48 ports total).
mux firmware also supports any number of mux objects, however, at most 4
individual upstream ports are supported. Multiple mux objects can use the same
upstream port if the VLAN ID of the upstream port is different. Please refer to
VLAN support for further details. As with the
switch firmware, all mux
mux firmware support an unlimited number of downstream ports.
For example, a single mux with 47 downstream ports can be created (48 ports
total). Similarly one could create 3 muxes each with 15 downstream ports (48
ports total). However, 6 muxes with 6 different upstream ports is not supported
mux firmware mode.
fastmux firmware mode the underlying FPGA firmware has 4 different
fixed sized mux resources in it. This means that the
fastmux firmware can create
up to 4 mux objects each with its own upstream port. The number of downstream
ports supported by each mux object depends on the hardware resource allocated to
it. It may be:
- 15 ports to 1
- 11 ports to 1
- 11 ports to 1
- 7 ports to 1
The management processor will attempt to automatically map mux objects to one of the 4 muxes within the FPGA, depending on the number of ports in the configuration. For example, if a 14:1 mux and a 5:1 mux are created, the software will map the 14:1 mux to the 15:1 FPGA mux resource and the 5:1 mux to the 7:1 FPGA mux resource.
The allocation logic will wait until an upstream port is specified before making a decision on which FPGA mux to use.
If additional ports are added to a mux object, causing it to exceed the number
of ports that the underlying hardware can support, the following error will be
WARNING: Failed to allocate switch resources for port C3. In this
config optimize needs to be run to reallocate resources within the
system. If any mux object has more than 15 downstream ports, this command will
optimize command may cause traffic flows to be momentarily
As an example, assume that a single 5:1 mux is required and there are no other mux objects defined. In this case, the following steps would be taken:
admin@N3550-F> config mux my_mux Created mux "my_mux" WARNING: Mux object "my_mux" disabled: Mux objects must be set to raw mode when using fastmux module function admin@N3550-F(config-mux:my_mux)> mode raw Set mux "my_mux" to raw mode admin@N3550-F(config-mux:my_mux)> port a1-a5 Added downstream port "A1" to mux "my_mux" Added downstream port "A2" to mux "my_mux" Added downstream port "A3" to mux "my_mux" Added downstream port "A4" to mux "my_mux" Added downstream port "A5" to mux "my_mux" admin@N3550-F(config-mux:my_mux)> port up b1 Added upstream port "B1" to mux "my_mux"
This mux has 5 downstream ports so the allocation logic will map this mux to the 7:1 hardware mux resource within the FPGA.
If, at a later stage, an additional 3 ports needs to be added:
admin@N3550-F(config-mux:my_mux)> port c1-c3 Added downstream port "C1" to mux "my_mux" Added downstream port "C2" to mux "my_mux" Added downstream port "C3" to mux "my_mux" WARNING: Failed to allocate switch resources for port C3 WARNING: The "optimize" command may help to resolve allocation failures admin@N3550-F(config-mux:my_mux)>
When the 8th port is added (ie C3), an "allocation" error results.
optimize command can then be issued to instruct the software to swap ports
and objects around to fit the requested config.
admin@N3550-F(config-mux:my_mux)> exit WARNING: Failed to allocate switch resources for port C3 WARNING: The "optimize" command may help to resolve allocation failures admin@N3550-F(config)> optimize This may cause an interruption in network traffic Are you sure? y Optimizing switch resources... Operation completed admin@N3550-F(config)> admin@N3550-F(config)>
Fastmux trade-offs and limitations
Users should not approach
Fastmux as if it works as a general purpose switch. Instead,
use the general model that
Fastmux works more like a direct attach cable, connecting
your downstream devices to the upstream port (and visa versa). This means that
firmware mode makes some trade-offs in management features, in order to achieve the lowest
latency muxing available on the Nexus 3550-F. So features like LLDP, BGP and IGMP are not available
fastmux firmware mode.
Users should also be aware of the following limitations in
- Errors received on a downstream port will be forwarded to the upstream port.
- MACs connected to downstream ports (e.g. from custom FPGA devices) which do not adhere to the Ethernet spec may cause fastmux to transmit errors.
- If a downstream port stops receiving a link signal, the upstream port may temporarily stop transmitting a link signal.
fastmuxobjects on the same Nexus 3550-F via a cable or virtual port is not supported (a separate Nexus 3550-F must be used to chain
This page was last updated on Feb-19-2021.