Mirror and timestamping - Cisco Nexus 3550-F Fusion
This page describes the operation of mirror objects for the Cisco Nexus 3550-F Fusion (formerly ExaLINK Fusion). For details on the operation of mirror objects for the Cisco Nexus 3550-F Fusion HPT (formerly ExaLINK Fusion HPT), please refer to this page.
The mirror configuration provides a mechanism for configuring the logging and timestamping functionality within the Nexus 3550-F. A mirror replicates any data traversing its member objects out of another port. Ports, mux and switch objects can be added to a mirror to replicate traffic that traverses them.
The mirror can also be configured to enable timestamping. All packets that enter the Nexus 3550-F are timestamped on arrival at the ingress port. This timestamp is then inserted into the packet body when packets arrive at the mirror egress port.
Example use of a mirror, showing port A1 and port B3 mirrored to output port C3.
Creating a mirror
Port mirrors can be created after specifying a name, completed in either
configuration mode, or prefixing the command with the
admin@N3550-F(config)> mirror mymirror Mirror "mymirror" created
Creating a mirror will place the command line into a config-mirror state.
Within this state an output port for the mirror can be assigned using the
output command. The output port can be any of the available front panel
admin@N3550-F(config-mirror:mymirror)> output C3 Added output port "C3" to mirror "mymirror"
Note: It is not possible to use a port that is already in use for the output port.
Note: There are only four timestamping/mux outputs available in the FPGA. These can be assigned arbitrarily to front-panel ports. Adding an output port to a mirror will consume one of these outputs. If you need a lot of outputs from a mirror, it's worth considering a mirror with only one output port, and using a tap object from that output port to the other ports.
The mirror will replicate the traffic of any of its member objects. To assign
an object to the mirror use the object type followed by the name. Objects that
can be mirrored are
switch objects. For example, to replicate
all traffic received on ports A1 and B3:
admin@N3550-F(config-mirror:mymirror)> port A1; port B3 Added port "A1" to mirror "mymirror" Added port "B3" to mirror "mymirror"
In addition to replicating specific ports, it is also possible to replicate all
traffic received by a given mux or switch object. For example, to
replicate all of the traffic passing through an existing mux object, use the
admin@N3550-F(config-mirror:mymirror)> mux mymux Added mux "mymux" to mirror "mymirror"
Note: It is not possible to explicitly mirror outputs to a port using a mirror object, only inputs to ports or mux/switch objects. In order to take a copy of all traffic leaving a port, it is suggested to use a tap object and tap the output port.
Note: Mirroring is not supported when using
To remove an object from the mirror, use the
no form of the commands:
admin@N3550-F(config-mirror:mymirror)> no mux mymux Removed mux "mymux" from mirror "mymirror" admin@N3550-F(config-mirror:mymirror)> no port A1 Removed port "A1" from mirror "mymirror"
Removing a mirror
When a port mirror is no longer needed it is possible to remove the mirror.
no form of the mirror command:
admin@N3550-F> config no mirror mymirror Mirror "mymirror" deleted
The Nexus 3550-F timestamps packets upon arrival using a 350MHz 64 bit
timestamp counter. Mirror output ports can be configured to transmit the
timestamp along with packet in a number of different modes using the
timestamp command. Timestamps are taken at the moment the first bit of the
incoming packet arrives.
A utility that allows reception and decoding of these timestamped streams is available on GitHub. This can be used to write out a pcap file with nanosecond accurate timestamps.
In order to replace the Ethernet FCS (Frame Check Sequence) with the lower 32
bits of the timestamp counter, the timestamp mode should be set to
admin@N3550-F> config mirror mymirror timestamp fcs Set timestamp mode "fcs" on mirror "mymirror" admin@N3550-F> config mirror mymirror no timestamp Timestamping disabled on "mymirror"
Each increment of the 32 bit counter represents approximately 2.86 ns of time. The lower 32 bits of this counter wrap every 12.3 seconds. To allow the capturing host to reconstruct the absolute UTC (Universal Time Coordinated) time of each packet, the mirror port will send periodic keyframes that are not part of the capture stream. These keyframes contain the full 64 bit counter and the UTC time in nanoseconds associated with the counter value.
These keyframes and FCS timestamps allow the remote host to reconstruct the absolute time of arrival of all packets in the captured feed.
The Nexus 3550-F also supports an FCS Compatibility mode, whereby instead of replacing
the FCS with the lower 32 bits of the timestamp counter the FCS is replaced with a
timestamp format compatible with Arista devices and systems or applications designed
to recognize and process the timestamp format provided by Arista. Similar to the FCS mode, the
command can be used again:
admin@N3550-F> config mirror mymirror timestamp fcs-compat Set timestamp mode "fcs-compat" on mirror "mymirror"
In this format the timestamp counter's lower 31 bits are written over the FCS, with a gap found in the highest order bit of the lower order byte is a zeroed padding (i.e. 31 bits of data in a 32-bit format). Note that even in FCS Compatibility mode, timestamps are taken immediately upon packet arrival at an ingress port, and not upon arrival of the FCS field.
For instances where it is desirable for the timestamped frame to include a valid FCS, another two modes are supported where the timestamp is inserted in place of the original FCS bytes, and a new FCS for the whole packet is calculated and transmitted, therefore increasing the size of each transmitted frame by 4 bytes. Again, the selection between standard and compat modes are available:
admin@N3550-F> config mirror mymirror timestamp append Set timestamp mode "append" on mirror "mymirror" admin@N3550-F> config mirror mymirror timestamp append-compat Set timestamp mode "append-compat" on mirror "mymirror"
When a mirror object is configured to append a new FCS, the traffic path through the mirror changes from "cut through" to "store and forward". This does not impact the latency of traffic flow for mux/switch objects (or indeed any layer 1 paths).
Accurate time synchronization of the Nexus 3550-F is important when using a mirror with timestamping enabled. Synchronization allows the Nexus 3550-F to provide keyframes that accurately correlate the internal timestamp counter with absolute "wall clock" time. Refer to System Time for details on setting and maintaining accurate time synchronization.
This page was last updated on Feb-19-2021.