xref: /freebsd-10.2-release/contrib/ofed/management/doc/ibtracer.txt

ibtracer
1/11/05

Description:
ibtracer is used to build a source route into a UD packet and validate the
path taken to a destination. It is based on a client/server architecture and
relies on a special ibtracer IB agent in each node along the way. It can
deal with switches which do not currently run this agent but validation for
that part of the path is impossible.

Syntax:
ibtracer [-I mthca0] [-p port] [-r <# retries>] [-t <timeout in msec>] \
         [-l LID] [-g DGID] [-v]

Architecture:
IBA 1.2 defines a new set of vendor specific MADs which include OUI. OpenIB
will use one of these classes (0x30) to implement ibtracer (and the vendor
MAD option of ibping).

Note that these are general service MADs and rely on the network being up.
If the network is not up, then DR SMPs must be used. There are a number of
separate SMP tools for this.

The OpenIB vendor specific MAD agent will support the following attributes:
ClassPortInfo (0x0001) and SourceRoute (0x0010). There may be an additional
attribute (TBD) to support ibping but this can be done out of the same agent.

Only the VendorGet method needs to be supported by this agent. No traps
are currently defined for this class.

Although from the ibtracer client perspective, these vendor MADs are sent
on outgoing ports, it is the server (agent) which needs to validate the
incoming port. As a result of this, it is the expected incoming port
at the next hop which needs to be added to the SourceRoute attribute.
SourceRoute requests (Gets) and responses (GetResps) are exchanged
directly between the source node where the ibtracer command is initiated
and each hop along the way to the destination until the destination is
reached. As the hops to the destination are walked, the incoming ports are
added to the SourceRoute attribute and checked when the packet is received
by that hop that it did arrive on that port. If it did not arrive on that
port, an error is indicated in the status field (status 7). In either case,
the port it did arrive on is put in the SourceRoute attribute in the GetResp.

One of DLID or DGID must be specified in the ibtracer invocation.
If DGID is specified, a PathRecord request is made to the SA
to obtain the DLID. Other than that, no SM or SA is involved with
ibtracer although the SM is needed to set up the forwarding tables.

Once the DLID is obtained from either the command invoication or the SA,
a DR SMP packet is sent to the next hop to obtain the PortInfo attribute
to obtain the base LID for the next hop. A VendorGet(ClassPortInfo) is
then attempted to see if this management class is supported on that node.
If it is (a VendorGetResp is received), a VendorGet(SourceRoute)
to the next hop LID is attempted after updating the source route attribute
with the local port number from the returned PortInfo attribute. Upon
receipt of the VendorGet(SourceRoute), the receiving agent validates the
port number it is received on with the port number in the SourceRoute
attribute. It indicates failure when they do not match and in either case
the port is was received on is put back into the VendorGetResp(SourceRoute).

If the next hop does not support this management class, this is indicated
(if -v is enabled) and the algorithm proceeds with the next hop. The
algorithm is terminated when the next hop LID is the DLID (factoring in
the LMC).

Note that rather than doing much of this with DR SMPs directly,
these could be SA requests (using PortInfoRecords and LinkRecords,
or TraceRecords). Investigation would need to be done to validate
whether these SA attributes are supported by the various SMs
(although OpenSM is most important in terms of OpenIB). TraceRecords
are optional and are not believed to be currently supported. It can
be done with just PortInfoRecords and LinkRecords.

Since vendor MADs are UD, there is a retransmission strategy (timeout/retry)
which have defaults but are settable on the command line.

-v option displays entire path. Note that the incoming (rather than outgoing)
ports are displayed. Without -v specified, just success or failure is
displayed.

Reversible paths are used for the responses. Note that the path from A to
B might not be the same from B to A so ibtracer needs to be initiated at
both ends if this is of interest.

This tool cannot currently be used for multicast tracing. There are a
couple of reasons for this. Base switch port 0 does not support
multicast and it is not a requirement of enhanced switch port 0
to support this so there would be more hop skipping. Also, the attribute
format would need to be enhanced for this as well as the client needing
to handle multiple responses to a single request.


SourceRoute attribute format

Actual Incoming Port Number (valid on response) - 1 byte
Current Hop Count                               - 1 byte
Vector of Incoming Port Numbers (0-63)          - 64 bytes


Outstanding Questions

Should SL be supported rather than assume SL 0 ?

Should GRH be supported (and tied to GID specification in command invocation) ?

Is multicast tracing important ?