1ibtracer 21/11/05 3 4Description: 5ibtracer is used to build a source route into a UD packet and validate the 6path taken to a destination. It is based on a client/server architecture and 7relies on a special ibtracer IB agent in each node along the way. It can 8deal with switches which do not currently run this agent but validation for 9that part of the path is impossible. 10 11Syntax: 12ibtracer [-I mthca0] [-p port] [-r <# retries>] [-t <timeout in msec>] \ 13 [-l LID] [-g DGID] [-v] 14 15Architecture: 16IBA 1.2 defines a new set of vendor specific MADs which include OUI. OpenIB 17will use one of these classes (0x30) to implement ibtracer (and the vendor 18MAD option of ibping). 19 20Note that these are general service MADs and rely on the network being up. 21If the network is not up, then DR SMPs must be used. There are a number of 22separate SMP tools for this. 23 24The OpenIB vendor specific MAD agent will support the following attributes: 25ClassPortInfo (0x0001) and SourceRoute (0x0010). There may be an additional 26attribute (TBD) to support ibping but this can be done out of the same agent. 27 28Only the VendorGet method needs to be supported by this agent. No traps 29are currently defined for this class. 30 31Although from the ibtracer client perspective, these vendor MADs are sent 32on outgoing ports, it is the server (agent) which needs to validate the 33incoming port. As a result of this, it is the expected incoming port 34at the next hop which needs to be added to the SourceRoute attribute. 35SourceRoute requests (Gets) and responses (GetResps) are exchanged 36directly between the source node where the ibtracer command is initiated 37and each hop along the way to the destination until the destination is 38reached. As the hops to the destination are walked, the incoming ports are 39added to the SourceRoute attribute and checked when the packet is received 40by that hop that it did arrive on that port. If it did not arrive on that 41port, an error is indicated in the status field (status 7). In either case, 42the port it did arrive on is put in the SourceRoute attribute in the GetResp. 43 44One of DLID or DGID must be specified in the ibtracer invocation. 45If DGID is specified, a PathRecord request is made to the SA 46to obtain the DLID. Other than that, no SM or SA is involved with 47ibtracer although the SM is needed to set up the forwarding tables. 48 49Once the DLID is obtained from either the command invoication or the SA, 50a DR SMP packet is sent to the next hop to obtain the PortInfo attribute 51to obtain the base LID for the next hop. A VendorGet(ClassPortInfo) is 52then attempted to see if this management class is supported on that node. 53If it is (a VendorGetResp is received), a VendorGet(SourceRoute) 54to the next hop LID is attempted after updating the source route attribute 55with the local port number from the returned PortInfo attribute. Upon 56receipt of the VendorGet(SourceRoute), the receiving agent validates the 57port number it is received on with the port number in the SourceRoute 58attribute. It indicates failure when they do not match and in either case 59the port is was received on is put back into the VendorGetResp(SourceRoute). 60 61If the next hop does not support this management class, this is indicated 62(if -v is enabled) and the algorithm proceeds with the next hop. The 63algorithm is terminated when the next hop LID is the DLID (factoring in 64the LMC). 65 66Note that rather than doing much of this with DR SMPs directly, 67these could be SA requests (using PortInfoRecords and LinkRecords, 68or TraceRecords). Investigation would need to be done to validate 69whether these SA attributes are supported by the various SMs 70(although OpenSM is most important in terms of OpenIB). TraceRecords 71are optional and are not believed to be currently supported. It can 72be done with just PortInfoRecords and LinkRecords. 73 74Since vendor MADs are UD, there is a retransmission strategy (timeout/retry) 75which have defaults but are settable on the command line. 76 77-v option displays entire path. Note that the incoming (rather than outgoing) 78ports are displayed. Without -v specified, just success or failure is 79displayed. 80 81Reversible paths are used for the responses. Note that the path from A to 82B might not be the same from B to A so ibtracer needs to be initiated at 83both ends if this is of interest. 84 85This tool cannot currently be used for multicast tracing. There are a 86couple of reasons for this. Base switch port 0 does not support 87multicast and it is not a requirement of enhanced switch port 0 88to support this so there would be more hop skipping. Also, the attribute 89format would need to be enhanced for this as well as the client needing 90to handle multiple responses to a single request. 91 92 93SourceRoute attribute format 94 95Actual Incoming Port Number (valid on response) - 1 byte 96Current Hop Count - 1 byte 97Vector of Incoming Port Numbers (0-63) - 64 bytes 98 99 100Outstanding Questions 101 102Should SL be supported rather than assume SL 0 ? 103 104Should GRH be supported (and tied to GID specification in command invocation) ? 105 106Is multicast tracing important ? 107