1libibmad: 2 3Overview: libibmad provides the following functionalities: 4 * common declarations: IB structures, fields, and enumerations 5 * general IB mad interface encapsulation (init port, registration, 6 etc.) 7 * IB mads marshaling and de-marshaling 8 * Reliable mad RPC mechanisms (solicited mads) 9 * Server side mad io functions (send, receive) 10 * General SMP support 11 * General SA (queries) support 12 * Port performance class support 13 * IB addresses resolution (path record queries) 14 * Fields parsing and dump functions 15 * Debugging support 16 17Model of operation: libibmad is designed to easy the implementation of MAD 18 client and server tools and applications. Mad clients (i.e. application 19 that do not reply to requests) should use the mad RPC mechanism. Mad 20 servers should use the send/receive mechanisms. Applications are 21 assumed to be single threaded, but some multiple threading support 22 is provided. Most IO mechanisms may be blocking. Currently no 23 explicit asynchronous support is implemented. 24 25Marshaling/De-marshaling model: libibmad handles two types of data - opaque 26 network data images and native host ordered fields. libibmad do 27 not define C structures to access MAD fields. Instead, it defines 28 a field structure for every separate field and implements a set 29 of conversion functions from the native types to the opaque 30 network image and back. The advantage of this approach is 31 that the marshaling/de-marshaling problems are transparent to most 32 of the application code resulting a clean, machine independent 33 code. Furthermore, even the marshaling/de-marshaling code itself 34 is extremely straight-forward due that fact that the library 35 automatically knows what marshaling/de-marshaling method it has 36 to apply to each field. The disadvantage of this approach is that 37 the marshaling/de-marshaling implementation itself is somehow less 38 efficient then manually crafted manipulations, but this seem a fair 39 tradeoff comparing to the simplicity and cleanness factors. 40 41Field dump functions: a side benefit of the marshaling/de-marshaling model 42 (see above), is that the library is aware to the size and the type 43 of each field and therefore is able to print out a human readable 44 representation of the field value. 45 46 47Library objects: 48 49ib_field_t: IB field structure 50 51ib_dr_path_t: direct routed address structure 52 53ib_portid_t: (endpoint) address structure 54 55ib_attr_t: mad attribute and modifier 56 57ib_rpc_t: encapsulate information required for the RPC mechanism 58 59ib_rmpp_hdr_t: RMPP information structure (currently not supported) 60 61ib_sa_call_t: SA request structure 62 63ib_vendor_call_t: vendor specific mad structure 64 65 66Mad RPC functions: 67 68madrpc_init: 69 70Synopsis: 71 void madrpc_init(char *dev_name, int dev_port, 72 int *mgmt_classes, int num_classes); 73 74Description: library main initialization function. Open the user mad port 75specified by 'dev_name' and 'dev_port', and registers the application as mad 76client for the 'num_classes' management classes specified in 'mgmt_classes' 77array. This function must be called before any other call to the library. 78Initialization errors cause this function to panic. 79 80madrpc: 81 82Synopsis: 83 void * madrpc(ib_rpc_t *rpc, ib_portid_t *dport, 84 void *payload, void *rcvdata); 85 86Description: Perform RPC to the destination port specified by 'dport' using 87'rpc' parameters. If 'payload' in non-null, copy the payload buffer to the 88outgoing packet, while if the 'rcvdata' is non-null, copy the received packet 89payload to the 'rcvdata' buffer. Both buffer must be big enough to contain the 90maximal mad data payload length. If in doubt, use 256 bytes sized buffers. 91Return rcvdata pointer on success, and null on errors. 92 93madrpc_rmpp: 94 95Synopsis: 96 void * madrpc_rmpp(ib_rpc_t *rpc, ib_portid_t *dport, 97 ib_rmpp_hdr_t *rmpp, void *data); 98 99Description: Same as madrpc but supports also RMPP mads. 100 101Bugs: 102 RMPP is not supported yet. 103 104madrpc_portid: 105 106Synopsis: 107 int madrpc_portid(void); 108 109Description: return the portid the library uses. See libibumad:portid for 110details. 111 112See also: 113 libibumad:umad_open_port 114 115madrpc_set_retries: 116 117Synopsis: 118 int madrpc_set_retries(int retries); 119 120Description: Change the maximal number of retries attempted by the library 121before it times out to 'retries'. Non-positive values are ignored. Return 122the current retries count. 123 124madrpc_set_timeout: 125 126Synopsis: 127 int madrpc_set_timeout(int timeout); 128 129Description: Change the default timeout value used for solicited mads to 130'timeout' milliseconds. Return 0 on success, -1 on errors. Note that the 131'timeout' value is used per retry, meaning the total timeout value is acctualy 132'timeout' * max_retries (see madrpc_set_retries()). 133 134madrpc_save_mad: 135 136Synopsis: 137 void madrpc_save_mad(void *madbuf, int len); 138 139Description: Save the next replied mad image in 'madbuf', copying maximux 'len' 140bytes. In fact, this function snoop a single incoming mad. To snoop several 141packets, this function has to be called repeatedly after each RPC operation. 142 143Bugs: 144 Not applicable to mad_receive 145 146madrpc_lock: 147 148Synopsis: 149 void madrpc_lock(void); 150 151Description: Locks the mad RPC mechanism until madrpc_unlock() is called. Calls 152to this function while the RPC mechanism is already locked cause the calling 153process to be blocked until madrpc_unlock(). This function should be used 154only by multiple-threaded applications. 155 156See also: 157 madrpc_unlock 158 159madrpc_unlock: 160 161Synopsis: 162 void madrpc_unlock(void); 163 164Description: Unlock the mad RPC mechanism. See madrpc_lock() for details. 165 166madrpc_show_errors: 167 168Synopsis: 169 void madrpc_show_errors(int set); 170 171Description: If 'set' is non-null, print out warning messages on some error 172events: retries, timeouts, replies with error status, etc. Zero 'set' value 173causes the library to be quiet. 174 175ib_mad_dump_fn: 176 177Synopsis: 178 typedef void (ib_mad_dump_fn)(char *buf, int bufsz, 179 void *val, int valsz); 180 181Description: Dump the value given in 'val' that have 'valsz' size (in bytes), 182to the specified 'buf' buffer and limit the output to 'bufsz' bytes. The 183output is expected to be human readable. 184 185 186Management classes' registration functions: 187 188Synopsis: 189 int mad_register_client(int mgmt); 190 191Description: Register the application as a client of the specified 192'mgmt'. Return a non-negative agentid on success and -1 on errors. 193Note that madrpc_init provides more efficient method to register to several 194classes. 195 196See also: 197 madrpc_init 198 199mad_register_server: 200 201Synopsis: 202 int mad_register_server(int mgmt, uint32 method_mask[4], 203 uint32 class_oui); 204 205Description: Register the appication as the default responder of the class 206methods specified by 'mngt' and 'method_mask' bitmap. Vendor classes in 207range 2 require also non-zero 'class_oui'. Return a non-negative agentid on 208success and -1 on errors. 209 210mad_class_agent: 211 212Synopsis: 213 int mad_class_agent(int mgmt); 214 215Description: Map the given 'mgmt' class to agentid of the agent handling 216this class. Return non-negative agentid or -1 if the specified class is 217not registered. 218 219Synopsis: 220 int mad_agent_class(int agent); 221 222Description: Map the given 'agent' id to the management class registered 223for it. Return positive class value on success, 0 if no management class 224is registered for this agentid, or -1 if the agent id is invalid. 225 226MAD client functions: 227 228ib_vendor_call: 229 230Synopsis: 231 uint8 *ib_vendor_call(void *data, ib_portid_t *dport, 232 ib_vendor_call_t *call); 233 234Description: Perform vendor specific RPC specified by 'call' to the destination 235port specified by 'dport'. The buffer pointed by 'data' is used as payload 236for the outgoing packet, and the received packet payload is copied back 237to the 'data' buffer. The 'data' buffer must be big enough to contain the 238replied data. Note that if the 'call' method is not get/set/trap, then a 239simple send operation is performed and the function returns immediately. 240Return the 'data' pointer on success, or null on errors. 241 242mad_is_vendor_range1: 243 244Synopsis: 245 int mad_is_vendor_range1(int mgmt); 246 247Description: return non-zero value if 'mgmt' is in the vendor specific range 2481, and zero otherwise. 249 250mad_is_vendor_range2: 251 252Synopsis: 253 int mad_is_vendor_range2(int mgmt); 254 255Description: return non-zero value if 'mgmt' is in the vendor specific range 2562, and zero otherwise. 257 258smp_query: 259 260Synopsis: 261 uint8 * smp_query(void *buf, ib_portid_t *dport, 262 uint attrid, uint mod, uint timeout); 263 264Description: Perform the SMP query (get) RPC specified by 'attrid' and 'mod' 265to the destination port 'dport'. The data in 'buf' is used as the outgoing 266SMP payload, and the replied packet's data is copied back to 'buf'. The 267buffer pointed by 'buf' should be big enough to contain the reply - i.e. at 268least 64 bytes. If timeout is non-zero then it is used as the query's 269timeout. Otherwise the default timeout value is used. 270 271See also: 272 madrpc_set_timeout 273 274smp_set: 275 276Synopsis: 277 uint8 * smp_set(void *buf, ib_portid_t *dport, 278 uint attrid, uint mod, uint timeout); 279 280Description: Same as smp_query() but a set method is used instead of get. 281Note that SMP sets may lead to many (desired or less desired) results. 282Specifically it may cause the destination port to malfunction, confuse the 283current master SM, and lead to non-functioning network. Do not use this 284function unless you really know what you are doing. 285 286See also: 287 smp_set 288 289Bugs: 290 very dangerous. Shouldn't be allowed to non-privileged applications 291 292Synopsis: 293 uint8 * safe_smp_query(void *rcvbuf, ib_portid_t *portid, 294 uint attrid, uint mod, uint timeout) 295 296Description: Thread-safe version of smp_query(). 297 298See also: 299 smp_query 300 301safe_smp_set: 302 303Synopsis: 304 uint8 * safe_smp_set(void *rcvbuf, ib_portid_t *portid, 305 uint attrid, uint mod, uint timeout) 306 307Description: Thread-safe version of smp_set(). 308 309See also: 310 smp_set 311 312sa_call: 313 314Synopsis: 315 uint8 * sa_call(void *data, ib_portid_t *dport, 316 ib_sa_call_t *sa, uint timeout); 317 318Description: Perform SA RPC specified by 'sa' to the specified port 319'dport'. The 'data' buffer is used as the outgoing mad payload, and the 320returned packet's payload is copied back to the 'data' buffer. The buffer 321must be big enough to contain the response. If timeout is non-zero then it 322is used as the query's timeout. Otherwise the default timeout value is used. 323Return 'data' pointer on success, and null on errors. 324 325See also: 326 smp_query, smp_set_timeout 327 328Bugs: 329 RMPP support is missing, not all methods are supported 330 331ib_path_query: 332 333Synopsis: 334 int ib_path_query(ib_gid_t srcgid, ib_gid_t destgid, 335 ib_portid_t *sm_id, void *buf); 336 337Description: Perform a simple path record get query using the 'srcgid' and the 338'destgid' arguments. The query is targeted to the SM specified by 'sm_id'. 339Copy the query's result to the buffer 'buf' and returns the destination 340LID. If the query fails return -1. 341 342 343Synopsis: 344 uint8 * safe_sa_call(void *rcvbuf, ib_portid_t *portid, 345 ib_sa_call_t *sa, uint timeout); 346 347Description: Thread-safe version of sa_call(). 348 349See also 350 sa_call 351 352port_performance_query: 353 354Synopsis: 355 uint8 *port_performance_query(void *rcvbuf, ib_portid_t *dport, 356 int portnum, uint timeout); 357 358Description: Perform a port counters get query to the destination port(s) 359specified by 'dport' and portnum. Use portnum of 0xff to get the aggregated 360counters of the entire node. The query result is copied to the 'rcvbuf' that 361must be big enough to contain the response. If timeout is non-zero then it 362is used as the query's timeout. Otherwise the default timeout value is used. 363Return 'rcvbuf' pointer on success, and null on errors. 364 365port_performance_reset: 366 367Synopsis: 368 uint8 *port_performance_reset(void *rcvbuf, ib_portid_t *dest, 369 int portnum, uint mask, uint timeout); 370 371Description: Perform a port counters set operation to clear the counters of the 372destination port(s) specified by 'dport' and 'portnum'. the 'mask' bit-field 373is used to specify which counters are cleared. Use 'portnum' of 0xff to clear 374the aggregated counters of the entire node. The operation result is copied 375to the 'rcvbuf' that must be big enough to contain the response. If timeout 376is non-zero then it is used as the query's timeout. Otherwise the default 377timeout value is used. Return 'rcvbuf' pointer on success, and null on errors. 378 379Mad server functions: 380 381mad_send: 382 383Synopsis: 384 int mad_send(ib_rpc_t *rpc, ib_portid_t *dport, 385 ib_rmpp_hdr_t *rmpp, void *data); 386 387Description: Send a single mad to the destination port specified by 388'dport'. The mad is build using 'rpc' and rmpp arguments and the payload 389'data'. Note that this function operates similarly to send part of madrpc 390and madrpc_rmpp returns immediately after the send without retrying or 391waiting for the response (if any). Note that if solicited mads are send 392using this function, it is the caller responsibility to handle retries and 393timeouts. Return zero on success, -1 on errors. 394 395See also: 396 madrpc, madrpc_rmpp 397 398mad_receive: 399 400Synopsis: 401 void * mad_receive(void *umad, int timeout_ms); 402 403Description: Wait 'timeout_ms' milliseconds for a packet to be received. Once 404a packet is received, it is copied to the specified 'umad' buffer allocated 405by mad_alloc() or to a internally allocated umad buffer if 'umad' is null. In 406any case it is the caller responsibility to free the received packet using 407mad_free(). Negative 'timeout_ms' value makes the function to block until 408a packet is received. Zero 'timeout_ms' guarantees non blocking read, 409i.e. either the function returns immediately with new received packet, 410or it will return with error. Return a pointer to the received umad buffer 411or null in case of errors. 412 413mad_respond: 414 415Synopsis: 416 int mad_respond(void *umad, ib_portid_t *portid, uint32 rstatus); 417 418Description: Respond to the request mad specified by 'umad'. Send the 419response mad to the port specified by 'portid' or the original caller of 420'umad' if 'portid' is null. The status 'rstatus' is used to fill the mad 421status field. The following outgoing fields are set by the function using the 422original 'umad' fields: mgt_class, method, attribute_id, attribute_modifier, 423SA attribute offset, vendor class OUI, mad transaction id (only the relevant 424fields are set). Return zero on success, -1 on errors. 425 426mad_alloc: 427 428Synopsis: 429 void * mad_alloc(void); 430 431Description: Allocate a user mad buffer. This buffer should be de-allocated 432using mad_free(). The mad buffer (umad) should be used be used as opaque. 433Return a pointer to the buffer, or null if the allocation fails. 434 435See also: 436 mad_free 437 438Synopsis: 439 void mad_free(void *umad); 440 441Description: Free a umad buffer previously allocated by mad_alloc 442 443See also: 444 mad_alloc 445 446Address resolving functions: 447 448ib_resolve_smlid: 449 450Synopsis: 451 int ib_resolve_smlid(ib_portid_t *sm_id, int timeout); 452 453Description: Resolve the current SM address (LID) and copy it to 454'sm_id'. Internally this function queries the local port for the smlid 455field. 'timeout' is used similarly to madrpc(). Return zero on success, 456-1 on errors. 457 458ib_resolve_guid: 459 460Synopsis: 461 int ib_resolve_guid(ib_portid_t *portid, uint64_t *guid, 462 ib_portid_t *sm_id, int timeout); 463 464Description: Resolve the given 'guid' to find the port lid and set 'portid' 465accordingly. The resolving process is done by sending a path record query 466to the SM using the specified address 'sm_id'. If the 'sm_id' is null, the 467SM address is resolved using ib_resove_smlid(). 'timeout' is used similary 468to madrpc(). Return zero on success, -1 on errors. 469 470See also: 471 ib_resolve_smlid, ib_path_query, madrpc 472 473ib_resolve_portid_str: 474 475Synopsis: 476 int ib_resolve_portid_str(ib_portid_t *portid, char *addr_str, 477 int dest_type, ib_portid_t *sm_id); 478 479Description: Resolve the port address specified by the string 'addr_str' 480and the type 'dest_type' and set 'portid' accordingly. If the dest_type 481is IB_DEST_GUID, then a path record query is sent to the SM specified by 482'sm_id' or to the SM address found by ib_resolve_smlid() if sm_id is null. The 483following string formats are supported: 484 Type String 485 IB_DEST_LID: (Decimal/Hex) integer (see strtoul for details) 486 IB_DEST_DRPATH out-ports vector "p1,p2,p3" (e.g. "0,1,6,5,20,1") 487 IB_DEST_GUID: 64 bit integer (see strtoll for details) 488Return zero on success, -1 on errors. 489 490See also: 491 str2drpath, ib_resolve_smlid, ib_resolve_guid 492 493ib_resolve_self: 494 495Synopsis: 496 int ib_resolve_self(ib_portid_t *portid, int *portnum, 497 ib_gid_t *gid); 498 499Description: Resolve the local port address and set 'portid', 'portnum' and 500'gid' accordingly. The resolve process is done by issuing a NodeInfo and 501PortInfo to the local port. Return zero on success, -1 on errors. 502 503Port ID helper functions: 504 505portid2str: 506 507Synopsis: 508 char * portid2str(ib_portid_t *portid); 509 510Description: Return a string representation of the specified 'portid'. 511 512Bugs: 513 uses a static string buffer and therefore not thread safe. 514 515portid2portnum: 516 517Synopsis: 518 int portid2portnum(ib_portid_t *portid); 519 520Description: Return the port number of the destination port specified by 521the direct routed address 'portid'. Return -1 if the portid is not directed 522route address, and 0 if it is local port address (vector [0]). 523 524str2drpath: 525 526Synopsis: 527 int str2drpath(ib_dr_path_t *path, char *routepath, 528 int drslid, int drdlid); 529 530Description: Parse the 'routepath' string, and use the given 'drslid' and 531'drdlid' set the given 'path'. Return path count or -1 on invalid string. 532 533ib_portid_set: 534 535Synopsis: 536 int ib_portid_set(ib_portid_t *portid, int lid, int qp, int qkey); 537 538Description: Set the given 'portid' fields using the 'lid', 'qp' and 'qkey' 539arguments. 540 541Mad fields manipulation functions: 542 543mad_get_field: 544 545Synopsis: 546 uint32 mad_get_field(void *buf, int base_offset, int field); 547 548Description: Return the value of 'field' from the mad buffer specified by 549'buf' and the offset 'base_offset' within. The result is in host order. 550 551See also: 552 ib_mad_f fields array, model of operation 553 554mad_set_field: 555 556Synopsis: 557 void mad_set_field(void *buf, int base_offs, int field, uint32 val); 558 559Description: Set the value of 'field' in the mad buffer specified by 'buf' 560and the offset 'base_offset' within, using host ordered 'val'. 561 562See also: 563 ib_mad_f fields array, model of operation 564 565mad_get_field64: 566 567Synopsis: 568 uint64 mad_get_field64(void *buf, int base_offs, int field); 569 570Description: Same as mad_get_field, but for 64 bit fields. 571 572mad_set_field64: 573 574Synopsis: 575 void mad_set_field64(void *buf, int base_offs, 576 int field, uint64 val); 577 578Description: Same as mad_set_field, but for 64 bit fields. 579 580mad_set_array: 581 582Synopsis: 583 void mad_set_array(void *buf, int base_offs, int field, void *val); 584 585Description: Same as mad_get_field, but for opaque byte arrays. 586 587mad_get_array: 588 589Synopsis: 590 void mad_get_array(void *buf, int base_offs, int field, void *val); 591 592Description: Same as mad_set_field, but for opaque byte arrays. 593 594mad_decode_field: 595 596Synopsis: 597 void mad_decode_field(uint8 *buf, int field, void *val); 598 599Description: Decode 'field' within the mad buffer specified by 'buf' and 600return it in 'val'. The result is in host order. Note that the buffer pointer 601by 'val' must be big enough to contain the value. 602 603See also: 604 ib_mad_f fields array, model of operation 605 606mad_encode_field: 607 608Synopsis: 609 void mad_encode_field(uint8 *buf, int field, void *val); 610 611Description: Encode the 'field' within the mad buffer specified by 'buf' 612using the host ordered value 'val'. 613 614See also: 615 ib_mad_f fields array, model of operation 616 617mad_encode: 618 619Synopsis: 620 void * mad_encode(void *buf, ib_rpc_t *rpc, 621 ib_dr_path_t *drpath, void *data); 622Description: Encode an outgoing mad headers in 'buf' using the given 'rpc', 623the optional direct routed address 'drpath', and the optional payload 624'data'. Return a pointer to the first byte after the mad image, or null 625on errors. 626 627mad_trid: 628 629Synopsis: 630 uint64 mad_trid(void); 631 632Description: Set the given 'portid' fields using the 'lid', 'qp' and 'qkey' 633 634mad_build_pkt: 635 636Synopsis: 637 int mad_build_pkt(void *umad, ib_rpc_t *rpc, ib_portid_t *dport, 638 ib_rmpp_hdr_t *rmpp, void *data); 639 640Description: Encode a mad in the buffer 'umad' given the structures 'rpc', 641'dport', the optional 'rmpp' structure and the payload 'data'. Return 642number of encode bytes or a negative number if failed. 643 644Dump functions: 645 646mad_print_field: 647 648Synopsis: 649 int mad_print_field(int field, char *name, void *val); 650 651Description: Print a human readable format of the 'field' given the value 652'val' to the standard output. If 'name' is non-null, it is printed as the 653field name. Otherwise the default field name is used. Return the number of 654printed bytes. 655 656See also: 657 ib_mad_f fields array, model of operation 658 659mad_dump_field: 660 661Synopsis: 662 char * mad_dump_field(int field, char *buf, int bufsz, void *val); 663 664Description: Print a human readable format of the 'field' given the value 665'val' to the given buffer 'buf'. The default field name is used. No more than 666'bufsz' bytes are printed. Return the number of printed bytes. 667 668mad_dump_val: 669 670Synopsis: 671 char * mad_dump_val(int field, char *buf, int bufsz, void *val); 672 673Description: Same as mad_print_field, but only the field value is printed. 674 675Debugging support: 676 677ibdebug: 678 679Synopsis: 680 extern int ibdebug; 681 682Description: Control the library debugging level. The following levels 683are supported: 684 0 - no debugging 685 1 - print debugging information 686 2 - as level 1 but also xdump the umad IO 687 688