1Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. 2 3Contents 4======= 5- 1. Introduction 6- 2. Identifying the adapter/interface 7- 3. Features supported 8- 4. Command line parameters 9- 5. Performance suggestions 10- 6. Available Downloads 11 12 131. Introduction: 14This Linux driver supports Neterion's Xframe I PCI-X 1.0 and 15Xframe II PCI-X 2.0 adapters. It supports several features 16such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. 17See below for complete list of features. 18All features are supported for both IPv4 and IPv6. 19 202. Identifying the adapter/interface: 21a. Insert the adapter(s) in your system. 22b. Build and load driver 23# insmod s2io.ko 24c. View log messages 25# dmesg | tail -40 26You will see messages similar to: 27eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA 28eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA 29eth4: Device is on 64 bit 133MHz PCIX(M1) bus 30 31The above messages identify the adapter type(Xframe I/II), adapter revision, 32driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). 33In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed 34as well. 35 36To associate an interface with a physical adapter use "ethtool -p <ethX>". 37The corresponding adapter's LED will blink multiple times. 38 393. Features supported: 40a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes, 41modifiable using ifconfig command. 42 43b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit 44and receive, TSO. 45 46c. Multi-buffer receive mode. Scattering of packet across multiple 47buffers. Currently driver supports 2-buffer mode which yields 48significant performance improvement on certain platforms(SGI Altix, 49IBM xSeries). 50 51d. MSI/MSI-X. Can be enabled on platforms which support this feature 52(IA64, Xeon) resulting in noticeable performance improvement(upto 7% 53on certain platforms). 54 55e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt 56moderation. 57 58f. Statistics. Comprehensive MAC-level and software statistics displayed 59using "ethtool -S" option. 60 61g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, 62with multiple steering options. 63 644. Command line parameters 65a. tx_fifo_num 66Number of transmit queues 67Valid range: 1-8 68Default: 1 69 70b. rx_ring_num 71Number of receive rings 72Valid range: 1-8 73Default: 1 74 75c. tx_fifo_len 76Size of each transmit queue 77Valid range: Total length of all queues should not exceed 8192 78Default: 4096 79 80d. rx_ring_sz 81Size of each receive ring(in 4K blocks) 82Valid range: Limited by memory on system 83Default: 30 84 85e. intr_type 86Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X) 87Valid range: 1-3 88Default: 1 89 905. Performance suggestions 91General: 92a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) 93b. Set TCP windows size to optimal value. 94For instance, for MTU=1500 a value of 210K has been observed to result in 95good performance. 96# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" 97# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" 98For MTU=9000, TCP window size of 10 MB is recommended. 99# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" 100# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" 101 102Transmit performance: 103a. By default, the driver respects BIOS settings for PCI bus parameters. 104However, you may want to experiment with PCI bus parameters 105max-split-transactions(MOST) and MMRBC (use setpci command). 106A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. 107It could be different for your hardware. 108Set MMRBC to 4K**. 109 110For example you can set 111For opteron 112#setpci -d 17d5:* 62=1d 113For Itanium 114#setpci -d 17d5:* 62=3d 115 116For detailed description of the PCI registers, please see Xframe User Guide. 117 118b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this 119parameter. 120c. Turn on TSO(using "ethtool -K") 121# ethtool -K <ethX> tso on 122 123Receive performance: 124a. By default, the driver respects BIOS settings for PCI bus parameters. 125However, you may want to set PCI latency timer to 248. 126#setpci -d 17d5:* LATENCY_TIMER=f8 127For detailed description of the PCI registers, please see Xframe User Guide. 128b. Use 2-buffer mode. This results in large performance boost on 129certain platforms(eg. SGI Altix, IBM xSeries). 130c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to 131set/verify this option. 132d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network 133device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to 134bring down CPU utilization. 135 136** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are 137recommended as safe parameters. 138For more information, please review the AMD8131 errata at 139http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf 140 1416. Available Downloads 142Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up 143to date, also the latest "s2io" code (including support for 2.4 kernels) is 144available via "Support" link on the Neterion site: http://www.neterion.com. 145 146For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com, 147user: linuxdocs password: HALdocs 148 1497. Support 150For further support please contact either your 10GbE Xframe NIC vendor (IBM, 151HP, SGI etc.) or click on the "Support" link on the Neterion site: 152http://www.neterion.com. 153