Hardware.txt revision 84631
11541Srgrimes/* $FreeBSD: head/sys/dev/isp/Hardware.txt 84631 2001-10-07 18:26:47Z mjacob $ */ 21541Srgrimes 31541Srgrimes Hardware that is Known To or Should Work with This Driver 41541Srgrimes 51541Srgrimes 61541Srgrimes0. Intro 71541Srgrimes 81541Srgrimes This is not an endorsement for hardware vendors (there will be 91541Srgrimes no "where to buy" URLs here with a couple of exception). This 101541Srgrimes is simply a list of things I know work, or should work, plus 111541Srgrimes maybe a couple of notes as to what you should do to make it 121541Srgrimes work. Corrections accepted. Even better would be to send me 131541Srgrimes hardware to I can test it. 141541Srgrimes 151541Srgrimes I'll put a rough range of costs in US$ that I know about. No doubt 161541Srgrimes it'll differ from your expectations. 171541Srgrimes 181541Srgrimes1. HBAs 191541Srgrimes 201541SrgrimesQlogic 2100, 2102 211541Srgrimes 2200, 2202, 2204 221541Srgrimes 231541Srgrimes There are various suffices that indicate copper or optical 241541Srgrimes connectors, or 33 vs. 66MHz PCI bus operation. None of these 251541Srgrimes have a software impact. 261541Srgrimes 271541Srgrimes Approx cost: 1K$ for a 2200 281541Srgrimes 291541SrgrimesQlogic 2300, 2312 301541Srgrimes 311541Srgrimes These are the new 2-Gigabit cards. Optical only. 321541Srgrimes 331541Srgrimes Approx cost: ?????? 341541Srgrimes 351541Srgrimes 361541SrgrimesAntares P-0033, P-0034, P-0036 371541Srgrimes 381541Srgrimes There many other vendors that use the Qlogic 2X00 chipset. Some older 393374Sdg 2100 boards (not on this list) have a bug in the ROM that causes a 401541Srgrimes failure to download newer firmware that is larger than 0x7fff words. 411541Srgrimes 421541Srgrimes Approx cost: 850$ for a P-0036 431541Srgrimes 441541Srgrimes 451541Srgrimes 461541Srgrimes In general, the 2200 class chip is to be preferred. 471541Srgrimes 481541Srgrimes 491541Srgrimes2. Hubs 501541Srgrimes 511541SrgrimesVixel 1000 521541SrgrimesVixel 2000 531541Srgrimes Of the two, the 1000 (7 ports, vs. 12 ports) has had fewer problems- 541541Srgrimes it's an old workhorse. 551541Srgrimes 561541Srgrimes 571541Srgrimes Approx cost: 1.5K$ for Vixel 1000, 2.5K$ for 2000 581549Srgrimes 591541SrgrimesGadzoox Cappellix 3000 601541Srgrimes Don't forget to use telnet to configure the Cappellix ports 611541Srgrimes to the role you're using them for- otherwise things don't 621541Srgrimes work well at all. 631541Srgrimes 641541Srgrimes (cost: I have no idea... certainly less than a switch) 651541Srgrimes 661541Srgrimes3. Switches 671541Srgrimes 681541SrgrimesBrocade Silkworm II 691541SrgrimesBrocade 2400 701541Srgrimes(other brocades should be fine) 711541Srgrimes 721541Srgrimes Especially with revision 2 or higher f/w, this is now best 731541Srgrimes of breed for fabrics or segmented loop (which Brocade 741541Srgrimes calls "QuickLoop"). 751541Srgrimes 761541Srgrimes For the Silkworm II, set operating mode to "Tachyon" (mode 3). 771541Srgrimes 781541Srgrimes The web interace isn't good- but telnet is what I prefer anyhow. 791541Srgrimes 801549Srgrimes You can't connect a Silkworm II and the other Brocades together 811549Srgrimes as E-ports to make a large fabric (at least with the f/w *I* 821549Srgrimes had for the Silkworm II). 831549Srgrimes 841549Srgrimes Approx cost of a Brocade 2400 with no GBICs is about 8K$ when 853374Sdg I recently checked the US Government SEWP price list- no doubt 863374Sdg it'll be a bit more for others. I'd assume around 10K$. 873374Sdg 883374SdgANCOR SA-8 893374Sdg 903374Sdg This also is a fine switch, but you have to use a browser 911541Srgrimes with working java to manage it- which is a bit of a pain. 921541Srgrimes This also supports fabric and segmented loop. 931541Srgrimes 941541Srgrimes These switches don't form E-ports with each other for a larger 951541Srgrimes fabric. 961541Srgrimes 971541Srgrimes (cost: no idea) 981541Srgrimes 991541SrgrimesMcData (model unknown) 1001541Srgrimes 1011541Srgrimes I tried one exactly once for 30 minutes. Seemed to work once 1021541Srgrimes I added the "register FC4 types" command to the driver. 1031541Srgrimes 1041541Srgrimes (cost: very very expensive, 40K$ plus) 1051541Srgrimes 1061541Srgrimes4. Cables/GBICs 1071541Srgrimes 1081541Srgrimes Multimode optical is adequate for Fibre Channel- the same cable is 1091541Srgrimes used for Gigabit Ethernet. 1101541Srgrimes 1111541Srgrimes Copper DB-9 and Copper HSS-DC connectors are also fine. Copper && 1121541Srgrimes Optical both are rated to 1.026Gbit- copper is naturally shorter 1131541Srgrimes (the longest I've used is a 15meter cable but it's supposed to go 1141541Srgrimes longer). 1151541Srgrimes 1161541Srgrimes The reason to use copper instead of optical is that if step on one of 1171541Srgrimes the really fat DB-9 cables you can get, it'll survive. Optical usually 1181541Srgrimes dies quickly if you step on it. 1191541Srgrimes 1201541Srgrimes Approx cost: I don't know what optical is- you can expect to pay maybe 1211541Srgrimes a 100$ for a 3m copper cable. 1221541Srgrimes 1231541SrgrimesGBICs- 1241541Srgrimes 1251541Srgrimes I use Finisar copper and IBM Opticals. 1261541Srgrimes 1271541Srgrimes Approx Cost: Copper GBICs are 70$ each. Opticals are twice that or more. 1281541Srgrimes 1291541Srgrimes 1303374SdgVendor: (this is the one exception I'll make because it turns out to be 1311549Srgrimes an incredible pain to find FC copper cabling and GBICs- the source I 1321549Srgrimes use for GBICs and copper cables is http://www.scsi-cables.com) 1331541Srgrimes 1341541Srgrimes 1351541SrgrimesOther: 1361541Srgrimes There now is apparently a source for little connector boards 1371541Srgrimes to connect to bare drives: http://www.cinonic.com. 1381541Srgrimes 1391541Srgrimes 1401541Srgrimes5. Storage JBODs/RAID 1411541Srgrimes 1421541SrgrimesJMR 4-Bay 1431541Srgrimes 1441541Srgrimes Rinky-tink, but a solid 4 bay loop only entry model. 1451541Srgrimes 1461541Srgrimes I paid 1000$ for mine- overprice, IMO. 1471541Srgrimes 1481564SdgJMR Fortra 1491564Sdg 1501564Sdg I rather like this box. The blue LEDs are a very nice touch- you 1511564Sdg can see them very clearly from 50 feet away. 1521564Sdg 1531541Srgrimes I paid 2000$ for one used. 1541564Sdg 1551564SdgSun A5X00 1561564Sdg 1571564Sdg Very expensive (in my opinion) but well crafted. Has two SES 1581564Sdg instances, so you can use the ses driver (and the example 1591564Sdg code in /usr/share/examples) for power/thermal/slot monitoring. 1601564Sdg 1611564Sdg Approx Cost: The last I saw for a price list item on this was 22K$ 1621564Sdg for a unpopulated (no disk drive) A5X00. 1631564Sdg 1641564Sdg 1651564SdgDataDirect E1000 RAID 1661564Sdg 1671564Sdg Don't connect both SCSI and FC interfaces at the same time- a SCSI 1681564Sdg reset will cause the DataDirect to think you want to use the SCSI 1691564Sdg interface and a LIP on the FC interface will cause it to think you 1701564Sdg want to use the FC interface. Use only one connector at a time so 1711564Sdg both you and the DataDirect are sure about what you want. 1721564Sdg 1731564Sdg Cost: I have no idea. 1741541Srgrimes 1751541SrgrimesVeritas ServPoint 1761541Srgrimes 1771541Srgrimes This is a software storage virtualization engine that 1781541Srgrimes runs on Sparc/Solaris in target mode for frontend 1791541Srgrimes and with other FC or SCSI as the backend storage. FreeBSD 1801541Srgrimes has been used extensively to test it. 1811541Srgrimes 1821541Srgrimes 1831541Srgrimes Cost: I have no idea. 1841541Srgrimes 1851541Srgrimes6. Disk Drives 1862112Swollman 1872112Swollman I have used lots of different Seagate and a few IBM drives and 1882112Swollman typically have had few problems with them. These are the bare 1892112Swollman drives with 40-pin SCA connectors in back. They go into the JBODs 1902112Swollman you assemble. 1912112Swollman 1922112Swollman Seagate does make, but I can no longer find, a little paddleboard 1932112Swollman single drive connector that goes from DB-9 FC to the 40-pin SCA 1941541Srgrimes connector- primarily for you to try and evaluate a single FC drive. 1951541Srgrimes 1963098Sphk All FC-AL disk drives are dual ported (i.e., have separte 'A' and 1973098Sphk 'B' ports- which are completely separate loops). This seems to work 1981541Srgrimes reasonably enough, but I haven't tested it much. It really depends 1991541Srgrimes on the JBOD you put them to carry this dual port to the outside 2001541Srgrimes world. The JMR boxes have it. The Sun A5X00 you have to pay for 2011541Srgrimes an extra IB card to carry it out. 2023098Sphk 2033098Sphk Approx Cost: You'll find that FC drives are the same cost if not 2043098Sphk slightly cheaper than the equivalent Ultra3 SCSI drives. 2051549Srgrimes 2063098Sphk7. Recommended Configurations 2073098Sphk 2083098SphkThese are recommendations that are biased toward the cautious side. They 2093098Sphkdo not represent formal engineering commitments- just suggestions as to 2103098Sphkwhat I would expect to work. 2113098Sphk 2123098SphkA. The simpletst form of a connection topology I can suggest for 2133098Sphka small SAN (i.e., replacement for SCSI JBOD/RAID): 2141541Srgrimes 2151541SrgrimesHOST 2161541Srgrimes2xxx <----------> Single Unit of Storage (JBOD, RAID) 2171541Srgrimes 2181541SrgrimesThis is called a PL_DA (Private Loop, Direct Attach) topology. 2193098Sphk 2203098SphkB. The next most simple form of a connection topology I can suggest for 2213098Sphka medium local SAN (where you do not plan to do dynamic insertion 2223098Sphkand removal of devices while I/Os are active): 2231541Srgrimes 2241541SrgrimesHOST 2251541Srgrimes2xxx <----------> +-------- 226 | Vixel | 227 | 1000 | 228 | +<---> Storage 229 | | 230 | +<---> Storage 231 | | 232 | +<---> Storage 233 -------- 234 235This is a Private Loop topology. Remember that this can get very unstable 236if you make it too long. A good practice is to try it in a staged fashion. 237 238It is possible with some units to "daisy chain", e.g.: 239 240HOST 2412xxx <----------> (JBOD, RAID) <--------> (JBOD, RAID) 242 243In practice I have had poor results with these configurations. They *should* 244work fine, but for both the JMR and the Sun A5X00 I tend to get LIP storms 245and so the second unit just isn't seen and the loop isn't stable. 246 247Now, this could simply be my lack of clean, newer, h/w (or, in general, 248a lack of h/w), but I would recommend the use of a hub if you want to 249stay with Private Loop and have more than one FC target. 250 251You should also note this can begin to be the basis for a shared SAN 252solution. For example, the above configuration can be extended to be: 253 254HOST 2552xxx <----------> +-------- 256 | Vixel | 257 | 1000 | 258 | +<---> Storage 259 | | 260 | +<---> Storage 261 | | 262 | +<---> Storage 263HOST | | 2642xxx <----------> +-------- 265 266However, note that there is nothing to mediate locking of devices, and 267it is also conceivable that the reboot of one host can, by causing 268a LIP storm, cause problems with the I/Os from the other host. 269(in other words, this topology hasn't really been made safe yet for 270this driver). 271 272D. You can repeat the topology in #B with a switch that is set to be 273in segmented loop mode. This avoids LIPs propagating where you don't 274want them to- and this makes for a much more reliable, if more expensive, 275SAN. 276 277E. The next level of complexity is a Switched Fabric. The following topology 278is good when you start to begin to get to want more performance. Private 279and Public Arbitrated Loop, while 100MB/s, is a shared medium. Direct 280connections to a switch can run full-duplex at full speed. 281 282HOST 2832xxx <----------> +--------- 284 | Brocade| 285 | 2400 | 286 | +<---> Storage 287 | | 288 | +<---> Storage 289 | | 290 | +<---> Storage 291HOST | | 2922xxx <----------> +--------- 293 294 295I would call this the best configuration available now. It can expand 296substantially if you cascade switches. 297 298There is a hard limit of about 253 devices for each Qlogic HBA- and the 299fabric login policy is simplistic (log them in as you find them). If 300somebody actually runs into a configuration that's larger, let me know 301and I'll work on some tools that would allow you some policy choices 302as to which would be interesting devices to actually connect to. 303 304 305