Hardware.txt revision 256281
1280924Sdteske/* $FreeBSD: stable/10/sys/dev/isp/Hardware.txt 108533 2003-01-01 18:49:04Z schweikh $ */ 2280924Sdteske 3280924Sdteske Hardware that is Known To or Should Work with This Driver 4280924Sdteske 5280924Sdteske 6280924Sdteske0. Intro 7280924Sdteske 8280924Sdteske This is not an endorsement for hardware vendors (there will be 9280924Sdteske no "where to buy" URLs here with a couple of exception). This 10280924Sdteske is simply a list of things I know work, or should work, plus 11280924Sdteske maybe a couple of notes as to what you should do to make it 12280924Sdteske work. Corrections accepted. Even better would be to send me 13280924Sdteske hardware to I can test it. 14280924Sdteske 15280924Sdteske I'll put a rough range of costs in US$ that I know about. No doubt 16280924Sdteske it'll differ from your expectations. 17280924Sdteske 18280924Sdteske1. HBAs 19280924Sdteske 20280924SdteskeQlogic 2100, 2102 21280924Sdteske 2200, 2202, 2204 22280924Sdteske 23280924Sdteske There are various suffices that indicate copper or optical 24280924Sdteske connectors, or 33 vs. 66MHz PCI bus operation. None of these 25280924Sdteske have a software impact. 26115410Sscottl 27115410Sscottl Approx cost: 1K$ for a 2200 28115410Sscottl 29115410SscottlQlogic 2300, 2312 30280937Sdteske 31280937Sdteske These are the new 2-Gigabit cards. Optical only. 32280937Sdteske 33280924Sdteske Approx cost: ?????? 34280924Sdteske 35115410Sscottl 36115410SscottlAntares P-0033, P-0034, P-0036 37115410Sscottl 38115410Sscottl There many other vendors that use the Qlogic 2X00 chipset. Some older 39115410Sscottl 2100 boards (not on this list) have a bug in the ROM that causes a 40115410Sscottl failure to download newer firmware that is larger than 0x7fff words. 41115410Sscottl 42115410Sscottl Approx cost: 850$ for a P-0036 43244048Sdteske 44244048Sdteske 45280936Sdteske 46244048Sdteske In general, the 2200 class chip is to be preferred. 47244048Sdteske 48244048Sdteske 49124648Snyan2. Hubs 50124648Snyan 51124648SnyanVixel 1000 52124648SnyanVixel 2000 53124648Snyan Of the two, the 1000 (7 ports, vs. 12 ports) has had fewer problems- 54124648Snyan it's an old workhorse. 55124648Snyan 56124648Snyan 57124648Snyan Approx cost: 1.5K$ for Vixel 1000, 2.5K$ for 2000 58124648Snyan 59124648SnyanGadzoox Cappellix 3000 60124648Snyan Don't forget to use telnet to configure the Cappellix ports 61124648Snyan to the role you're using them for- otherwise things don't 62124648Snyan work well at all. 63124648Snyan 64124648Snyan (cost: I have no idea... certainly less than a switch) 65124648Snyan 66124648Snyan3. Switches 67124648Snyan 68124648SnyanBrocade Silkworm II 69124648SnyanBrocade 2400 70124648Snyan(other brocades should be fine) 71124648Snyan 72124648Snyan Especially with revision 2 or higher f/w, this is now best 73124648Snyan of breed for fabrics or segmented loop (which Brocade 74124648Snyan calls "QuickLoop"). 75124648Snyan 76124648Snyan For the Silkworm II, set operating mode to "Tachyon" (mode 3). 77124648Snyan 78124648Snyan The web interace isn't good- but telnet is what I prefer anyhow. 79124648Snyan 80124648Snyan You can't connect a Silkworm II and the other Brocades together 81124648Snyan as E-ports to make a large fabric (at least with the f/w *I* 82124648Snyan had for the Silkworm II). 83124648Snyan 84124648Snyan Approx cost of a Brocade 2400 with no GBICs is about 8K$ when 85124648Snyan I recently checked the US Government SEWP price list- no doubt 86124648Snyan it'll be a bit more for others. I'd assume around 10K$. 87124648Snyan 88124648SnyanANCOR SA-8 89124648Snyan 90124648Snyan This also is a fine switch, but you have to use a browser 91124648Snyan with working java to manage it- which is a bit of a pain. 92115410Sscottl This also supports fabric and segmented loop. 93280937Sdteske 94280937Sdteske These switches don't form E-ports with each other for a larger 95115410Sscottl fabric. 96115410Sscottl 97115410Sscottl (cost: no idea) 98115410Sscottl 99115410SscottlMcData (model unknown) 100115410Sscottl 101115410Sscottl I tried one exactly once for 30 minutes. Seemed to work once 102244048Sdteske I added the "register FC4 types" command to the driver. 103244048Sdteske 104244048Sdteske (cost: very very expensive, 40K$ plus) 105244048Sdteske 106244048Sdteske4. Cables/GBICs 107244048Sdteske 108244048Sdteske Multimode optical is adequate for Fibre Channel- the same cable is 109244048Sdteske used for Gigabit Ethernet. 110244048Sdteske 111115410Sscottl Copper DB-9 and Copper HSS-DC connectors are also fine. Copper && 112244048Sdteske Optical both are rated to 1.026Gbit- copper is naturally shorter 113115410Sscottl (the longest I've used is a 15meter cable but it's supposed to go 114115410Sscottl longer). 115115410Sscottl 116115410Sscottl The reason to use copper instead of optical is that if step on one of 117115410Sscottl the really fat DB-9 cables you can get, it'll survive. Optical usually 118115410Sscottl dies quickly if you step on it. 119115410Sscottl 120115410Sscottl Approx cost: I don't know what optical is- you can expect to pay maybe 121115410Sscottl a 100$ for a 3m copper cable. 122280936Sdteske 123280936SdteskeGBICs- 124280936Sdteske 125280936Sdteske I use Finisar copper and IBM Opticals. 126280936Sdteske 127115410Sscottl Approx Cost: Copper GBICs are 70$ each. Opticals are twice that or more. 128115410Sscottl 129115410Sscottl 130115410SscottlVendor: (this is the one exception I'll make because it turns out to be 131115410Sscottl an incredible pain to find FC copper cabling and GBICs- the source I 132115410Sscottl use for GBICs and copper cables is http://www.scsi-cables.com) 133115410Sscottl 134115410Sscottl 135115410SscottlOther: 136115410Sscottl There now is apparently a source for little connector boards 137115410Sscottl to connect to bare drives: http://www.cinonic.com. 138115410Sscottl 139115410Sscottl 140115410Sscottl5. Storage JBODs/RAID 141115410Sscottl 142115410SscottlJMR 4-Bay 143115410Sscottl 144115410Sscottl Rinky-tink, but a solid 4 bay loop only entry model. 145115410Sscottl 146115410Sscottl I paid 1000$ for mine- overprice, IMO. 147115410Sscottl 148115410SscottlJMR Fortra 149115410Sscottl 150115410Sscottl I rather like this box. The blue LEDs are a very nice touch- you 151115410Sscottl can see them very clearly from 50 feet away. 152115410Sscottl 153115410Sscottl I paid 2000$ for one used. 154115410Sscottl 155115410SscottlSun A5X00 156115410Sscottl 157115410Sscottl Very expensive (in my opinion) but well crafted. Has two SES 158115410Sscottl instances, so you can use the ses driver (and the example 159115410Sscottl code in /usr/share/examples) for power/thermal/slot monitoring. 160115410Sscottl 161115410Sscottl Approx Cost: The last I saw for a price list item on this was 22K$ 162115410Sscottl for an unpopulated (no disk drive) A5X00. 163115410Sscottl 164280937Sdteske 165280937SdteskeDataDirect E1000 RAID 166 167 Don't connect both SCSI and FC interfaces at the same time- a SCSI 168 reset will cause the DataDirect to think you want to use the SCSI 169 interface and a LIP on the FC interface will cause it to think you 170 want to use the FC interface. Use only one connector at a time so 171 both you and the DataDirect are sure about what you want. 172 173 Cost: I have no idea. 174 175Veritas ServPoint 176 177 This is a software storage virtualization engine that 178 runs on Sparc/Solaris in target mode for frontend 179 and with other FC or SCSI as the backend storage. FreeBSD 180 has been used extensively to test it. 181 182 183 Cost: I have no idea. 184 1856. Disk Drives 186 187 I have used lots of different Seagate and a few IBM drives and 188 typically have had few problems with them. These are the bare 189 drives with 40-pin SCA connectors in back. They go into the JBODs 190 you assemble. 191 192 Seagate does make, but I can no longer find, a little paddleboard 193 single drive connector that goes from DB-9 FC to the 40-pin SCA 194 connector- primarily for you to try and evaluate a single FC drive. 195 196 All FC-AL disk drives are dual ported (i.e., have separte 'A' and 197 'B' ports- which are completely separate loops). This seems to work 198 reasonably enough, but I haven't tested it much. It really depends 199 on the JBOD you put them to carry this dual port to the outside 200 world. The JMR boxes have it. The Sun A5X00 you have to pay for 201 an extra IB card to carry it out. 202 203 Approx Cost: You'll find that FC drives are the same cost if not 204 slightly cheaper than the equivalent Ultra3 SCSI drives. 205 2067. Recommended Configurations 207 208These are recommendations that are biased toward the cautious side. They 209do not represent formal engineering commitments- just suggestions as to 210what I would expect to work. 211 212A. The simpletst form of a connection topology I can suggest for 213a small SAN (i.e., replacement for SCSI JBOD/RAID): 214 215HOST 2162xxx <----------> Single Unit of Storage (JBOD, RAID) 217 218This is called a PL_DA (Private Loop, Direct Attach) topology. 219 220B. The next most simple form of a connection topology I can suggest for 221a medium local SAN (where you do not plan to do dynamic insertion 222and removal of devices while I/Os are active): 223 224HOST 2252xxx <----------> +-------- 226 | Vixel | 227 | 1000 | 228 | +<---> Storage 229 | | 230 | +<---> Storage 231 | | 232 | +<---> Storage 233 -------- 234 235This is a Private Loop topology. Remember that this can get very unstable 236if you make it too long. A good practice is to try it in a staged fashion. 237 238It is possible with some units to "daisy chain", e.g.: 239 240HOST 2412xxx <----------> (JBOD, RAID) <--------> (JBOD, RAID) 242 243In practice I have had poor results with these configurations. They *should* 244work fine, but for both the JMR and the Sun A5X00 I tend to get LIP storms 245and so the second unit just isn't seen and the loop isn't stable. 246 247Now, this could simply be my lack of clean, newer, h/w (or, in general, 248a lack of h/w), but I would recommend the use of a hub if you want to 249stay with Private Loop and have more than one FC target. 250 251You should also note this can begin to be the basis for a shared SAN 252solution. For example, the above configuration can be extended to be: 253 254HOST 2552xxx <----------> +-------- 256 | Vixel | 257 | 1000 | 258 | +<---> Storage 259 | | 260 | +<---> Storage 261 | | 262 | +<---> Storage 263HOST | | 2642xxx <----------> +-------- 265 266However, note that there is nothing to mediate locking of devices, and 267it is also conceivable that the reboot of one host can, by causing 268a LIP storm, cause problems with the I/Os from the other host. 269(in other words, this topology hasn't really been made safe yet for 270this driver). 271 272D. You can repeat the topology in #B with a switch that is set to be 273in segmented loop mode. This avoids LIPs propagating where you don't 274want them to- and this makes for a much more reliable, if more expensive, 275SAN. 276 277E. The next level of complexity is a Switched Fabric. The following topology 278is good when you start to begin to get to want more performance. Private 279and Public Arbitrated Loop, while 100MB/s, is a shared medium. Direct 280connections to a switch can run full-duplex at full speed. 281 282HOST 2832xxx <----------> +--------- 284 | Brocade| 285 | 2400 | 286 | +<---> Storage 287 | | 288 | +<---> Storage 289 | | 290 | +<---> Storage 291HOST | | 2922xxx <----------> +--------- 293 294 295I would call this the best configuration available now. It can expand 296substantially if you cascade switches. 297 298There is a hard limit of about 253 devices for each Qlogic HBA- and the 299fabric login policy is simplistic (log them in as you find them). If 300somebody actually runs into a configuration that's larger, let me know 301and I'll work on some tools that would allow you some policy choices 302as to which would be interesting devices to actually connect to. 303 304 305