1 2 How To Write Linux PCI Drivers 3 4 by Martin Mares <mj@ucw.cz> on 07-Feb-2000 5 updated by Grant Grundler <grundler@parisc-linux.org> on 23-Dec-2006 6 7~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 8The world of PCI is vast and full of (mostly unpleasant) surprises. 9Since each CPU architecture implements different chip-sets and PCI devices 10have different requirements (erm, "features"), the result is the PCI support 11in the Linux kernel is not as trivial as one would wish. This short paper 12tries to introduce all potential driver authors to Linux APIs for 13PCI device drivers. 14 15A more complete resource is the third edition of "Linux Device Drivers" 16by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman. 17LDD3 is available for free (under Creative Commons License) from: 18 19 http://lwn.net/Kernel/LDD3/ 20 21However, keep in mind that all documents are subject to "bit rot". 22Refer to the source code if things are not working as described here. 23 24Please send questions/comments/patches about Linux PCI API to the 25"Linux PCI" <linux-pci@atrey.karlin.mff.cuni.cz> mailing list. 26 27 28 290. Structure of PCI drivers 30~~~~~~~~~~~~~~~~~~~~~~~~~~~ 31PCI drivers "discover" PCI devices in a system via pci_register_driver(). 32Actually, it's the other way around. When the PCI generic code discovers 33a new device, the driver with a matching "description" will be notified. 34Details on this below. 35 36pci_register_driver() leaves most of the probing for devices to 37the PCI layer and supports online insertion/removal of devices [thus 38supporting hot-pluggable PCI, CardBus, and Express-Card in a single driver]. 39pci_register_driver() call requires passing in a table of function 40pointers and thus dictates the high level structure of a driver. 41 42Once the driver knows about a PCI device and takes ownership, the 43driver generally needs to perform the following initialization: 44 45 Enable the device 46 Request MMIO/IOP resources 47 Set the DMA mask size (for both coherent and streaming DMA) 48 Allocate and initialize shared control data (pci_allocate_coherent()) 49 Access device configuration space (if needed) 50 Register IRQ handler (request_irq()) 51 Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip) 52 Enable DMA/processing engines 53 54When done using the device, and perhaps the module needs to be unloaded, 55the driver needs to take the follow steps: 56 Disable the device from generating IRQs 57 Release the IRQ (free_irq()) 58 Stop all DMA activity 59 Release DMA buffers (both streaming and coherent) 60 Unregister from other subsystems (e.g. scsi or netdev) 61 Release MMIO/IOP resources 62 Disable the device 63 64Most of these topics are covered in the following sections. 65For the rest look at LDD3 or <linux/pci.h> . 66 67If the PCI subsystem is not configured (CONFIG_PCI is not set), most of 68the PCI functions described below are defined as inline functions either 69completely empty or just returning an appropriate error codes to avoid 70lots of ifdefs in the drivers. 71 72 73 741. pci_register_driver() call 75~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 76 77PCI device drivers call pci_register_driver() during their 78initialization with a pointer to a structure describing the driver 79(struct pci_driver): 80 81 field name Description 82 ---------- ------------------------------------------------------ 83 id_table Pointer to table of device ID's the driver is 84 interested in. Most drivers should export this 85 table using MODULE_DEVICE_TABLE(pci,...). 86 87 probe This probing function gets called (during execution 88 of pci_register_driver() for already existing 89 devices or later if a new device gets inserted) for 90 all PCI devices which match the ID table and are not 91 "owned" by the other drivers yet. This function gets 92 passed a "struct pci_dev *" for each device whose 93 entry in the ID table matches the device. The probe 94 function returns zero when the driver chooses to 95 take "ownership" of the device or an error code 96 (negative number) otherwise. 97 The probe function always gets called from process 98 context, so it can sleep. 99 100 remove The remove() function gets called whenever a device 101 being handled by this driver is removed (either during 102 deregistration of the driver or when it's manually 103 pulled out of a hot-pluggable slot). 104 The remove function always gets called from process 105 context, so it can sleep. 106 107 suspend Put device into low power state. 108 suspend_late Put device into low power state. 109 110 resume_early Wake device from low power state. 111 resume Wake device from low power state. 112 113 (Please see Documentation/power/pci.txt for descriptions 114 of PCI Power Management and the related functions.) 115 116 enable_wake Enable device to generate wake events from a low power 117 state. 118 119 shutdown Hook into reboot_notifier_list (kernel/sys.c). 120 Intended to stop any idling DMA operations. 121 Useful for enabling wake-on-lan (NIC) or changing 122 the power state of a device before reboot. 123 e.g. drivers/net/e100.c. 124 125 err_handler See Documentation/pci-error-recovery.txt 126 127 128The ID table is an array of struct pci_device_id entries ending with an 129all-zero entry. Each entry consists of: 130 131 vendor,device Vendor and device ID to match (or PCI_ANY_ID) 132 133 subvendor, Subsystem vendor and device ID to match (or PCI_ANY_ID) 134 subdevice, 135 136 class Device class, subclass, and "interface" to match. 137 See Appendix D of the PCI Local Bus Spec or 138 include/linux/pci_ids.h for a full list of classes. 139 Most drivers do not need to specify class/class_mask 140 as vendor/device is normally sufficient. 141 142 class_mask limit which sub-fields of the class field are compared. 143 See drivers/scsi/sym53c8xx_2/ for example of usage. 144 145 driver_data Data private to the driver. 146 Most drivers don't need to use driver_data field. 147 Best practice is to use driver_data as an index 148 into a static list of equivalent device types, 149 instead of using it as a pointer. 150 151 152Most drivers only need PCI_DEVICE() or PCI_DEVICE_CLASS() to set up 153a pci_device_id table. 154 155New PCI IDs may be added to a device driver pci_ids table at runtime 156as shown below: 157 158echo "vendor device subvendor subdevice class class_mask driver_data" > \ 159/sys/bus/pci/drivers/{driver}/new_id 160 161All fields are passed in as hexadecimal values (no leading 0x). 162The vendor and device fields are mandatory, the others are optional. Users 163need pass only as many optional fields as necessary: 164 o subvendor and subdevice fields default to PCI_ANY_ID (FFFFFFFF) 165 o class and classmask fields default to 0 166 o driver_data defaults to 0UL. 167 168Once added, the driver probe routine will be invoked for any unclaimed 169PCI devices listed in its (newly updated) pci_ids list. 170 171When the driver exits, it just calls pci_unregister_driver() and the PCI layer 172automatically calls the remove hook for all devices handled by the driver. 173 174 1751.1 "Attributes" for driver functions/data 176 177Please mark the initialization and cleanup functions where appropriate 178(the corresponding macros are defined in <linux/init.h>): 179 180 __init Initialization code. Thrown away after the driver 181 initializes. 182 __exit Exit code. Ignored for non-modular drivers. 183 184 185 __devinit Device initialization code. 186 Identical to __init if the kernel is not compiled 187 with CONFIG_HOTPLUG, normal function otherwise. 188 __devexit The same for __exit. 189 190Tips on when/where to use the above attributes: 191 o The module_init()/module_exit() functions (and all 192 initialization functions called _only_ from these) 193 should be marked __init/__exit. 194 195 o Do not mark the struct pci_driver. 196 197 o The ID table array should be marked __devinitdata. 198 199 o The probe() and remove() functions should be marked __devinit 200 and __devexit respectively. All initialization functions 201 exclusively called by the probe() routine, can be marked __devinit. 202 Ditto for remove() and __devexit. 203 204 o If mydriver_remove() is marked with __devexit(), then all address 205 references to mydriver_remove must use __devexit_p(mydriver_remove) 206 (in the struct pci_driver declaration for example). 207 __devexit_p() will generate the function name _or_ NULL if the 208 function will be discarded. For an example, see drivers/net/tg3.c. 209 210 o Do NOT mark a function if you are not sure which mark to use. 211 Better to not mark the function than mark the function wrong. 212 213 214 2152. How to find PCI devices manually 216~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 217 218PCI drivers should have a really good reason for not using the 219pci_register_driver() interface to search for PCI devices. 220The main reason PCI devices are controlled by multiple drivers 221is because one PCI device implements several different HW services. 222E.g. combined serial/parallel port/floppy controller. 223 224A manual search may be performed using the following constructs: 225 226Searching by vendor and device ID: 227 228 struct pci_dev *dev = NULL; 229 while (dev = pci_get_device(VENDOR_ID, DEVICE_ID, dev)) 230 configure_device(dev); 231 232Searching by class ID (iterate in a similar way): 233 234 pci_get_class(CLASS_ID, dev) 235 236Searching by both vendor/device and subsystem vendor/device ID: 237 238 pci_get_subsys(VENDOR_ID,DEVICE_ID, SUBSYS_VENDOR_ID, SUBSYS_DEVICE_ID, dev). 239 240You can use the constant PCI_ANY_ID as a wildcard replacement for 241VENDOR_ID or DEVICE_ID. This allows searching for any device from a 242specific vendor, for example. 243 244These functions are hotplug-safe. They increment the reference count on 245the pci_dev that they return. You must eventually (possibly at module unload) 246decrement the reference count on these devices by calling pci_dev_put(). 247 248 249 2503. Device Initialization Steps 251~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 252 253As noted in the introduction, most PCI drivers need the following steps 254for device initialization: 255 256 Enable the device 257 Request MMIO/IOP resources 258 Set the DMA mask size (for both coherent and streaming DMA) 259 Allocate and initialize shared control data (pci_allocate_coherent()) 260 Access device configuration space (if needed) 261 Register IRQ handler (request_irq()) 262 Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip) 263 Enable DMA/processing engines. 264 265The driver can access PCI config space registers at any time. 266(Well, almost. When running BIST, config space can go away...but 267that will just result in a PCI Bus Master Abort and config reads 268will return garbage). 269 270 2713.1 Enable the PCI device 272~~~~~~~~~~~~~~~~~~~~~~~~~ 273Before touching any device registers, the driver needs to enable 274the PCI device by calling pci_enable_device(). This will: 275 o wake up the device if it was in suspended state, 276 o allocate I/O and memory regions of the device (if BIOS did not), 277 o allocate an IRQ (if BIOS did not). 278 279NOTE: pci_enable_device() can fail! Check the return value. 280NOTE2: Also see pci_enable_device_bars() below. Drivers can 281 attempt to enable only a subset of BARs they need. 282 283[ OS BUG: we don't check resource allocations before enabling those 284 resources. The sequence would make more sense if we called 285 pci_request_resources() before calling pci_enable_device(). 286 Currently, the device drivers can't detect the bug when when two 287 devices have been allocated the same range. This is not a common 288 problem and unlikely to get fixed soon. 289 290 This has been discussed before but not changed as of 2.6.19: 291 http://lkml.org/lkml/2006/3/2/194 292] 293 294pci_set_master() will enable DMA by setting the bus master bit 295in the PCI_COMMAND register. It also fixes the latency timer value if 296it's set to something bogus by the BIOS. 297 298If the PCI device can use the PCI Memory-Write-Invalidate transaction, 299call pci_set_mwi(). This enables the PCI_COMMAND bit for Mem-Wr-Inval 300and also ensures that the cache line size register is set correctly. 301Check the return value of pci_set_mwi() as not all architectures 302or chip-sets may support Memory-Write-Invalidate. 303 304 3053.2 Request MMIO/IOP resources 306~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 307Memory (MMIO), and I/O port addresses should NOT be read directly 308from the PCI device config space. Use the values in the pci_dev structure 309as the PCI "bus address" might have been remapped to a "host physical" 310address by the arch/chip-set specific kernel support. 311 312See Documentation/IO-mapping.txt for how to access device registers 313or device memory. 314 315The device driver needs to call pci_request_region() to verify 316no other device is already using the same address resource. 317Conversely, drivers should call pci_release_region() AFTER 318calling pci_disable_device(). 319The idea is to prevent two devices colliding on the same address range. 320 321[ See OS BUG comment above. Currently (2.6.19), The driver can only 322 determine MMIO and IO Port resource availability _after_ calling 323 pci_enable_device(). ] 324 325Generic flavors of pci_request_region() are request_mem_region() 326(for MMIO ranges) and request_region() (for IO Port ranges). 327Use these for address resources that are not described by "normal" PCI 328BARs. 329 330Also see pci_request_selected_regions() below. 331 332 3333.3 Set the DMA mask size 334~~~~~~~~~~~~~~~~~~~~~~~~~ 335[ If anything below doesn't make sense, please refer to 336 Documentation/DMA-API.txt. This section is just a reminder that 337 drivers need to indicate DMA capabilities of the device and is not 338 an authoritative source for DMA interfaces. ] 339 340While all drivers should explicitly indicate the DMA capability 341(e.g. 32 or 64 bit) of the PCI bus master, devices with more than 34232-bit bus master capability for streaming data need the driver 343to "register" this capability by calling pci_set_dma_mask() with 344appropriate parameters. In general this allows more efficient DMA 345on systems where System RAM exists above 4G _physical_ address. 346 347Drivers for all PCI-X and PCIe compliant devices must call 348pci_set_dma_mask() as they are 64-bit DMA devices. 349 350Similarly, drivers must also "register" this capability if the device 351can directly address "consistent memory" in System RAM above 4G physical 352address by calling pci_set_consistent_dma_mask(). 353Again, this includes drivers for all PCI-X and PCIe compliant devices. 354Many 64-bit "PCI" devices (before PCI-X) and some PCI-X devices are 35564-bit DMA capable for payload ("streaming") data but not control 356("consistent") data. 357 358 3593.4 Setup shared control data 360~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 361Once the DMA masks are set, the driver can allocate "consistent" (a.k.a. shared) 362memory. See Documentation/DMA-API.txt for a full description of 363the DMA APIs. This section is just a reminder that it needs to be done 364before enabling DMA on the device. 365 366 3673.5 Initialize device registers 368~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 369Some drivers will need specific "capability" fields programmed 370or other "vendor specific" register initialized or reset. 371E.g. clearing pending interrupts. 372 373 3743.6 Register IRQ handler 375~~~~~~~~~~~~~~~~~~~~~~~~ 376While calling request_irq() is the last step described here, 377this is often just another intermediate step to initialize a device. 378This step can often be deferred until the device is opened for use. 379 380All interrupt handlers for IRQ lines should be registered with IRQF_SHARED 381and use the devid to map IRQs to devices (remember that all PCI IRQ lines 382can be shared). 383 384request_irq() will associate an interrupt handler and device handle 385with an interrupt number. Historically interrupt numbers represent 386IRQ lines which run from the PCI device to the Interrupt controller. 387With MSI and MSI-X (more below) the interrupt number is a CPU "vector". 388 389request_irq() also enables the interrupt. Make sure the device is 390quiesced and does not have any interrupts pending before registering 391the interrupt handler. 392 393MSI and MSI-X are PCI capabilities. Both are "Message Signaled Interrupts" 394which deliver interrupts to the CPU via a DMA write to a Local APIC. 395The fundamental difference between MSI and MSI-X is how multiple 396"vectors" get allocated. MSI requires contiguous blocks of vectors 397while MSI-X can allocate several individual ones. 398 399MSI capability can be enabled by calling pci_enable_msi() or 400pci_enable_msix() before calling request_irq(). This causes 401the PCI support to program CPU vector data into the PCI device 402capability registers. 403 404If your PCI device supports both, try to enable MSI-X first. 405Only one can be enabled at a time. Many architectures, chip-sets, 406or BIOSes do NOT support MSI or MSI-X and the call to pci_enable_msi/msix 407will fail. This is important to note since many drivers have 408two (or more) interrupt handlers: one for MSI/MSI-X and another for IRQs. 409They choose which handler to register with request_irq() based on the 410return value from pci_enable_msi/msix(). 411 412There are (at least) two really good reasons for using MSI: 4131) MSI is an exclusive interrupt vector by definition. 414 This means the interrupt handler doesn't have to verify 415 its device caused the interrupt. 416 4172) MSI avoids DMA/IRQ race conditions. DMA to host memory is guaranteed 418 to be visible to the host CPU(s) when the MSI is delivered. This 419 is important for both data coherency and avoiding stale control data. 420 This guarantee allows the driver to omit MMIO reads to flush 421 the DMA stream. 422 423See drivers/infiniband/hw/mthca/ or drivers/net/tg3.c for examples 424of MSI/MSI-X usage. 425 426 427 4284. PCI device shutdown 429~~~~~~~~~~~~~~~~~~~~~~~ 430 431When a PCI device driver is being unloaded, most of the following 432steps need to be performed: 433 434 Disable the device from generating IRQs 435 Release the IRQ (free_irq()) 436 Stop all DMA activity 437 Release DMA buffers (both streaming and consistent) 438 Unregister from other subsystems (e.g. scsi or netdev) 439 Disable device from responding to MMIO/IO Port addresses 440 Release MMIO/IO Port resource(s) 441 442 4434.1 Stop IRQs on the device 444~~~~~~~~~~~~~~~~~~~~~~~~~~~ 445How to do this is chip/device specific. If it's not done, it opens 446the possibility of a "screaming interrupt" if (and only if) 447the IRQ is shared with another device. 448 449When the shared IRQ handler is "unhooked", the remaining devices 450using the same IRQ line will still need the IRQ enabled. Thus if the 451"unhooked" device asserts IRQ line, the system will respond assuming 452it was one of the remaining devices asserted the IRQ line. Since none 453of the other devices will handle the IRQ, the system will "hang" until 454it decides the IRQ isn't going to get handled and masks the IRQ (100,000 455iterations later). Once the shared IRQ is masked, the remaining devices 456will stop functioning properly. Not a nice situation. 457 458This is another reason to use MSI or MSI-X if it's available. 459MSI and MSI-X are defined to be exclusive interrupts and thus 460are not susceptible to the "screaming interrupt" problem. 461 462 4634.2 Release the IRQ 464~~~~~~~~~~~~~~~~~~~ 465Once the device is quiesced (no more IRQs), one can call free_irq(). 466This function will return control once any pending IRQs are handled, 467"unhook" the drivers IRQ handler from that IRQ, and finally release 468the IRQ if no one else is using it. 469 470 4714.3 Stop all DMA activity 472~~~~~~~~~~~~~~~~~~~~~~~~~ 473It's extremely important to stop all DMA operations BEFORE attempting 474to deallocate DMA control data. Failure to do so can result in memory 475corruption, hangs, and on some chip-sets a hard crash. 476 477Stopping DMA after stopping the IRQs can avoid races where the 478IRQ handler might restart DMA engines. 479 480While this step sounds obvious and trivial, several "mature" drivers 481didn't get this step right in the past. 482 483 4844.4 Release DMA buffers 485~~~~~~~~~~~~~~~~~~~~~~~ 486Once DMA is stopped, clean up streaming DMA first. 487I.e. unmap data buffers and return buffers to "upstream" 488owners if there is one. 489 490Then clean up "consistent" buffers which contain the control data. 491 492See Documentation/DMA-API.txt for details on unmapping interfaces. 493 494 4954.5 Unregister from other subsystems 496~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 497Most low level PCI device drivers support some other subsystem 498like USB, ALSA, SCSI, NetDev, Infiniband, etc. Make sure your 499driver isn't losing resources from that other subsystem. 500If this happens, typically the symptom is an Oops (panic) when 501the subsystem attempts to call into a driver that has been unloaded. 502 503 5044.6 Disable Device from responding to MMIO/IO Port addresses 505~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 506io_unmap() MMIO or IO Port resources and then call pci_disable_device(). 507This is the symmetric opposite of pci_enable_device(). 508Do not access device registers after calling pci_disable_device(). 509 510 5114.7 Release MMIO/IO Port Resource(s) 512~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 513Call pci_release_region() to mark the MMIO or IO Port range as available. 514Failure to do so usually results in the inability to reload the driver. 515 516 517 5185. How to access PCI config space 519~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 520 521You can use pci_(read|write)_config_(byte|word|dword) to access the config 522space of a device represented by struct pci_dev *. All these functions return 0 523when successful or an error code (PCIBIOS_...) which can be translated to a text 524string by pcibios_strerror. Most drivers expect that accesses to valid PCI 525devices don't fail. 526 527If you don't have a struct pci_dev available, you can call 528pci_bus_(read|write)_config_(byte|word|dword) to access a given device 529and function on that bus. 530 531If you access fields in the standard portion of the config header, please 532use symbolic names of locations and bits declared in <linux/pci.h>. 533 534If you need to access Extended PCI Capability registers, just call 535pci_find_capability() for the particular capability and it will find the 536corresponding register block for you. 537 538 539 5406. Other interesting functions 541~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 542 543pci_find_slot() Find pci_dev corresponding to given bus and 544 slot numbers. 545pci_set_power_state() Set PCI Power Management state (0=D0 ... 3=D3) 546pci_find_capability() Find specified capability in device's capability 547 list. 548pci_resource_start() Returns bus start address for a given PCI region 549pci_resource_end() Returns bus end address for a given PCI region 550pci_resource_len() Returns the byte length of a PCI region 551pci_set_drvdata() Set private driver data pointer for a pci_dev 552pci_get_drvdata() Return private driver data pointer for a pci_dev 553pci_set_mwi() Enable Memory-Write-Invalidate transactions. 554pci_clear_mwi() Disable Memory-Write-Invalidate transactions. 555 556 557 5587. Miscellaneous hints 559~~~~~~~~~~~~~~~~~~~~~~ 560 561When displaying PCI device names to the user (for example when a driver wants 562to tell the user what card has it found), please use pci_name(pci_dev). 563 564Always refer to the PCI devices by a pointer to the pci_dev structure. 565All PCI layer functions use this identification and it's the only 566reasonable one. Don't use bus/slot/function numbers except for very 567special purposes -- on systems with multiple primary buses their semantics 568can be pretty complex. 569 570Don't try to turn on Fast Back to Back writes in your driver. All devices 571on the bus need to be capable of doing it, so this is something which needs 572to be handled by platform and generic code, not individual drivers. 573 574 575 5768. Vendor and device identifications 577~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 578 579One is not not required to add new device ids to include/linux/pci_ids.h. 580Please add PCI_VENDOR_ID_xxx for vendors and a hex constant for device ids. 581 582PCI_VENDOR_ID_xxx constants are re-used. The device ids are arbitrary 583hex numbers (vendor controlled) and normally used only in a single 584location, the pci_device_id table. 585 586Please DO submit new vendor/device ids to pciids.sourceforge.net project. 587 588 589 5909. Obsolete functions 591~~~~~~~~~~~~~~~~~~~~~ 592 593There are several functions which you might come across when trying to 594port an old driver to the new PCI interface. They are no longer present 595in the kernel as they aren't compatible with hotplug or PCI domains or 596having sane locking. 597 598pci_find_device() Superseded by pci_get_device() 599pci_find_subsys() Superseded by pci_get_subsys() 600pci_find_slot() Superseded by pci_get_slot() 601 602 603The alternative is the traditional PCI device driver that walks PCI 604device lists. This is still possible but discouraged. 605 606 607 60810. pci_enable_device_bars() and Legacy I/O Port space 609~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 610 611Large servers may not be able to provide I/O port resources to all PCI 612devices. I/O Port space is only 64KB on Intel Architecture[1] and is 613likely also fragmented since the I/O base register of PCI-to-PCI 614bridge will usually be aligned to a 4KB boundary[2]. On such systems, 615pci_enable_device() and pci_request_region() will fail when 616attempting to enable I/O Port regions that don't have I/O Port 617resources assigned. 618 619Fortunately, many PCI devices which request I/O Port resources also 620provide access to the same registers via MMIO BARs. These devices can 621be handled without using I/O port space and the drivers typically 622offer a CONFIG_ option to only use MMIO regions 623(e.g. CONFIG_TULIP_MMIO). PCI devices typically provide I/O port 624interface for legacy OSes and will work when I/O port resources are not 625assigned. The "PCI Local Bus Specification Revision 3.0" discusses 626this on p.44, "IMPLEMENTATION NOTE". 627 628If your PCI device driver doesn't need I/O port resources assigned to 629I/O Port BARs, you should use pci_enable_device_bars() instead of 630pci_enable_device() in order not to enable I/O port regions for the 631corresponding devices. In addition, you should use 632pci_request_selected_regions() and pci_release_selected_regions() 633instead of pci_request_regions()/pci_release_regions() in order not to 634request/release I/O port regions for the corresponding devices. 635 636[1] Some systems support 64KB I/O port space per PCI segment. 637[2] Some PCI-to-PCI bridges support optional 1KB aligned I/O base. 638 639 640 64111. MMIO Space and "Write Posting" 642~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 643 644Converting a driver from using I/O Port space to using MMIO space 645often requires some additional changes. Specifically, "write posting" 646needs to be handled. Many drivers (e.g. tg3, acenic, sym53c8xx_2) 647already do this. I/O Port space guarantees write transactions reach the PCI 648device before the CPU can continue. Writes to MMIO space allow the CPU 649to continue before the transaction reaches the PCI device. HW weenies 650call this "Write Posting" because the write completion is "posted" to 651the CPU before the transaction has reached its destination. 652 653Thus, timing sensitive code should add readl() where the CPU is 654expected to wait before doing other work. The classic "bit banging" 655sequence works fine for I/O Port space: 656 657 for (i = 8; --i; val >>= 1) { 658 outb(val & 1, ioport_reg); /* write bit */ 659 udelay(10); 660 } 661 662The same sequence for MMIO space should be: 663 664 for (i = 8; --i; val >>= 1) { 665 writeb(val & 1, mmio_reg); /* write bit */ 666 readb(safe_mmio_reg); /* flush posted write */ 667 udelay(10); 668 } 669 670It is important that "safe_mmio_reg" not have any side effects that 671interferes with the correct operation of the device. 672 673Another case to watch out for is when resetting a PCI device. Use PCI 674Configuration space reads to flush the writel(). This will gracefully 675handle the PCI master abort on all platforms if the PCI device is 676expected to not respond to a readl(). Most x86 platforms will allow 677MMIO reads to master abort (a.k.a. "Soft Fail") and return garbage 678(e.g. ~0). But many RISC platforms will crash (a.k.a."Hard Fail"). 679