1Device Power Management 2 3Copyright (c) 2010 Rafael J. Wysocki <rjw@sisk.pl>, Novell Inc. 4Copyright (c) 2010 Alan Stern <stern@rowland.harvard.edu> 5 6 7Most of the code in Linux is device drivers, so most of the Linux power 8management (PM) code is also driver-specific. Most drivers will do very 9little; others, especially for platforms with small batteries (like cell 10phones), will do a lot. 11 12This writeup gives an overview of how drivers interact with system-wide 13power management goals, emphasizing the models and interfaces that are 14shared by everything that hooks up to the driver model core. Read it as 15background for the domain-specific work you'd do with any specific driver. 16 17 18Two Models for Device Power Management 19====================================== 20Drivers will use one or both of these models to put devices into low-power 21states: 22 23 System Sleep model: 24 Drivers can enter low-power states as part of entering system-wide 25 low-power states like "suspend" (also known as "suspend-to-RAM"), or 26 (mostly for systems with disks) "hibernation" (also known as 27 "suspend-to-disk"). 28 29 This is something that device, bus, and class drivers collaborate on 30 by implementing various role-specific suspend and resume methods to 31 cleanly power down hardware and software subsystems, then reactivate 32 them without loss of data. 33 34 Some drivers can manage hardware wakeup events, which make the system 35 leave the low-power state. This feature may be enabled or disabled 36 using the relevant /sys/devices/.../power/wakeup file (for Ethernet 37 drivers the ioctl interface used by ethtool may also be used for this 38 purpose); enabling it may cost some power usage, but let the whole 39 system enter low-power states more often. 40 41 Runtime Power Management model: 42 Devices may also be put into low-power states while the system is 43 running, independently of other power management activity in principle. 44 However, devices are not generally independent of each other (for 45 example, a parent device cannot be suspended unless all of its child 46 devices have been suspended). Moreover, depending on the bus type the 47 device is on, it may be necessary to carry out some bus-specific 48 operations on the device for this purpose. Devices put into low power 49 states at run time may require special handling during system-wide power 50 transitions (suspend or hibernation). 51 52 For these reasons not only the device driver itself, but also the 53 appropriate subsystem (bus type, device type or device class) driver and 54 the PM core are involved in runtime power management. As in the system 55 sleep power management case, they need to collaborate by implementing 56 various role-specific suspend and resume methods, so that the hardware 57 is cleanly powered down and reactivated without data or service loss. 58 59There's not a lot to be said about those low-power states except that they are 60very system-specific, and often device-specific. Also, that if enough devices 61have been put into low-power states (at runtime), the effect may be very similar 62to entering some system-wide low-power state (system sleep) ... and that 63synergies exist, so that several drivers using runtime PM might put the system 64into a state where even deeper power saving options are available. 65 66Most suspended devices will have quiesced all I/O: no more DMA or IRQs (except 67for wakeup events), no more data read or written, and requests from upstream 68drivers are no longer accepted. A given bus or platform may have different 69requirements though. 70 71Examples of hardware wakeup events include an alarm from a real time clock, 72network wake-on-LAN packets, keyboard or mouse activity, and media insertion 73or removal (for PCMCIA, MMC/SD, USB, and so on). 74 75 76Interfaces for Entering System Sleep States 77=========================================== 78There are programming interfaces provided for subsystems (bus type, device type, 79device class) and device drivers to allow them to participate in the power 80management of devices they are concerned with. These interfaces cover both 81system sleep and runtime power management. 82 83 84Device Power Management Operations 85---------------------------------- 86Device power management operations, at the subsystem level as well as at the 87device driver level, are implemented by defining and populating objects of type 88struct dev_pm_ops: 89 90struct dev_pm_ops { 91 int (*prepare)(struct device *dev); 92 void (*complete)(struct device *dev); 93 int (*suspend)(struct device *dev); 94 int (*resume)(struct device *dev); 95 int (*freeze)(struct device *dev); 96 int (*thaw)(struct device *dev); 97 int (*poweroff)(struct device *dev); 98 int (*restore)(struct device *dev); 99 int (*suspend_noirq)(struct device *dev); 100 int (*resume_noirq)(struct device *dev); 101 int (*freeze_noirq)(struct device *dev); 102 int (*thaw_noirq)(struct device *dev); 103 int (*poweroff_noirq)(struct device *dev); 104 int (*restore_noirq)(struct device *dev); 105 int (*runtime_suspend)(struct device *dev); 106 int (*runtime_resume)(struct device *dev); 107 int (*runtime_idle)(struct device *dev); 108}; 109 110This structure is defined in include/linux/pm.h and the methods included in it 111are also described in that file. Their roles will be explained in what follows. 112For now, it should be sufficient to remember that the last three methods are 113specific to runtime power management while the remaining ones are used during 114system-wide power transitions. 115 116There also is a deprecated "old" or "legacy" interface for power management 117operations available at least for some subsystems. This approach does not use 118struct dev_pm_ops objects and it is suitable only for implementing system sleep 119power management methods. Therefore it is not described in this document, so 120please refer directly to the source code for more information about it. 121 122 123Subsystem-Level Methods 124----------------------- 125The core methods to suspend and resume devices reside in struct dev_pm_ops 126pointed to by the pm member of struct bus_type, struct device_type and 127struct class. They are mostly of interest to the people writing infrastructure 128for buses, like PCI or USB, or device type and device class drivers. 129 130Bus drivers implement these methods as appropriate for the hardware and the 131drivers using it; PCI works differently from USB, and so on. Not many people 132write subsystem-level drivers; most driver code is a "device driver" that builds 133on top of bus-specific framework code. 134 135For more information on these driver calls, see the description later; 136they are called in phases for every device, respecting the parent-child 137sequencing in the driver model tree. 138 139 140/sys/devices/.../power/wakeup files 141----------------------------------- 142All devices in the driver model have two flags to control handling of wakeup 143events (hardware signals that can force the device and/or system out of a low 144power state). These flags are initialized by bus or device driver code using 145device_set_wakeup_capable() and device_set_wakeup_enable(), defined in 146include/linux/pm_wakeup.h. 147 148The "can_wakeup" flag just records whether the device (and its driver) can 149physically support wakeup events. The device_set_wakeup_capable() routine 150affects this flag. The "should_wakeup" flag controls whether the device should 151try to use its wakeup mechanism. device_set_wakeup_enable() affects this flag; 152for the most part drivers should not change its value. The initial value of 153should_wakeup is supposed to be false for the majority of devices; the major 154exceptions are power buttons, keyboards, and Ethernet adapters whose WoL 155(wake-on-LAN) feature has been set up with ethtool. 156 157Whether or not a device is capable of issuing wakeup events is a hardware 158matter, and the kernel is responsible for keeping track of it. By contrast, 159whether or not a wakeup-capable device should issue wakeup events is a policy 160decision, and it is managed by user space through a sysfs attribute: the 161power/wakeup file. User space can write the strings "enabled" or "disabled" to 162set or clear the should_wakeup flag, respectively. Reads from the file will 163return the corresponding string if can_wakeup is true, but if can_wakeup is 164false then reads will return an empty string, to indicate that the device 165doesn't support wakeup events. (But even though the file appears empty, writes 166will still affect the should_wakeup flag.) 167 168The device_may_wakeup() routine returns true only if both flags are set. 169Drivers should check this routine when putting devices in a low-power state 170during a system sleep transition, to see whether or not to enable the devices' 171wakeup mechanisms. However for runtime power management, wakeup events should 172be enabled whenever the device and driver both support them, regardless of the 173should_wakeup flag. 174 175 176/sys/devices/.../power/control files 177------------------------------------ 178Each device in the driver model has a flag to control whether it is subject to 179runtime power management. This flag, called runtime_auto, is initialized by the 180bus type (or generally subsystem) code using pm_runtime_allow() or 181pm_runtime_forbid(); the default is to allow runtime power management. 182 183The setting can be adjusted by user space by writing either "on" or "auto" to 184the device's power/control sysfs file. Writing "auto" calls pm_runtime_allow(), 185setting the flag and allowing the device to be runtime power-managed by its 186driver. Writing "on" calls pm_runtime_forbid(), clearing the flag, returning 187the device to full power if it was in a low-power state, and preventing the 188device from being runtime power-managed. User space can check the current value 189of the runtime_auto flag by reading the file. 190 191The device's runtime_auto flag has no effect on the handling of system-wide 192power transitions. In particular, the device can (and in the majority of cases 193should and will) be put into a low-power state during a system-wide transition 194to a sleep state even though its runtime_auto flag is clear. 195 196For more information about the runtime power management framework, refer to 197Documentation/power/runtime_pm.txt. 198 199 200Calling Drivers to Enter and Leave System Sleep States 201====================================================== 202When the system goes into a sleep state, each device's driver is asked to 203suspend the device by putting it into a state compatible with the target 204system state. That's usually some version of "off", but the details are 205system-specific. Also, wakeup-enabled devices will usually stay partly 206functional in order to wake the system. 207 208When the system leaves that low-power state, the device's driver is asked to 209resume it by returning it to full power. The suspend and resume operations 210always go together, and both are multi-phase operations. 211 212For simple drivers, suspend might quiesce the device using class code 213and then turn its hardware as "off" as possible during suspend_noirq. The 214matching resume calls would then completely reinitialize the hardware 215before reactivating its class I/O queues. 216 217More power-aware drivers might prepare the devices for triggering system wakeup 218events. 219 220 221Call Sequence Guarantees 222------------------------ 223To ensure that bridges and similar links needing to talk to a device are 224available when the device is suspended or resumed, the device tree is 225walked in a bottom-up order to suspend devices. A top-down order is 226used to resume those devices. 227 228The ordering of the device tree is defined by the order in which devices 229get registered: a child can never be registered, probed or resumed before 230its parent; and can't be removed or suspended after that parent. 231 232The policy is that the device tree should match hardware bus topology. 233(Or at least the control bus, for devices which use multiple busses.) 234In particular, this means that a device registration may fail if the parent of 235the device is suspending (i.e. has been chosen by the PM core as the next 236device to suspend) or has already suspended, as well as after all of the other 237devices have been suspended. Device drivers must be prepared to cope with such 238situations. 239 240 241System Power Management Phases 242------------------------------ 243Suspending or resuming the system is done in several phases. Different phases 244are used for standby or memory sleep states ("suspend-to-RAM") and the 245hibernation state ("suspend-to-disk"). Each phase involves executing callbacks 246for every device before the next phase begins. Not all busses or classes 247support all these callbacks and not all drivers use all the callbacks. The 248various phases always run after tasks have been frozen and before they are 249unfrozen. Furthermore, the *_noirq phases run at a time when IRQ handlers have 250been disabled (except for those marked with the IRQ_WAKEUP flag). 251 252Most phases use bus, type, and class callbacks (that is, methods defined in 253dev->bus->pm, dev->type->pm, and dev->class->pm). The prepare and complete 254phases are exceptions; they use only bus callbacks. When multiple callbacks 255are used in a phase, they are invoked in the order: <class, type, bus> during 256power-down transitions and in the opposite order during power-up transitions. 257For example, during the suspend phase the PM core invokes 258 259 dev->class->pm.suspend(dev); 260 dev->type->pm.suspend(dev); 261 dev->bus->pm.suspend(dev); 262 263before moving on to the next device, whereas during the resume phase the core 264invokes 265 266 dev->bus->pm.resume(dev); 267 dev->type->pm.resume(dev); 268 dev->class->pm.resume(dev); 269 270These callbacks may in turn invoke device- or driver-specific methods stored in 271dev->driver->pm, but they don't have to. 272 273 274Entering System Suspend 275----------------------- 276When the system goes into the standby or memory sleep state, the phases are: 277 278 prepare, suspend, suspend_noirq. 279 280 1. The prepare phase is meant to prevent races by preventing new devices 281 from being registered; the PM core would never know that all the 282 children of a device had been suspended if new children could be 283 registered at will. (By contrast, devices may be unregistered at any 284 time.) Unlike the other suspend-related phases, during the prepare 285 phase the device tree is traversed top-down. 286 287 The prepare phase uses only a bus callback. After the callback method 288 returns, no new children may be registered below the device. The method 289 may also prepare the device or driver in some way for the upcoming 290 system power transition, but it should not put the device into a 291 low-power state. 292 293 2. The suspend methods should quiesce the device to stop it from performing 294 I/O. They also may save the device registers and put it into the 295 appropriate low-power state, depending on the bus type the device is on, 296 and they may enable wakeup events. 297 298 3. The suspend_noirq phase occurs after IRQ handlers have been disabled, 299 which means that the driver's interrupt handler will not be called while 300 the callback method is running. The methods should save the values of 301 the device's registers that weren't saved previously and finally put the 302 device into the appropriate low-power state. 303 304 The majority of subsystems and device drivers need not implement this 305 callback. However, bus types allowing devices to share interrupt 306 vectors, like PCI, generally need it; otherwise a driver might encounter 307 an error during the suspend phase by fielding a shared interrupt 308 generated by some other device after its own device had been set to low 309 power. 310 311At the end of these phases, drivers should have stopped all I/O transactions 312(DMA, IRQs), saved enough state that they can re-initialize or restore previous 313state (as needed by the hardware), and placed the device into a low-power state. 314On many platforms they will gate off one or more clock sources; sometimes they 315will also switch off power supplies or reduce voltages. (Drivers supporting 316runtime PM may already have performed some or all of these steps.) 317 318If device_may_wakeup(dev) returns true, the device should be prepared for 319generating hardware wakeup signals to trigger a system wakeup event when the 320system is in the sleep state. For example, enable_irq_wake() might identify 321GPIO signals hooked up to a switch or other external hardware, and 322pci_enable_wake() does something similar for the PCI PME signal. 323 324If any of these callbacks returns an error, the system won't enter the desired 325low-power state. Instead the PM core will unwind its actions by resuming all 326the devices that were suspended. 327 328 329Leaving System Suspend 330---------------------- 331When resuming from standby or memory sleep, the phases are: 332 333 resume_noirq, resume, complete. 334 335 1. The resume_noirq callback methods should perform any actions needed 336 before the driver's interrupt handlers are invoked. This generally 337 means undoing the actions of the suspend_noirq phase. If the bus type 338 permits devices to share interrupt vectors, like PCI, the method should 339 bring the device and its driver into a state in which the driver can 340 recognize if the device is the source of incoming interrupts, if any, 341 and handle them correctly. 342 343 For example, the PCI bus type's ->pm.resume_noirq() puts the device into 344 the full-power state (D0 in the PCI terminology) and restores the 345 standard configuration registers of the device. Then it calls the 346 device driver's ->pm.resume_noirq() method to perform device-specific 347 actions. 348 349 2. The resume methods should bring the the device back to its operating 350 state, so that it can perform normal I/O. This generally involves 351 undoing the actions of the suspend phase. 352 353 3. The complete phase uses only a bus callback. The method should undo the 354 actions of the prepare phase. Note, however, that new children may be 355 registered below the device as soon as the resume callbacks occur; it's 356 not necessary to wait until the complete phase. 357 358At the end of these phases, drivers should be as functional as they were before 359suspending: I/O can be performed using DMA and IRQs, and the relevant clocks are 360gated on. Even if the device was in a low-power state before the system sleep 361because of runtime power management, afterwards it should be back in its 362full-power state. There are multiple reasons why it's best to do this; they are 363discussed in more detail in Documentation/power/runtime_pm.txt. 364 365However, the details here may again be platform-specific. For example, 366some systems support multiple "run" states, and the mode in effect at 367the end of resume might not be the one which preceded suspension. 368That means availability of certain clocks or power supplies changed, 369which could easily affect how a driver works. 370 371Drivers need to be able to handle hardware which has been reset since the 372suspend methods were called, for example by complete reinitialization. 373This may be the hardest part, and the one most protected by NDA'd documents 374and chip errata. It's simplest if the hardware state hasn't changed since 375the suspend was carried out, but that can't be guaranteed (in fact, it ususally 376is not the case). 377 378Drivers must also be prepared to notice that the device has been removed 379while the system was powered down, whenever that's physically possible. 380PCMCIA, MMC, USB, Firewire, SCSI, and even IDE are common examples of busses 381where common Linux platforms will see such removal. Details of how drivers 382will notice and handle such removals are currently bus-specific, and often 383involve a separate thread. 384 385These callbacks may return an error value, but the PM core will ignore such 386errors since there's nothing it can do about them other than printing them in 387the system log. 388 389 390Entering Hibernation 391-------------------- 392Hibernating the system is more complicated than putting it into the standby or 393memory sleep state, because it involves creating and saving a system image. 394Therefore there are more phases for hibernation, with a different set of 395callbacks. These phases always run after tasks have been frozen and memory has 396been freed. 397 398The general procedure for hibernation is to quiesce all devices (freeze), create 399an image of the system memory while everything is stable, reactivate all 400devices (thaw), write the image to permanent storage, and finally shut down the 401system (poweroff). The phases used to accomplish this are: 402 403 prepare, freeze, freeze_noirq, thaw_noirq, thaw, complete, 404 prepare, poweroff, poweroff_noirq 405 406 1. The prepare phase is discussed in the "Entering System Suspend" section 407 above. 408 409 2. The freeze methods should quiesce the device so that it doesn't generate 410 IRQs or DMA, and they may need to save the values of device registers. 411 However the device does not have to be put in a low-power state, and to 412 save time it's best not to do so. Also, the device should not be 413 prepared to generate wakeup events. 414 415 3. The freeze_noirq phase is analogous to the suspend_noirq phase discussed 416 above, except again that the device should not be put in a low-power 417 state and should not be allowed to generate wakeup events. 418 419At this point the system image is created. All devices should be inactive and 420the contents of memory should remain undisturbed while this happens, so that the 421image forms an atomic snapshot of the system state. 422 423 4. The thaw_noirq phase is analogous to the resume_noirq phase discussed 424 above. The main difference is that its methods can assume the device is 425 in the same state as at the end of the freeze_noirq phase. 426 427 5. The thaw phase is analogous to the resume phase discussed above. Its 428 methods should bring the device back to an operating state, so that it 429 can be used for saving the image if necessary. 430 431 6. The complete phase is discussed in the "Leaving System Suspend" section 432 above. 433 434At this point the system image is saved, and the devices then need to be 435prepared for the upcoming system shutdown. This is much like suspending them 436before putting the system into the standby or memory sleep state, and the phases 437are similar. 438 439 7. The prepare phase is discussed above. 440 441 8. The poweroff phase is analogous to the suspend phase. 442 443 9. The poweroff_noirq phase is analogous to the suspend_noirq phase. 444 445The poweroff and poweroff_noirq callbacks should do essentially the same things 446as the suspend and suspend_noirq callbacks. The only notable difference is that 447they need not store the device register values, because the registers should 448already have been stored during the freeze or freeze_noirq phases. 449 450 451Leaving Hibernation 452------------------- 453Resuming from hibernation is, again, more complicated than resuming from a sleep 454state in which the contents of main memory are preserved, because it requires 455a system image to be loaded into memory and the pre-hibernation memory contents 456to be restored before control can be passed back to the image kernel. 457 458Although in principle, the image might be loaded into memory and the 459pre-hibernation memory contents restored by the boot loader, in practice this 460can't be done because boot loaders aren't smart enough and there is no 461established protocol for passing the necessary information. So instead, the 462boot loader loads a fresh instance of the kernel, called the boot kernel, into 463memory and passes control to it in the usual way. Then the boot kernel reads 464the system image, restores the pre-hibernation memory contents, and passes 465control to the image kernel. Thus two different kernels are involved in 466resuming from hibernation. In fact, the boot kernel may be completely different 467from the image kernel: a different configuration and even a different version. 468This has important consequences for device drivers and their subsystems. 469 470To be able to load the system image into memory, the boot kernel needs to 471include at least a subset of device drivers allowing it to access the storage 472medium containing the image, although it doesn't need to include all of the 473drivers present in the image kernel. After the image has been loaded, the 474devices managed by the boot kernel need to be prepared for passing control back 475to the image kernel. This is very similar to the initial steps involved in 476creating a system image, and it is accomplished in the same way, using prepare, 477freeze, and freeze_noirq phases. However the devices affected by these phases 478are only those having drivers in the boot kernel; other devices will still be in 479whatever state the boot loader left them. 480 481Should the restoration of the pre-hibernation memory contents fail, the boot 482kernel would go through the "thawing" procedure described above, using the 483thaw_noirq, thaw, and complete phases, and then continue running normally. This 484happens only rarely. Most often the pre-hibernation memory contents are 485restored successfully and control is passed to the image kernel, which then 486becomes responsible for bringing the system back to the working state. 487 488To achieve this, the image kernel must restore the devices' pre-hibernation 489functionality. The operation is much like waking up from the memory sleep 490state, although it involves different phases: 491 492 restore_noirq, restore, complete 493 494 1. The restore_noirq phase is analogous to the resume_noirq phase. 495 496 2. The restore phase is analogous to the resume phase. 497 498 3. The complete phase is discussed above. 499 500The main difference from resume[_noirq] is that restore[_noirq] must assume the 501device has been accessed and reconfigured by the boot loader or the boot kernel. 502Consequently the state of the device may be different from the state remembered 503from the freeze and freeze_noirq phases. The device may even need to be reset 504and completely re-initialized. In many cases this difference doesn't matter, so 505the resume[_noirq] and restore[_norq] method pointers can be set to the same 506routines. Nevertheless, different callback pointers are used in case there is a 507situation where it actually matters. 508 509 510System Devices 511-------------- 512System devices (sysdevs) follow a slightly different API, which can be found in 513 514 include/linux/sysdev.h 515 drivers/base/sys.c 516 517System devices will be suspended with interrupts disabled, and after all other 518devices have been suspended. On resume, they will be resumed before any other 519devices, and also with interrupts disabled. These things occur in special 520"sysdev_driver" phases, which affect only system devices. 521 522Thus, after the suspend_noirq (or freeze_noirq or poweroff_noirq) phase, when 523the non-boot CPUs are all offline and IRQs are disabled on the remaining online 524CPU, then a sysdev_driver.suspend phase is carried out, and the system enters a 525sleep state (or a system image is created). During resume (or after the image 526has been created or loaded) a sysdev_driver.resume phase is carried out, IRQs 527are enabled on the only online CPU, the non-boot CPUs are enabled, and the 528resume_noirq (or thaw_noirq or restore_noirq) phase begins. 529 530Code to actually enter and exit the system-wide low power state sometimes 531involves hardware details that are only known to the boot firmware, and 532may leave a CPU running software (from SRAM or flash memory) that monitors 533the system and manages its wakeup sequence. 534 535 536Device Low Power (suspend) States 537--------------------------------- 538Device low-power states aren't standard. One device might only handle 539"on" and "off, while another might support a dozen different versions of 540"on" (how many engines are active?), plus a state that gets back to "on" 541faster than from a full "off". 542 543Some busses define rules about what different suspend states mean. PCI 544gives one example: after the suspend sequence completes, a non-legacy 545PCI device may not perform DMA or issue IRQs, and any wakeup events it 546issues would be issued through the PME# bus signal. Plus, there are 547several PCI-standard device states, some of which are optional. 548 549In contrast, integrated system-on-chip processors often use IRQs as the 550wakeup event sources (so drivers would call enable_irq_wake) and might 551be able to treat DMA completion as a wakeup event (sometimes DMA can stay 552active too, it'd only be the CPU and some peripherals that sleep). 553 554Some details here may be platform-specific. Systems may have devices that 555can be fully active in certain sleep states, such as an LCD display that's 556refreshed using DMA while most of the system is sleeping lightly ... and 557its frame buffer might even be updated by a DSP or other non-Linux CPU while 558the Linux control processor stays idle. 559 560Moreover, the specific actions taken may depend on the target system state. 561One target system state might allow a given device to be very operational; 562another might require a hard shut down with re-initialization on resume. 563And two different target systems might use the same device in different 564ways; the aforementioned LCD might be active in one product's "standby", 565but a different product using the same SOC might work differently. 566 567 568Power Management Notifiers 569-------------------------- 570There are some operations that cannot be carried out by the power management 571callbacks discussed above, because the callbacks occur too late or too early. 572To handle these cases, subsystems and device drivers may register power 573management notifiers that are called before tasks are frozen and after they have 574been thawed. Generally speaking, the PM notifiers are suitable for performing 575actions that either require user space to be available, or at least won't 576interfere with user space. 577 578For details refer to Documentation/power/notifiers.txt. 579 580 581Runtime Power Management 582======================== 583Many devices are able to dynamically power down while the system is still 584running. This feature is useful for devices that are not being used, and 585can offer significant power savings on a running system. These devices 586often support a range of runtime power states, which might use names such 587as "off", "sleep", "idle", "active", and so on. Those states will in some 588cases (like PCI) be partially constrained by the bus the device uses, and will 589usually include hardware states that are also used in system sleep states. 590 591A system-wide power transition can be started while some devices are in low 592power states due to runtime power management. The system sleep PM callbacks 593should recognize such situations and react to them appropriately, but the 594necessary actions are subsystem-specific. 595 596In some cases the decision may be made at the subsystem level while in other 597cases the device driver may be left to decide. In some cases it may be 598desirable to leave a suspended device in that state during a system-wide power 599transition, but in other cases the device must be put back into the full-power 600state temporarily, for example so that its system wakeup capability can be 601disabled. This all depends on the hardware and the design of the subsystem and 602device driver in question. 603 604During system-wide resume from a sleep state it's best to put devices into the 605full-power state, as explained in Documentation/power/runtime_pm.txt. Refer to 606that document for more information regarding this particular issue as well as 607for information on the device runtime power management framework in general. 608