#
300207 |
|
19-May-2016 |
ken |
Add support for managing Shingled Magnetic Recording (SMR) drives.
This change includes support for SCSI SMR drives (which conform to the Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to the Zoned ATA Command Set or ZAC spec) behind SAS expanders.
This includes full management support through the GEOM BIO interface, and through a new userland utility, zonectl(8), and through camcontrol(8).
This is now ready for filesystems to use to detect and manage zoned drives. (There is no work in progress that I know of to use this for ZFS or UFS, if anyone is interested, let me know and I may have some suggestions.)
Also, improve ATA command passthrough and dispatch support, both via ATA and ATA passthrough over SCSI.
Also, add support to camcontrol(8) for the ATA Extended Power Conditions feature set. You can now manage ATA device power states, and set various idle time thresholds for a drive to enter lower power states.
Note that this change cannot be MFCed in full, because it depends on changes to the struct bio API that break compatilibity. In order to avoid breaking the stable API, only changes that don't touch or depend on the struct bio changes can be merged. For example, the camcontrol(8) changes don't depend on the new bio API, but zonectl(8) and the probe changes to the da(4) and ada(4) drivers do depend on it.
Also note that the SMR changes have not yet been tested with an actual SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports ZBC to ZAC translation. I have not yet gotten a suitable drive or SAT layer, so any testing help would be appreciated. These changes have been tested with Seagate Host Aware SATA drives attached to both SAS and SATA controllers. Also, I do not have any SATA Host Managed devices, and I suspect that it may take additional (hopefully minor) changes to support them.
Thanks to Seagate for supplying the test hardware and answering questions.
sbin/camcontrol/Makefile: Add epc.c and zone.c.
sbin/camcontrol/camcontrol.8: Document the zone and epc subcommands.
sbin/camcontrol/camcontrol.c: Add the zone and epc subcommands.
Add auxiliary register support to build_ata_cmd(). Make sure to set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA flags as appropriate for ATA commands.
Add a new get_ata_status() function to parse ATA result from SCSI sense descriptors (for ATA passthrough over SCSI) and ATA I/O requests.
sbin/camcontrol/camcontrol.h: Update the build_ata_cmd() prototype
Add get_ata_status(), zone(), and epc().
sbin/camcontrol/epc.c: Support for ATA Extended Power Conditions features. This includes support for all features documented in the ACS-4 Revision 12 specification from t13.org (dated February 18, 2016).
The EPC feature set allows putting a drive into a power power mode immediately, or setting timeouts so that the drive will automatically enter progressively lower power states after various idle times.
sbin/camcontrol/fwdownload.c: Update the firmware download code for the new build_ata_cmd() arguments.
sbin/camcontrol/zone.c: Implement support for Shingled Magnetic Recording (SMR) drives via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA Command Set (ZAC).
These specs were developed in concert, and are functionally identical. The primary differences are due to SCSI and ATA differences. (SCSI is big endian, ATA is little endian, for example.)
This includes support for all commands defined in the ZBC and ZAC specs.
sys/cam/ata/ata_all.c: Decode a number of additional ATA command names in ata_op_string().
Add a new CCB building function, ata_read_log().
Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building functions. These support both DMA and NCQ encapsulation.
sys/cam/ata/ata_all.h: Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and ata_zac_mgmt_in().
sys/cam/ata/ata_da.c: Revamp the ada(4) driver to support zoned devices.
Add four new probe states to gather information needed for zone support.
Add a new adasetflags() function to avoid duplication of large blocks of flag setting between the async handler and register functions.
Add new sysctl variables that describe zone support and paramters.
Add support for the new BIO_ZONE bio, and all of its subcommands: DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP, DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.
sys/cam/scsi/scsi_all.c: Add command descriptions for the ZBC IN/OUT commands.
Add descriptions for ZBC Host Managed devices.
Add a new function, scsi_ata_pass() to do ATA passthrough over SCSI. This will eventually replace scsi_ata_pass_16() -- it can create the 12, 16, and 32-byte variants of the ATA PASS-THROUGH command, and supports setting all of the registers defined as of SAT-4, Revision 5 (March 11, 2016).
Change scsi_ata_identify() to use scsi_ata_pass() instead of scsi_ata_pass_16().
Add a new scsi_ata_read_log() function to facilitate reading ATA logs via SCSI.
sys/cam/scsi/scsi_all.h: Add the new ATA PASS-THROUGH(32) command CDB. Add extended and variable CDB opcodes.
Add Zoned Block Device Characteristics VPD page.
Add ATA Return SCSI sense descriptor.
Add prototypes for scsi_ata_read_log() and scsi_ata_pass().
sys/cam/scsi/scsi_da.c: Revamp the da(4) driver to support zoned devices.
Add five new probe states, four of which are needed for ATA devices.
Add five new sysctl variables that describe zone support and parameters.
The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC devices when they are attached via a SCSI to ATA Translation (SAT) layer. Since ZBC -> ZAC translation is a new feature in the T10 SAT-4 spec, most SATA drives will be supported via ATA commands sent via the SCSI ATA PASS-THROUGH command. The da(4) driver will prefer the ZBC interface, if it is available, for performance reasons, but will use the ATA PASS-THROUGH interface to the ZAC command set if the SAT layer doesn't support translation yet. As I mentioned above, ZBC command support is untested.
Add support for the new BIO_ZONE bio, and all of its subcommands: DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP, DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.
Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.
Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB building functions. Note that these have return values, unlike almost all other CCB building functions in CAM. The reason is that they can fail, depending upon the particular combination of input parameters. The primary failure case is if the user wants NCQ, but fails to specify additional CDB storage. NCQ requires using the 32-byte version of the SCSI ATA PASS-THROUGH command, and the current CAM CDB size is 16 bytes.
sys/cam/scsi/scsi_da.h: Add ZBC IN and ZBC OUT CDBs and opcodes.
Add SCSI Report Zones data structures.
Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and scsi_ata_zac_mgmt_in() prototypes.
sys/dev/ahci/ahci.c: Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.
ahci_setup_fis() previously set the top bits of the sector count register in the FIS to 0 for FPDMA commands. This is okay for read and write, because the PRIO field is in the only thing in those bits, and we don't implement that further up the stack.
But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that byte, so it needs to be transmitted to the drive.
In ahci_setup_fis(), always set the the top 8 bits of the sector count register. We need it in both the standard and NCQ / FPDMA cases.
sys/geom/eli/g_eli.c: Pass BIO_ZONE commands through the GELI class.
sys/geom/geom.h: Add g_io_zonecmd() prototype.
sys/geom/geom_dev.c: Add new DIOCZONECMD ioctl, which allows sending zone commands to disks.
sys/geom/geom_disk.c: Add support for BIO_ZONE commands.
sys/geom/geom_disk.h: Add a new flag, DISKFLAG_CANZONE, that indicates that a given GEOM disk client can handle BIO_ZONE commands.
sys/geom/geom_io.c: Add a new function, g_io_zonecmd(), that handles execution of BIO_ZONE commands.
Add permissions check for BIO_ZONE commands.
Add command decoding for BIO_ZONE commands.
sys/geom/geom_subr.c: Add DDB command decoding for BIO_ZONE commands.
sys/kern/subr_devstat.c: Record statistics for REPORT ZONES commands. Note that the number of bytes transferred for REPORT ZONES won't quite match what is received from the harware. This is because we're necessarily counting bytes coming from the da(4) / ada(4) drivers, which are using the disk_zone.h interface to communicate up the stack. The structure sizes it uses are slightly different than the SCSI and ATA structure sizes.
sys/sys/ata.h: Add many bit and structure definitions for ZAC, NCQ, and EPC command support.
sys/sys/bio.h: Convert the bio_cmd field to a straight enumeration. This will yield more space for additional commands in the future. After change r297955 and other related changes, this is now possible. Converting to an enumeration will also prevent use as a bitmask in the future.
sys/sys/disk.h: Define the DIOCZONECMD ioctl.
sys/sys/disk_zone.h: Add a new API for managing zoned disks. This is very close to the SCSI ZBC and ATA ZAC standards, but uses integers in native byte order instead of big endian (SCSI) or little endian (ATA) byte arrays.
This is intended to offer to the complete feature set of the ZBC and ZAC disk management without requiring the application developer to include SCSI or ATA headers. We also use one set of headers for ioctl consumers and kernel bio-level consumers.
sys/sys/param.h: Bump __FreeBSD_version for sys/bio.h command changes, and inclusion of SMR support.
usr.sbin/Makefile: Add the zonectl utility.
usr.sbin/diskinfo/diskinfo.c Add disk zoning capability to the 'diskinfo -v' output.
usr.sbin/zonectl/Makefile: Add zonectl makefile.
usr.sbin/zonectl/zonectl.8 zonectl(8) man page.
usr.sbin/zonectl/zonectl.c The zonectl(8) utility. This allows managing SCSI or ATA zoned disks via the disk_zone.h API. You can report zones, reset write pointers, get parameters, etc.
Sponsored by: Spectra Logic Differential Revision: https://reviews.freebsd.org/D6147 Reviewed by: wblock (documentation)
|
#
276835 |
|
08-Jan-2015 |
ken |
Improve camcontrol(8) handling of drive defect data.
This includes a new summary mode (-s) for camcontrol defects that quickly tells the user the most important thing: how many defects are in the requested list. The actual location of the defects is less important.
Modern drives frequently have more than the 8191 defects that can be reported by the READ DEFECT DATA (10) command. If they don't have that many grown defects, they certainly have more than 8191 defects in the primary (i.e. factory) defect list.
The READ DEFECT DATA (12) command allows for longer parameter lists, as well as indexing into the list of defects, and so allows reporting many more defects.
This has been tested with HGST drives and Seagate drives, but does not fully work with Seagate drives. Once I have a Seagate spec I may be able to determine whether it is possible to make it work with Seagate drives.
scsi_da.h: Add a definition for the new long block defect format.
Add bit and mask definitions for the new extended physical sector and bytes from index defect formats.
Add a prototype for the new scsi_read_defects() CDB building function.
scsi_da.c: Add a new scsi_read_defects() CDB building function. camcontrol(8) was previously composing CDBs manually. This is long overdue.
camcontrol.c: Revamp the camcontrol defects subcommand. We now go through multiple stages in trying to get defect data off the drive while avoiding various drive firmware quirks.
We start off by requesting the defect header with the 10 byte command. If we're in summary mode (-s) and the drive reports fewer defects than can be represented in the 10 byte header, we're done. Otherwise, we know that we need to issue the 12 byte command if the drive reports the maximum number of defects.
If we're in summary mode, we're done if we get a good response back when asking for the 12 byte header.
If the user has asked for the full list, then we use the address descriptor index field in the 12 byte CDB to step through the list in 64K chunks. 64K is small enough to work with most any ancient or modern SCSI controller.
Add support for printing the new long block defect format, as well as the extended physical sector and bytes from index formats. I don't have any drives that support the new formats.
Add a hexadecimal output format that can be turned on with -X.
Add a quiet mode (-q) that can be turned on with the summary mode (-s) to just print out a number.
Revamp the error detection and recovery code for the defects command to work with HGST drives.
Call the new scsi_read_defects() CDB building function instead of rolling the CDB ourselves.
Pay attention to the residual from the defect list request when printing it out, so we don't run off the end of the list.
Use the new scsi_nv library routines to convert from strings to numbers and back.
camcontrol.8: Document the new defect formats (longblock, extbfi, extphys) and command line options (-q, -s, -S and -X) for the defects subcommand.
Explain a little more about what drives generally do and don't support.
Sponsored by: Spectra Logic MFC after: 1 week
|
#
229997 |
|
11-Jan-2012 |
ken |
Add the CAM Target Layer (CTL).
CTL is a disk and processor device emulation subsystem originally written for Copan Systems under Linux starting in 2003. It has been shipping in Copan (now SGI) products since 2005.
It was ported to FreeBSD in 2008, and thanks to an agreement between SGI (who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is available under a BSD-style license. The intent behind the agreement was that Spectra would work to get CTL into the FreeBSD tree.
Some CTL features:
- Disk and processor device emulation. - Tagged queueing - SCSI task attribute support (ordered, head of queue, simple tags) - SCSI implicit command ordering support. (e.g. if a read follows a mode select, the read will be blocked until the mode select completes.) - Full task management support (abort, LUN reset, target reset, etc.) - Support for multiple ports - Support for multiple simultaneous initiators - Support for multiple simultaneous backing stores - Persistent reservation support - Mode sense/select support - Error injection support - High Availability support (1) - All I/O handled in-kernel, no userland context switch overhead.
(1) HA Support is just an API stub, and needs much more to be fully functional.
ctl.c: The core of CTL. Command handlers and processing, character driver, and HA support are here.
ctl.h: Basic function declarations and data structures.
ctl_backend.c, ctl_backend.h: The basic CTL backend API.
ctl_backend_block.c, ctl_backend_block.h: The block and file backend. This allows for using a disk or a file as the backing store for a LUN. Multiple threads are started to do I/O to the backing device, primarily because the VFS API requires that to get any concurrency.
ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a small amount of memory to act as a source and sink for reads and writes from an initiator. Therefore it cannot be used for any real data, but it can be used to test for throughput. It can also be used to test initiators' support for extremely large LUNs.
ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes, and command handler functions defined for supported opcodes.
ctl_debug.h: Debugging support.
ctl_error.c, ctl_error.h: CTL-specific wrappers around the CAM sense building functions.
ctl_frontend.c, ctl_frontend.h: These files define the basic CTL frontend port API.
ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM. This frontend allows for using CTL without any target-capable hardware. So any LUNs you create in CTL are visible in CAM via this port.
ctl_frontend_internal.c, ctl_frontend_internal.h: This is a frontend port written for Copan to do some system-specific tasks that required sending commands into CTL from inside the kernel. This isn't entirely relevant to FreeBSD in general, but can perhaps be repurposed.
ctl_ha.h: This is a stubbed-out High Availability API. Much more is needed for full HA support. See the comments in the header and the description of what is needed in the README.ctl.txt file for more details.
ctl_io.h: This defines most of the core CTL I/O structures. union ctl_io is conceptually very similar to CAM's union ccb.
ctl_ioctl.h: This defines all ioctls available through the CTL character device, and the data structures needed for those ioctls.
ctl_mem_pool.c, ctl_mem_pool.h: Generic memory pool implementation used by the internal frontend.
ctl_private.h: Private data structres (e.g. CTL softc) and function prototypes. This also includes the SCSI vendor and product names used by CTL.
ctl_scsi_all.c, ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.
ctl_ser_table.c: Command serialization table. This defines what happens when one type of command is followed by another type of command.
ctl_util.c, ctl_util.h: CTL utility functions, primarily designed to be used from userland. See ctladm for the primary consumer of these functions. These include CDB building functions.
scsi_ctl.c: CAM target peripheral driver and CTL frontend port. This is the path into CTL for commands from target-capable hardware/SIMs.
README.ctl.txt: CTL code features, roadmap, to-do list.
usr.sbin/Makefile: Add ctladm.
ctladm/Makefile, ctladm/ctladm.8, ctladm/ctladm.c, ctladm/ctladm.h, ctladm/util.c: ctladm(8) is the CTL management utility. It fills a role similar to camcontrol(8). It allow configuring LUNs, issuing commands, injecting errors and various other control functions.
usr.bin/Makefile: Add ctlstat.
ctlstat/Makefile ctlstat/ctlstat.8, ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8). It reports I/O statistics for CTL.
sys/conf/files: Add CTL files.
sys/conf/NOTES: Add device ctl.
sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB length field is now 2 bytes long.
Add several mode page definitions for CTL.
sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.
sys/dev/ciss/ciss.c, sys/dev/ata/atapi-cam.c, sys/cam/scsi/scsi_targ_bh.c, scsi_target/scsi_cmds.c, mlxcontrol/interface.c: Update for 2 byte inquiry length field.
scsi_da.h: Add versions of the format and rigid disk pages that are in a more reasonable format for CTL.
amd64/conf/GENERIC, i386/conf/GENERIC, ia64/conf/GENERIC, sparc64/conf/GENERIC: Add device ctl.
i386/conf/PAE: The CTL frontend SIM at least does not compile cleanly on PAE.
Sponsored by: Copan Systems, SGI and Spectra Logic MFC after: 1 month
|