Pawel Wojcik's Weblog Pawel Wojcik's Weblog

In previous parts of the overview I have described the general Solaris SATA architecture and the interface betwen sata module and HBA drivers. Let's see how the sata module and HBA drivers initialize themselves and register with the scsi layer.

SATA HBA drivers are linked with the '-n' option specifying that the sata module have to be loaded before the sata HBA driver. So when the sata HBA driver is identified as a proper driver for a given SATA controller type, the sata module is loaded and initialized before the driver _init and attach routine is called. From _init function sata HBA driver calls sata_hba_init() routine, passing a pointer to its modldrv structure. Sata module follows up with calling scsi_hba_init(), passing up the pointer to the HBA modldrv structure, but replacing the driver's devo_cb_ops with it's own devo_cb_ops structure, allowing it to capture open, close and ioctl calls directed to the driver instance. Scsi_hba_init routine registers the sata hba driver/sata module instance combination as an instance of scsi HBA driver. Note, that this calling sequence is executed only once for a sata hba driver, regardless of a number of HBA instances that will be controlled by the HBA driver. The Solaris kernel will call HBA driver attach routine once for each instance of the sata controller.

SATA HBA driver actions:

Each sata HBA driver has to do following in its attach function (simplified list):

• allocate and initialize data structures specific for this controller instance;
• allocate/initialize all mutexes needed for multithreaded operation;
• map any controller registers;
• allocate DMA resources necessary for controller operation;
• detect suported interrupt type and register interrupts with the system;
• perform sata controller's initialization, including PHY setup (if necessary).

At this point the controller and the HBA driver has to be ready to perform any operation asked for by the sata module, including sending commands to an atached SATA device. The final step, from the sata HBA driver point of view, is to call sata_hba_attach() function, passing to the sata module the sata_hba_tran structure (described in previous part of the overview). The HBA-specific part of the initialization is completed and the rest of the initilization is now driven by the sata module from within the sata_hba_attach(0 function.

SATA module actions:

For each sata HBA instance, SATA module creates sata_hba_inst structure, to which all others instance-specific pieces of data are attached. All sata_hba_inst structures are double-linked into a list, allowing dynamic insertion and removal of new sata HBA instances.

Do you remember that sata hba instance/sata module pretends to be a scsi hba instance? For each controller instance being attached, sata module has to create a scsi_hba_tran structure for the scsi layer. Scsi_hba_tran is an equivalent of sata_hba_tran structure described before, that sata module uses to interface with a specific instance of the sata HBA. Scsi_hba_tran structure contains entry points of the scsi hba driver through which a generic scsi layer interfaces with the specific instance of the scsi hba driver. All these entry points (functions) are contained in sata module. The newly created scsi_hab_tran structure is sent to scsi layer as a parameter of scsi_hba_attach_setup() call.

After scsi_hba_tran is created and successfully registered with scsi layer, sata module links both sata and scsi structures to the newly created sata_hba_inst structure, and stores there a pointer to the HBA's devinfo. SATA HBA devinfo structure is used to identify HBA instance by all parties involved in the operation: sata hba driver, sata module and scsi layer.

Few other sata module internal initialization is performed, among others a creation of a control node used by the system for open, close and ioctl operation for this HBA instance.

Note, that up to this point sata module did not talk to sata HBA driver. Now, the interfaces between the sata module, sata hba driver instance and the rest of the systems are set-up. Sata module is ready for the most interesting part of the initialization - detecting attached SATA devices, identifying them and making them accessible for the system.

SATA ports probing and device attachment:

SATA modules knows how many SATA device ports are active on the attaching SATA controller - this information was made available via sata_tran argument (pointer to the sata_hba_tran structure) of the sata_hba_attach(). It is worthwhile to notice, that SATA controller may have more device ports than those physically available for attaching SATA devices - sata HBA driver has to report only the number of device ports where SATA device may be connected (this situation may occur when a motherboard manufacturer reduces number of available SATA ports even though an embedded controller is capable of supporting a greater number of ports). These ports are addressed in the range of '0' to 'number_of_ports minus one', so sata HBA drive may have to maitain the mapping of these port numbers to the actual physical ports. SATA module creates an internal representation of each SATA port and a connected device. I have mentioned before that the anchor point of internal sata module structures related to a HBA instance is sata_hba_inst structure. Other structures are linked to it in the 'tree' fashion, as shown in the example below:

                        sata_hba_inst (SATA controller)
                              |
      ---------------------------------------------------- .....
      | (device port)         | (device port)            | (device port)
      |                       |                          |
sata_cport_info        sata_cport_info             sata_cport_info
      |                       |                          |
sata_drive_info        sata_pmult_info              (no device)
   (device)           (port multiplier)
                              |
                  --------------------------- ....
                  |                         |
          sata_pmport_info           sata_pmport_info
	 (port multiplier's          (port multiplier's
           device  port)                device port)
                  |                         |
              (no device)             sata_drive_info
                                         (device)

Since the number of SATA device ports remains static, sata module creates an attachment point for each of the SATA port, regardless if there is a device connected to it or not. An attachment point is a 'location' where a SATA device may be connected. Within the Solaris system, it is represented by a minor node, created by the sata module. The system may monitor the state of the attachment point by issuing DEVCTL IOCTLs request for such minor node. The name of the node (as visible in /devices/'controller path'/) is based on the SATA device port number. The macro SATA_MAKE_AP_MINOR in sata.h file describes how the minor node name is composed.

Now sata module checks what is the state of the SATA device port, i.e. is the port active, what is the status of the serial link and is any device connected to the port. The sata HBA driver's entry point sata_tran_probe_port is used to request port probing.

In response, sata HBA driver has to return the state of the port ( SATA_PSTATE_PWRON, SATA_PSTATE_PWROFF, SATA_PSTATE_SHUTDOWN or SATA_PSTATE_FAILED), a content of the standard SATA port registers and a type of the attached device (SATA_DTYPE_XXXX , specified in sata_hba.h file). At this point sata module is interested only in finding whether there is any device attached and whether the attached device is a port multiplier. If there is no device connected to the port - there is nothing to do. If there is a port multiplier connected to the port, a special probing sequence is required (even though port multipliers are not supported yet). When there is any other device connected, the device itself is probed.

Serial ATA specification dictates that SATA device after the reset should provide so-called 'device signature'. It is a hex value read from one of the task file registers that identifies the type of the device. The device signature is made available to the driver by majority of SATA controllers, but unfortunately not all. To accomodate all controllers, sata module does not rely on the device type identified by the sata HBA driver through the device signature value (and passed via device_type field of the sata_device structure), but instead probes a connected device itself by issuing Device Identify command to this device (using sata_tran_start entry point). The first word of the returned Identify Device data describes the device type. ATAPI devices do not respond to this command, so if Identify Device command fails, sata module tries to send Identify Packet Device command to check if the attached device is of the ATAPI type rather than ATA type. The entire Identify Device (or Identify Packet Device) data is stored in a sata_drive_info structure. Various device-related information are later extracted from this data.

Only SATA disk devices are supported at this time. If such device is identified, sata module tries to initilialize it to a know state. What is the known state? - The initial setting of the DMA mode and read and write cache mode. Even if setting of the initial mode fails for some reason, the device is still usable, but possibly with a degraded performance. For each device fund, there is a series of messages logged in the system log, describing what was found and what are some of the device capabilities.

Last step in the port probing during the HBA attach is making a connected device accessible to the system. System accesses all devices through their corresponding device nodes, so sata module creates target device nodes for each connected and accessible disk device. The target device node name is based on a compatible property. In SATA disk case, the name of the node is "disk", unlike the parellel scsi disk target node which (for historical reasons) is based on the name of the target driver name, 'sd". The target node, to be addressed the same way as the scsi device, has to have following properties set:

• 'device-type' ("scsi"),
• 'target' (scsi target number, based in this case on controller's device port number and potentially port multiplier's device port number),
• 'lun' (always set to zero for SATA disks),
• 'compatible' (list of all compatible strings).