SteveJay's Weblog
Sunday May 14, 2006
The 1394 Software Framework - Incoming Asynchronous Requests (Part 1 of 3) 
In this next installment of my series of entries about the Solaris 1394 Software Framework, I'm continuing with a discussion of asynchronous request handling. (Looking for details on isochronous handling? Be patient.) But this time I'm going to be talking about handling and responding to incoming asynchronous requests.
When the 1394 Framework receives a request from the bus, it must determine to which target driver the request is intended. This is made possible by requiring target drivers to first reserve ranges within the 1394 host address space. Then, when a request arrives, the Framework can look up the target driver that owns the range and processing of the request can begin.
The other side of incoming request processing, of course, is sending a response.
The Framework provides four ways for target drivers to do this:
- have all responsibility for building each response
- have the 1394 Framework respond on their behalf by using 'backing store'
- have the 1394 Framework prepare a response and send it to the target driver for modifications
- have the interface hardware automatically handle the response
IEEE 1394 Addressing Overview
IEEE 1212-1994 (also known as ISO/IEC 13213) is a specification describing the generalized command and status register arcitecture that can be used, by a variety of different (and future) protocols, to provide commonality and some interoperability.IEEE 1394 is based on IEEE 1212 and uses IEEE 1212 style 64-bit addressing. In the 64-bit address the high-order 10 bits identifying the bus number (0x3FF for the local bus) and the next 6 bits identify the node ID. Node ID 63 is reserved for broadcast, so this allows for up to 63 devices (0 - 62) on the bus. The remaining 48 bits are used to address locations on the specified device.
Looking at it another way, each node (device) can assign, partition, and manage its own 48 bits of address space. The significance of this for a particular node is in terms of received requests, being sent to it from other nodes on the bus. By ascribing meaning or functionality to certain addresses, a node can handle incoming asynchronous requests as defined in a standards specification or as otherwise planned. From the point of view of transmitting a request, it is relevant to know the meaning of address locations at the destination node.
Again according to IEEE 1212 (and used by IEEE 1394), some of the highest 48-bit addresses, those beginning with 0xFFFF_Fxxx_xxxx, are resered for use by various protocols, including IEEE 1394 itself.
Physical I/O
One of the features of IEEE 1394 bus architecture is the ability to directly access device or host memory without having to pass messages or packets via software. Protocols that rely on a lot of asynchronous I/O throughput, such as SBP-2 (Serial Bus Protocol 2) for disks, are designed to take advantage of this feature. But these protocols do occasionally need to ensure software intervention, for example notifying software of a status update.To satisfy this need, the 48-bit space is paritition into sections:
Currently, the 1394 Software Framework supports a physical address range within the low 32 bits (0x0000_0000_0000 to 0x0000_FFFF_FFFF). Received write and read requests that have a 48bit destination offset with that range can be automatically handled by hardware to or from host memory.
Posted Writes and CSR
A posted write is a received write request for which an ack_complete (instead of an ack_pending) is sent to the requesting node, indicating that the write was performed successfully before the write is actually completed. Posted writes are very efficient when they succeed because they obviate the need for sending a write response packet. Typically, post writes do eventually complete successfully, but in some circumstances the write fails and the requesting node cannot be notified. For some applications and protocols this functionality is acceptable, but for others it is not.In addition, IEEE 1212-1994 reserves a region of address space for control and status registers, some of which may be used by protocols. This space is near the highest end of the address space.
The 1394 Software Framework further divides the 48-bit address space as shown:
Address Allocation
The 1394 Software Framework uses two mechanisms for assigning a range of address space to a target driver.- t1394_alloc_addr() and t1394_free_addr() calls are used to acquire address space in the non-physical ranges.
- Physical addresses require additional setup (described later) and are assigned using the T1394_ADDR_FIXED flag in t1394_alloc_addr.
The routines for allocating and freeing 1394 address space do not themselves result in any transactions being sent over the bug and therefore are not handled like the outgoin asynchronous request commands. Instead, these routines perform the requested function and return immediately with DDI_SUCCESS or DDI_FAILURE
As with most 1394 Software Framework calls, these are called with the t1394_handle and a parameters structure as defined below.
Address Allocation Data Structures
/*
* t1394_alloc_addr_t
* is passed to t1394_alloc_addr(), when 1394 address space is being
* allocated, to describe the type of address space. The target driver
* is responsible for specifying the aa_enable, aa_type, and aa_evts
* fields described above as well as the size of the allocated block.
* Additionally, the target driver may specify backing store
* (aa_kmem_bufp), a specific address (in aa_address if aa_type is
* T1394_ADDR_FIXED), and a callback argument (in aa_arg) to be
* passed to the target in any of its callback routines.
* When it returns, t1394_alloc_addr() will return in aa_address the
* starting address of the requested block of 1394 address space and
* and address block handle (aa_hdl) used to free the address block
* in a call to t1394_free_addr().
*/
typedef struct t1394_alloc_addr {
t1394_addr_type_t aa_type; /* IN: address region */
size_t aa_length; /* IN: # bytes requested */
t1394_addr_enable_t aa_enable; /* IN: request enables */
t1394_addr_evts_t aa_evts; /* IN: event callbacks */
opaque_t aa_arg; /* IN: evt callback arg */
caddr_t aa_kmem_bufp; /* IN: backing-store buf */
uint64_t aa_address; /* IN/OUT: alloced address */
t1394_addr_handle_t aa_hdl; /* OUT: returned to target */
} t1394_alloc_addr_t;
- aa_type - The target driver provides aa_type to specify the address range from which the allocation is requested. (See 'Address Type' below for details)
- aa_length - The target driver provides aa_length, which is the size in bytes of the requested allocation.
- aa_enable - The target driver provides aa_enable to enable read, write and/or lock requests for the allocated address region. (See 'Address Access Enable' below for details)
- aa_evts - The target driver provides aa_evtws to specify callback entrypoints for the 1394 Software Framework to call when receiving requests to the allocated range. (See 'Address Event Callbacks' below for details)
- aa_arg - This is the target driver argument to be passed back to the target driver when the 1394 Software Framework invokes an event callback. The event callbacks receive a cmd_1394_cmd_t structure containing information about the received request. In addition, the 1394 Software Framework fills in the command's cmd_callback_arg field with this aa_arg for this address space.
- aa_kmem_bufp - This argument is provided by the target driver.
If this argument is a non-NULL address, then the 1394 Software Framework
will record this kernel virtual address in the 1394 address allocation
stuctures and use the address to provide backing-store functionality for
requests to the range. The memory pointed to by aa_kmem_bufp must
be at least aa_length bytes long. This backing-store capability
means that the 1394 Software Framework will automatically process reads,
writes, and locks to the assigned address range by offseting accordingly
into the provided kernel buffer and building the appropriate response.
However, even with backing-store enabledm the target driver may still
choose to be notified of request arrival events. If configured to be
notified of received requests (see aa_evts below) and aa_kmem_bufp
is not NULL, then the 1394 Software Framework builds the command response
based on the aa_kmem_bufp backing store, and then calls the event
callback. In the target driver's callback routine, it then has the opportunity
to alter the prepared response before the response is transmitted.
Target drivers specify NULL for aa_kmem_bufp if they do not want the 1394 Software Framework to perform backing-store for the allocated range. - aa_address - The aa_address field is the lowest IEEE 1394
address of the allocated range. So the specified range will be allocated
from aa_address to aa_address + (aa_length-1). Only the
lower 48 bits are valid. The upper 16 bits are always zero (0).
If the aa_type is T1394_ADDR_NORMAL, T1394_ADDR_POSTED_WRITE, or T1394_ADDR_CSR, then the 1394 Software Framework will fill in the aa_address field to return the target driver.
If the aa_type is T1394_ADDR_FIXED, then the target driver provides the address to designate to the 1394 Software Framework the specific address region to reserve. - aa_hdl - This is an opaque handle returned to the target driver to track the allocated address range. The 1394 Software Framework returns a different aa_hdl for each allocation. The target driver must provide the aa_hdl when calling t1394_free_addr().
Address Type
Target drivers select the address space type for the allocation by selecting one of the address type choices below.
/*
* t1394_addr_type_t
* is used in the t1394_alloc_addr_t structure, passed to
* t1394_alloc_addr(), to indicate what type of address block the
* target driver would like to allocate.
* T1394_ADDR_POSTED_WRITE indicates posted write memory, where
* incoming write requests are automatically acknowledged as complete.
* T1394_ADDR_NORMAL indicates memory, unlike the posted write area,
* where all requests regardless of type are ack_pended upon receipt
* and are subsequently responded to.
* T1394_ADDR_CSR memory range is generally used by target drivers
* that are implementing a well-defined protocol.
* And T1394_ADDR_FIXED is used to indicate to t1394_alloc_addr()
* that a specific set of addresses are needed. Unlike the other three
* types, this type of request is used to choose a specific address or
* range of addresses in 1394 address space.
*/
typedef enum {
T1394_ADDR_POSTED_WRITE = 0,
T1394_ADDR_NORMAL = 1,
T1394_ADDR_CSR = 2,
T1394_ADDR_FIXED = 3
} t1394_addr_type_t;
- T1394_ADDR_POSTED_WRITE - The 1394 Software Framework allocates from the local 1394 address space within the range 0x0001_0000_0000 to 0xFFFE_FFFF_FFFF. The hardware automatically sends ack_complete for incoming write requests to this region, and therefore the framework and target driver do not send write responses. The hardware will automatically send ack_pending for incoming read and lock requests to this region, and therefore the framework and target driver do send read and lock responses.
- T1394_ADDR_NORMAL - The 1394 Software Framework allocates from the local 1394 address space within the range 0xFFFF_0000_0000 to 0xFFFF_EFFF_FFFF. The hardware will automatically send ack_pending for all incoming read, write, and lock requests to this region, and therefore the framework and target driver do send read, write, and lock responses.
- T1394_ADDR_CSR - The 1394 Software Framework allocates from the local 1394 address space within the range 0xFFFF_F001_0000 to 0xFFFF_FFFF_FFFF. The hardware will automatically send ack_pending for all incoming read, write, and lock requests to this region. This region is for use by protocol modules that must have addresses in the reserved CSR range.
- T1394_ADDR_FIXED - There are three (3) situations in which a target
driver might use this type:
Physical Range Allocation - a target driver specifies the address of the physical range that has been allocated and bound using the Solaris DDI Framework's DMA routines.
Specific Address Allocation - a target driver specifies the actual address it is requesting from the 1394 Software Framework. This option is for use by protocol modules which require the use of specific "hardcoded" addresses.
CSR Address Allocation - a target driver specifies the actual address it is requesting from the 1394 Software Framework within the range 0xFFFF_F000_0000 to 0xFFFF_F000_FFFF. Some protocols have reserved fields in this range.
Address Access Enables
When allocating local 1394 address space, target drivers specify which request types should be allowed in that space. If a request is received for an address range which, by the setting of these flags, does not permit the request type, then the 1394 Software Framework sends the appropriate response packet with a response code of resp_type_error. See IEEE 1394-1995 Section 9.5 for response code information.Target drivers may specify any combination of these access enables when allocating local 1394 address space.
/*
* t1394_addr_enable_t
* is used in the t1394_alloc_addr_t structure, passed to
* t1394_alloc_addr(), to indicate what types of (incoming)
* asynchronous requests will be allowed in a given address block.
* If, for example, an address block is intended to be read-only,
* then only the T1394_ADDR_RDENBL bit should be enabled at allocation
* time. Then, when incoming requests of an inappropriate type (write
* or lock requests, in this case) arrive, the 1394 Framework can
* automatically respond to them with TYPE_ERROR in the response
* without having to notify the target driver.
*/
typedef enum {
T1394_ADDR_RDENBL = (1 << 0),
T1394_ADDR_WRENBL = (1 << 1),
T1394_ADDR_LKENBL = (1 << 2)
} t1394_addr_enable_t;
- T1394_ADDR_RDENBL - If the target driver sets this flag, it enables incoming read requests (block and quadlet) to the allocated address range.
- T1394_ADDR_WRENBL - If the target driver sets this flag, it enables incoming write requests (block and quadlet) to the allocated address range.
- T1394_ADDR_LKENBL - If the target driver sets this flag, it enables incoming lock requests (all sizes) to the allocated address range.
Address Event Callbacks
To be notified when a read, write, or lock request is received for the allocated range, a target driver sets the address event callbacks shown below. If it does not want to be notified, it sets the corresponding entry point to NULL./*
* t1394_addr_evts_t
* is used in the t1394_alloc_addr_t structure, passed to
* t1394_alloc_addr(), to specify callback routines for the
* allocated address block. When a request of the appropriate type
* (read/write/lock) is received to a target driver's address
* block, the appropriate callback routine is consulted and if it is
* non-NULL it is called and passed a cmd1394_cmd_t structure used to
* describe the incoming asynch request.
*/
typedef struct t1394_addr_evts {
void (*recv_read_request)(cmd1394_cmd_t *req);
void (*recv_write_request)(cmd1394_cmd_t *req);
void (*recv_lock_request)(cmd1394_cmd_t *req);
} t1394_addr_evts_t;
- recv_read_request()
- recv_write_request()
- recv_lock_request() - When the 1394 Software Framework invokes an address event callback, it supplies a command containing information about the request. Note that the command structure is the same as that used for outgoing asynchronous requests. However, for incoming asynchronous requests the structre fields are used somewat differently. This will be described is a subsequent entry.