Charlie Chen's Weblog

How does MSI/MSI-X work - MSI-X part

While msi defines its control information directly in config space through pci capability, msi-x puts its in system addressable memory by using pci bar's.

This is the structure of msi-x, with the number in parentheses indicating number of bits :

msg control(16) nxt ptr(8) Cap ID(8)
tab offset(29)            tab BIR(3)
PBA offset(29)            PBA BIR(3)

• Cap ID is 0x11.
• msg control defines number of vectors in msi-x table.
• BIR indicates which BAR is used to define memory segment for msi-x table or msi-x PBA. 0 for BAR at 0x10, 1 0x14, and so on, and 6 and 7 are reserved. Table or PBA could use the same BAR.

Table entry is defined as:

vector control(32) msg data msg(32) upper-addr(32) msg addr(32)

• Thus one entry uses 16 bytes. Then nth entry's address can be calculated as:
  

table base address + 16*n

• msi-x data, unlike msi, couldn't be modified by the function of device. 
• vector control, whose the most least significant bit indicates if this function is masked from sending message to system. If set, the function is prohibited from sending interrupt upstream. The default valuse is 1(masked). 

PBA consists of series of 64bit qwords each bit of them is for one vector entry of msi-x table.
These fomula can be used to calculate the qword address and bit number for the pending bit for nth vector:

qword address = PBA base + (n / 64)*8
qword bit# = n % 64

msi-x table and PBA should not share memory page with other data for other use. Suggestion is that 8k bytes is a natural boundary because some architecture use 8kB as normal page size.
 

Posted at 12:44AM Dec 12, 2007 by gnunu in General  |  Comments[1]