SPARC Enterprise M-class Servers The Secrets of Olympus

In a prior posting, SBs, XSBs, and LSBs, I wrote how Sun SPARC-Enterprise M-class servers use logical system board (LSB) numbers, not physical system board (SB) numbers, to assign CPU IDs and I/O bus addresses in Solaris. One question that common arises is: In Solaris, how can you figure out what the physical system board is for a given CPU ID or I/O device path?

For I/O, the easiest method happens to be the fmtopo command. fmtopo will list all of the I/O devices, the device path, the FRU (Field Replaceable Unit) and FRU Label. Here's a snippet of the output showing the device node /devices/pci@90,600000:

    # /usr/lib/fm/fmd/fmtopo -p
    ...
    hc:///chassis=0/ioboard=0/hostbridge=0/pciexrc=0
            ASRU: dev:////pci@90,600000
            FRU: hc:///component=iou#0
            Label: iou#0
    ...

The PCI-Express root complex pci@90,600000 belongs to the logical system board (LSB) 9 (you can tell from the "90" after the @ sign). But the output of fmtopo shows that the device is actually on the physical FRU iou#0, which is part of SB#0.

Now, knowing that LSB#9 is really SB#0, one can infer that the cpuids associated with LSB#9 (i.e., cpuids 9*32 through 10*32-1, or 288-319) are also on SB#0.

So, how does fmtopo figure out how to map LSBs to SBs? Turns out that there is one memory controller per LSB, and the memory controller node has two properties of interest, board# and physical-board#. The board# is the LSB number, while the physical-board# is the SB number. Other nodes in the device tree have the board# property (I/O hostbridges, CPUs, etc), but only the memory controller node has the physical-board# property.

To see what I mean, you can use prtconf, for example:

    # /usr/sbin/prtconf -pv | grep "board#"
        physical-board#:  00000000
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009
        board#:  00000009

If you look at the full output of prtconf, you'll see that the first two lines belong to the memory controller node (pseudo-mc) with an LSB board# of 9 and an SB physical-board# of 0. The other board# properties belong to the CPUs and I/O host bridges.